config.json, moonshotai/Kimi-Linear-48B-A3B-Instruct

Fields

id

sources/arch-hf-config-kimi-linear

URL

https://huggingface.co/moonshotai/Kimi-Linear-48B-A3B-Instruct/raw/main/config.json

Publisher

Hugging Face

Kind

repo

Accessed

2026-10-03

notes

27 layers; linear_attn_config.kda_layers lists 20 KDA layers and full_attn_layers lists [4, 8, 12, 16, 20, 24, 27] = 7 full attention layers, confirming the 3:1 ratio in the paper. kv_lora_rank 512, qk_rope_head_dim 64, head_dim 72, num_experts 256, num_experts_per_token 8.

Notes

27 layers; linear_attn_config.kda_layers lists 20 KDA layers and full_attn_layers lists [4, 8, 12, 16, 20, 24, 27] = 7 full attention layers, confirming the 3:1 ratio in the paper. kv_lora_rank 512, qk_rope_head_dim 64, head_dim 72, num_experts 256, num_experts_per_token 8.

Cross-references

This record references no other records.