DeepSeek-V3 config.json (huggingface.co/deepseek-ai/DeepSeek-V3)

Authoritative architecture config for DeepSeek-V3: hidden_size 7168, 61 layers, num_attention_heads 128, kv_lora_rank 512, q_lora_rank 1536, qk_nope_head_dim 128, qk_rope_head_dim 64, v_head_dim 128, n_routed_experts 256, n_shared_experts 1, num_experts_per_tok 8, first_k_dense_replace 3, intermediate_size 18432, moe_intermediate_size 2048, max_position_embeddings 163840, rope_theta 10000, rope_scaling type yarn factor 40. Note num_key_value_heads is 128 in this config because MLA expands latents at runtime; the resident cache is the 512-rank latent, not 128 heads.