config.json, deepseek-ai/DeepSeek-V3

Fields

id

sources/arch-hf-config-deepseek-v3

URL

https://huggingface.co/deepseek-ai/DeepSeek-V3/raw/main/config.json

Publisher

Hugging Face

Kind

repo

Accessed

2026-10-03

notes

Authoritative architecture config read 2026-10-03: 61 layers, hidden 7168, intermediate 18432, num_attention_heads 128, qk_nope_head_dim 128, qk_rope_head_dim 64, v_head_dim 128, kv_lora_rank 512, q_lora_rank 1536, n_routed_experts 256, n_shared_experts 1, num_experts_per_tok 8, n_group 8, topk_group 4, first_k_dense_replace 3, max_position_embeddings 163840. Used for the MLA KV-cache arithmetic in this repo.

Notes

Authoritative architecture config read 2026-10-03: 61 layers, hidden 7168, intermediate 18432, num_attention_heads 128, qk_nope_head_dim 128, qk_rope_head_dim 64, v_head_dim 128, kv_lora_rank 512, q_lora_rank 1536, n_routed_experts 256, n_shared_experts 1, num_experts_per_tok 8, n_group 8, topk_group 4, first_k_dense_replace 3, max_position_embeddings 163840. Used for the MLA KV-cache arithmetic in this repo.

Cross-references

This record references no other records.