DeepSeek-V4-Pro config.json · inference-research

Fields

id

sources/mbl-cfg-deepseek-v4-pro

URL

https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro/raw/main/config.json

Publisher

Hugging Face

Kind

repo

Accessed

2026-10-04

notes

deepseek_v4, DeepseekV4ForCausalLM. num_hidden_layers 61, hidden_size 7168, num_attention_heads 128, num_key_value_heads 1, head_dim 512, qk_rope_head_dim 64, q_lora_rank 1536, index_topk 1024, sliding_window 128, num_hash_layers 3, o_groups 16. MoE: n_routed_experts 384, n_shared_experts 1, num_experts_per_tok 6, moe_intermediate_size 3072. rope_scaling {"type":"yarn","factor":16,"original_max_position_embeddings":65536}. max_position_embeddings 1048576. NO kv_lora_rank field. Read raw 2026-10-04.

Notes

deepseek_v4, DeepseekV4ForCausalLM. num_hidden_layers 61, hidden_size 7168, num_attention_heads 128, num_key_value_heads 1, head_dim 512, qk_rope_head_dim 64, q_lora_rank 1536, index_topk 1024, sliding_window 128, num_hash_layers 3, o_groups 16. MoE: n_routed_experts 384, n_shared_experts 1, num_experts_per_tok 6, moe_intermediate_size 3072. rope_scaling {"type":"yarn","factor":16,"original_max_position_embeddings":65536}. max_position_embeddings 1048576. NO kv_lora_rank field. Read raw 2026-10-04.

Cross-references

This record references no other records.