DeepSeek-V4-Flash config.json

deepseek_v4, DeepseekV4ForCausalLM. num_hidden_layers 43, hidden_size 4096, num_attention_heads 64, num_key_value_heads 1, head_dim 512, qk_rope_head_dim 64, q_lora_rank 1024, index_head_dim 128 / index_n_heads 64 / index_topk 512, sliding_window 128, num_hash_layers 3, compress_rope_theta 160000, compress_ratios per-layer [0,0,4,128,...] (1.0 == no compression, 128 == full), o_groups 8, o_lora_rank 1024. MoE: n_routed_experts 256, n_shared_experts 1, num_experts_per_tok 6, topk_method noaux_tc, expert_dtype fp4, moe_intermediate_size 2048. rope_scaling {"type":"yarn","factor":16,"original_max_position_embeddings":65536,"beta_fast":32,"beta_slow":1}, rope_theta 10000. max_position_embeddings 1048576. NO kv_lora_rank field. Read raw 2026-10-04.