FINAL-Bench/Darwin-180B-RSI config.json

Raw config.json fetched 2026-10-18. Top-level model_type qwen4_exp, architectures Qwen4ExpForConditionalGeneration, text_config model_type qwen4_exp_text. hidden_size 2560, 48 layers, 24 query heads, 2 kv heads, head_dim 256. MoE: num_experts 512, num_experts_per_tok 10, moe_intermediate_size 640, shared_expert_intermediate_size 640. layer_types is 36 linear_attention plus 12 full_attention at indices 3,7,11,15,19,23,27,31,35,39,43,47 (full_attention_interval 4). Linear block: linear_num_key_heads 16, linear_num_value_heads 48, key and value head dim 128, linear_conv_kernel_dim 4. vocab 248320, tie_word_embeddings false, max_position_embeddings 262144, silu. rope_parameters: rope_type default, rope_theta 10000000, partial_rotary_factor 0.25, mrope_interleaved true, mrope_section [11,11,10]. mtp_num_hidden_layers 1. Same geometry as Qwen/Qwen3.8-Flash-Next; the model_type string changed from qwen3_5 to qwen4_exp.