XiaomiMiMo/MiMo-V2.6-Pro-RL config.json

Fetched 2026-10-18 from XiaomiMiMo/MiMo-V2.6-Pro-RL config.json (main). model_type mimo_v2, architectures MiMoV2ForCausalLM. hidden_size 6144, num_hidden_layers 70, num_attention_heads 128, num_key_value_heads 8, head_dim 192, v_head_dim 128, swa_num_attention_heads 128, swa_num_key_value_heads 8, swa_head_dim 192, swa_v_head_dim 128, sliding_window 128. hybrid_layer_pattern length 70 with 10 global (value 0) at indices 0,7,15,23,31,39,47,55,62,69 and 60 sliding (value 1). n_routed_experts 384, num_experts_per_tok 8, n_shared_experts null, moe_intermediate_size 2048, moe_layer_freq is 0 then 69 ones so only layer 0 is a dense FFN. max_position_embeddings 1048576, rope_parameters rope_type default, rope_theta 10000000, partial_rotary_factor 0.334, no rope_scaling. vocab 152576, hidden_act silu, tie_word_embeddings false. quantization_config: fp8 e4m3, activation_scheme dynamic.