XiaomiMiMo/MiMo-V2.6-Flash-RL config.json

Fetched 2026-10-18 from resolve/main/config.json. model_type mimo_v2, architectures MiMoV2ForCausalLM. hidden_size 4096, num_hidden_layers 48, num_attention_heads 64, num_key_value_heads 4, head_dim 192, v_head_dim 128, swa_num_attention_heads 64, swa_num_key_value_heads 8, swa_head_dim 192, swa_v_head_dim 128, sliding_window 128, n_routed_experts 256, num_experts_per_tok 8, n_shared_experts null, moe_intermediate_size 2048, intermediate_size 16384, vocab_size 152576, max_position_embeddings 1048576, hidden_act silu, tie_word_embeddings false, num_nextn_predict_layers 3. hybrid_layer_pattern has 9 global (0) and 39 sliding (1) layers, global at indices 0, 5, 11, 17, 23, 29, 35, 41, 47. moe_layer_freq is 0 then 47 ones, so only layer 0 is a dense FFN. rope_parameters: rope_type default, rope_theta 10000000, partial_rotary_factor 0.334. quantization_config fmt e4m3, activation_scheme dynamic. No rope_scaling key.