MiMo-V2.6-Flash-RL

ADDED 2026-10-18. Open-weight sparse MoE, the efficiency checkpoint of the MiMo-V2.6 series, native omnimodal (text, image, video, audio) with a speculative decoder the card ships as a multi-token prediction head (config num_nextn_predict_layers 3). RUNNABLE WITH EXPERT OFFLOAD: 309B total but only the card's 15B activated per token, and only 8 of 256 experts routed with no shared expert, so the per-token expert traffic stays a small fraction of the checkpoint. That is the same shape as the reference Qwen3.8-Flash-Next (a few percent of the weights active, few experts per token) and is why it stays fast with experts in RAM. The card's own recipe serves it on 4x H200 with expert parallelism (vLLM --tensor-parallel-size 4 --enable-expert-parallel, reasoning-parser mimo). PARAMS: the card says 309B total and 15B activated, and those are the recorded figures; active_params_b is the vendor number, not a count derived here. HF API safetensors metadata sums to 310,756,322,688 parameters, of which U8 is 302,795,194,368 and the rest is BF16 4,101,308,032 + F8_E4M3 3,859,808,256 + F32 12,032. The U8 block is the FP8-quantized expert weights (quantization_config fmt e4m3, activation_scheme dynamic), so the stored checkpoint is far smaller than a 309B BF16 dump. flops_per_token_active = 2 x 15e9 = 3.0e10 is the schema one-pass approximation off the vendor active count. GEOMETRY from config.json fetched 2026-10-18: hidden 4096, 48 layers, global attention 64 query heads / 4 kv heads / head_dim 192 / v_head_dim 128 so gqa_ratio 16 on the global layers, sliding attention 64 query heads / 8 kv heads / head_dim 192 / swa_v_head_dim 128 with sliding_window 128. hybrid_layer_pattern is 9 global (value 0) and 39 sliding (value 1); global layers sit at indices 0, 5, 11, 17, 23, 29, 35, 41, 47. MoE: n_routed_experts 256, num_experts_per_tok 8, n_shared_experts null so shared_expert is false, moe_intermediate_size 2048, and moe_layer_freq is 0 then 47 ones so only layer 0 has a dense FFN. silu, vocab 152576, tie_word_embeddings false, partial_rotary_factor 0.334, rope_theta 10000000, rope_type default, no rope_scaling key, so context_scaling_method is native. KV (bf16), growing term only: the 9 global layers cache K at head_dim 192 and V at v_head_dim 128, so 9 x 4 kv heads x (192 + 128) x 2 bytes = 23,040 B/token. The 39 sliding layers hold a fixed 128-token window (8 kv heads, K 192 + V 128), 39 x 8 x 320 x 2 x 128 = 24.4 MB that does not grow with context, so kv_cache_bytes_per_token records the unbounded term only. If all 48 layers cached the full context at the global geometry it would be 122,880 B/token, about 5x the recorded number. BENCHMARKS ARE VENDOR CLAIMS. BEATS models/qwen3-8-27b, the catalog's best published Terminal-Bench 2.1 number, on Terminal-Bench 2.1: 87.6 vs 73.0. qwen3-8-27b is a 27.8B dense model; this one does the same job with 15B active out of 309B. The HF eval-results widget repeats the 87.6 on harborframework/terminal-bench-2.1. The card also lists SWE-bench Pro 69.4, SWE-bench Verified 79.6 and GPQA Diamond 90.1 in the same table, all vendor claims; the SWE-bench Verified number also beats models/xing4-0-29b-a4b (75.00), but that model is a 4B-active class and the qualifying comparison here is Terminal-Bench 2.1 against qwen3-8-27b.