Ling-3.1-flash

ADDED 2026-10-18. Open-weight hybrid-linear MoE, the successor to models/ling-3-0-flash. RUNNABLE WITH EXPERT OFFLOAD: 560B total but the card's 25B activated per token, and only 8 of 512 routed experts plus one shared expert, so the per-token expert traffic is a small fraction of the checkpoint. That is the same shape as the reference Qwen3.8-Flash-Next (a few percent of the weights active, a handful of experts per token) and is why it stays fast with experts in RAM. A dense model of this size would decode at RAM bandwidth once it spills out of VRAM; this one does not. PARAMS: the card says 560B total and 25B activated, and those are the recorded figures; active_params_b is the vendor number, not a count derived here. The checkpoint is FP8 (quantization_config fmt e4m3, activation_scheme dynamic, lm_head and embeddings left out), so the stored weights are about half a BF16 dump. flops_per_token_active = 2 x 25e9 = 5.0e10 uses the vendor active count. GEOMETRY from config.json fetched 2026-10-18: hidden 4096, 72 layers, layer_group_size 8, first 4 layers dense (first_k_dense_replace 4), 512 routed experts, 8 per token, 1 shared expert, moe_intermediate_size 1280. MLA on the attention layers: kv_lora_rank 512, qk_rope_head_dim 64, qk_nope_head_dim 128, v_head_dim 128, 64 attention heads, q_lora_rank 1536. num_kv_heads and gqa_ratio are null because the attention layers are MLA, not GQA. LAYER SPLIT from modeling_bailing_moe_v3.py, the same rule as models/ling-3-0-flash: attention iff (layer_idx+1) % layer_group_size == 0 or layer_idx >= num_hidden_layers // layer_group_size * layer_group_size, else linear_attention. Over 72 layers with group size 8 that is attention at 0-based indices [7, 15, 23, 31, 39, 47, 55, 63, 71] = 9 layers, and the other 63 are KDA layers with a fixed recurrent state. KV (bf16), growing term only: 9 x (512 + 64) x 2 bytes = 10,368 B/token. The card's README says 63 KDA + 9 gated MLA, which matches this count. max_position_embeddings is 262144 and rope_scaling is null, so context_scaling_method is native; the card's '1M-token context window' is not in config.json and is not recorded as the context length. silu, vocab 157184, tie_word_embeddings false, rope_theta 6000000. BENCHMARKS ARE VENDOR CLAIMS. The README names the benchmarks but prints no numbers; the scores come from the HF eval-results widget, which renders an open PR (discussions/6, commit 49e0e59a, .eval_results/Ling-3.1-flash.yaml, dated 2026-10-13) that is not merged to main, and every row is verified=false. BEATS models/beam-1-0-501b-a23b, the catalog's best published SWE-bench Pro number, on SWE-bench Pro: 68.74 vs 66.1. Beam is a 501B / 23.4B-active model routing 8 of 256 experts; this one does it with 25B active out of 560B routing 8 of 512. The same YAML also reports SWE-bench Multilingual resolved 88.5, Terminal-Bench 2.1 81.18 and Terminal-Bench 4.0 46.03. The Terminal-Bench 2.1 figure does NOT beat models/mimo-v2-6-flash-rl (87.6) or the 85.80 on models/qwen3-8-flash-next, so the qualifying comparison is SWE-bench Pro against beam-1-0-501b-a23b. The SWE-bench Pro and Terminal-Bench 4.0 rows note the LingCode harness, so the comparison is across harnesses.