Darwin-180B-RSI

ADDED 2026-10-18. Full-parameter fine-tune (the card calls it a self-improvement round) of Qwen/Qwen3.8-Flash-Next by VIDRAFT, published as FINAL-Bench/Darwin-180B-RSI. RUNNABLE WITH EXPERT OFFLOAD: same shape as the reference model. HF API safetensors total is 179,999,981,424 BF16, so params_b is 180.0, and the base card splits that into a 125B transformer body with 6B activated plus a 51B n-gram embedding table and 4B MTP heads. Routing is unchanged from the base: 512 experts, 10 routed per token, plus one shared expert (shared_expert_intermediate_size 640, moe_intermediate_size 640), so the per-token expert traffic stays the small slice that makes host-RAM offload viable. The 51B n-gram table is not per-token traffic and can stay on disk, exactly as on models/qwen3-8-flash-next. WHY IT IS HERE: it beats the current best recorded score on three ranked benchmarks, all vendor claims from the .eval_results YAML files on the model repo (dated 2026-09-28, bf16, temperature 1.0, top_p 0.95, top_k 20, thinking budget 131072). GPQA Diamond 94.44 (all 198 items, majority vote over up to 16 samples) beats models/veriloop-e2 at 93.9, the highest GPQA Diamond in data/models. AIME 2026 100.0 (all 30 problems, maj@16; the same file records mean accuracy 98.75, which also clears it) beats models/veriloop-e2 at 98.3. MMLU-Pro 88.12 (single sample, all 12032 items) is recorded but is NOT a qualifying win: models/intern-s2-397b is higher at 89.77. The qualifying wins are GPQA Diamond over models/veriloop-e2 and AIME 2026 over models/veriloop-e2. The base model models/qwen3-8-flash-next carries no benchmark numbers, so there is no within-family comparison to make. Not a quantization and not an abliteration: the repo is 131 BF16 safetensors shards and the card describes full-parameter continued training. GEOMETRY from config.json fetched 2026-10-18: identical to models/qwen3-8-flash-next except the model_type string, which is qwen4_exp / qwen4_exp_text with architectures Qwen4ExpForConditionalGeneration. hidden 2560, 48 layers, 24 query heads, 2 kv heads, head_dim 256, gqa_ratio 12. layer_types is 36 linear_attention + 12 full_attention at indices 3,7,11,15,19,23,27,31,35,39,43,47, full_attention_interval 4. Linear block: 16 key heads, 48 value heads, head dim 128, conv kernel 4. KV (bf16, full-attention layers only): 12 x 2 kv heads x 256 x 2 (K and V) x 2 bytes = 24,576 B/token. The 36 linear-attention layers hold a fixed recurrent state, not a growing cache, so they are not in kv_cache_bytes_per_token. RoPE is native mrope: rope_type default, rope_theta 10000000, partial_rotary_factor 0.25, mrope_interleaved true, mrope_section [11,11,10], no rope_scaling. context_scaling_method is mrope. flops_per_token_active is 2 x 6B = 1.2e10, carried over from the base because the active footprint did not change. License is the Qwen Community License 1.0 (bespoke, not OSI), the same license as the base, confirmed from the LICENSE file in the repo. Engines: the card points at its own llama.cpp fork (llama.cpp-darwin) and ships a 4-bit GGUF sibling, POCKET-Darwin-180B-GGUF; stock engine support is whatever already serves Qwen3.8-Flash-Next, since the weight layout matches.