Ornith-1.5-35B-A3B

ADDED 2026-10-18. Open-weight MoE in the ~3B-active class, continued pretraining and self-improvement RL on top of Ornith-1.0, which the card says was itself developed on top of Qwen3.5 and Gemma 4. The config model_type is qwen3_5_moe, so the geometry below matches the Qwen3.5-35B-A3B shape rather than a new architecture. RUNNABLE WITH EXPERT OFFLOAD: 35.952B total but only the card's ~3B activated per token, and only 8 of 256 experts routed, so the per-token expert traffic is a small slice of the 72 GB BF16 checkpoint. The card's own recipe serves the BF16 weights on one 80 GB GPU (vLLM --tensor-parallel-size 1, gpu-memory-utilization 0.90, 262144 context), and the same small active footprint is what keeps decode off RAM bandwidth when the experts spill. PARAMS: HF API safetensors total is 35,951,822,704 BF16 (index metadata total_size 71,903,645,408 bytes), so params_b is 35.952. active_params_b is the card's ~3B figure, not a count derived here. flops_per_token_active = 2 x 3.0e9 = 6.0e9 is the schema one-pass approximation. GEOMETRY from config.json fetched 2026-10-18 (commit 10fbf86fed7ecee4a061f8b499a618f46001cac1): hidden 2048, 40 layers, 16 query heads, 2 kv heads, head_dim 256, so gqa_ratio 8. MoE: 256 experts, 8 per token, moe_intermediate_size 512, and shared_expert_intermediate_size 512 so shared_expert is true. silu, vocab 248320, tie_word_embeddings false, mtp_num_hidden_layers 1. layer_types is 30 linear_attention + 10 full_attention at full_attention_interval 4 (indices 3, 7, 11, 15, 19, 23, 27, 31, 35, 39). KV (bf16): only the 10 full-attention layers grow with context, 10 x 2 kv heads x 256 x 2 (K+V) x 2 bytes = 20,480 B/token. The 30 linear-attention layers hold a fixed recurrent state, so they are not in that figure. If all 40 layers cached the full context it would be 81,920 B/token, 4x the recorded number. rope_parameters has rope_type default and no rope_scaling key, with mrope interleaved, so context_scaling_method is mrope. BENCHMARKS ARE VENDOR CLAIMS, averaged over five runs on the card. BEATS models/mellum2-1-12b-a2-5b-thinking, the catalog's best ~3B-active record with a published SWE-bench Verified number, on SWE-bench Verified: 74.2 vs 47.0. In the same class it also beats the catalog's nemotron-labs-dory1-4b (61.8) and minicpm5-2b (46.4) on that benchmark, and its GPQA Diamond 88.6 beats the catalog's best sub-10B-active figure, the 72.16 four-loop vendor claim on nemotron-labs-dory1-4b. It does NOT beat models/xing4-0-29b-a4b on SWE-bench Verified (74.2 vs 75.00) or on Terminal-Bench 2.1 (56.4 vs 57.50); those are the catalog's best in the next size up, a 4B-active model. AIME 2026 is 96.7 in the same table, against kolibri-1's 96, but Kolibri's 96 is a different card's vendor claim and is not treated as the win here. The comparison rows the card prints for Qwen3.6-35B-A3B (SWE-bench Verified 76.0, Terminal-Bench 2.1 54.4, GPQA Diamond 86.7) and Gemma-4-31B are not catalog records.