Solar Open 2

ADDED 2026-10-18. Open-weight hybrid-attention MoE in the ~15B-active class. Card nameplate is 250B-A15B: 250B stored, 15B activated per token. RUNNABLE WITH EXPERT OFFLOAD: 8 of 320 routed experts plus one shared expert fire per token, and only 12 of 48 layers keep a KV cache, so both the per-token expert traffic and the long-context memory stay small relative to the 250B checkpoint. The card states the design explicitly: a hybrid stack that interleaves three linear-attention layers with one softmax-attention layer, and only those 12 softmax layers hold KV. PARAMS: the card and the HF API agree. Safetensors total is 250,287,810,304 (BF16 250,287,794,944 plus F32 15,360); the card quotes the BF16 count 250,287,794,944 as the 250B, so params_b is 250.288. active_params_b is the card 15B figure, not a count derived from the config. flops_per_token_active = 2 x 15e9 = 3.0e10 is the schema one-pass approximation off that vendor figure. BEATS models/kimi-k2-6, the catalog best open-weight record with a published LiveCodeBench v6 number, on LiveCodeBench v6: 92.4 vs 89.6. Both numbers are vendor claims and were not re-run on one harness. The Solar figure is the card English benchmark table, which also lists DeepSeek-V4-Flash at 92.3 on the same row, so 92.4 edges both that row and the catalog Kimi record. Same offload class: both are large-total MoE with a far smaller active footprint and a handful of experts routed per token, so the expert bank can sit in RAM. This one activates 15B of 250B and routes 8 of 320; Kimi-K2.6 is the larger active model the catalog currently leads with on this benchmark. The same table also reports MMLU-Pro 86.2, which does NOT beat models/qwen3-5-122b-a10b (86.7), GPQA Diamond 86.3, which does NOT beat models/kimi-k2-6 (90.5) or models/step-5-preview (93.5), AIME 2026 95.7, which does NOT beat models/beam-1-0-501b-a23b (97.7), and SWE-bench Verified 70.4, which does NOT beat models/qwen3-6-35b-a3b (76.0). LiveCodeBench v6 is the qualifying comparison. The table also prints IFBench 80.0, which is above the 79 four-loop vendor claim on models/nemotron-labs-dory1-4b but that is a sub-10B dense model on a different harness, so it is noted and not the class comparison. ARCHITECTURE from config.json fetched 2026-10-18, and the card model-overview table matches it: model_type solar_open2, architectures SolarOpen2ForCausalLM. The card says production serving needs the Upstage vLLM fork and local experimentation needs the Upstage transformers branch, so it does not load in stock transformers. hidden 4096, 48 layers, softmax attention 64 query heads / 8 kv heads, head_dim 128 explicit, so gqa_ratio 8. gqa_interval 3 and gqa_layers lists the 12 softmax layers (0,4,8,...,44); the other 36 are linear attention (linear_attn_config: short_conv_kernel_size 4, head_dim 128, num_heads 64). use_rope is false, use_gqa_gate true, kda_use_full_proj false, kda_allow_neg_eigval true. MoE: n_routed_experts 320, num_experts_per_tok 8, n_shared_experts 1 (the card counts 321 experts total), moe_intermediate_size 1280, first_k_dense_replace 0 so every layer is MoE, routed_scaling_factor 1.0, norm_topk_prob true. vocab 196608, tie_word_embeddings false, bos 1, eos 2. The config does not name an activation, so activation is left null rather than guessed. KV (bf16): only the 12 softmax GQA layers grow with context, 12 x 8 kv heads x 128 x 2 (K+V) x 2 bytes = 49,152 B/token. The 36 linear-attention layers hold a fixed recurrent state, so they are not in that figure. If all 48 layers cached the full context it would be 196,608 B/token, 4x the recorded number, which is the quarter the card claims. CONTEXT: max_position_embeddings 1,048,576, matching the card 1M-token window. The card says positional encoding is removed (NoPE) because the linear layers encode order in their recurrent state, and config use_rope is false with no rope_scaling key. context_scaling_method is none-verified rather than native because the mechanism is described in prose and the config does not name a rope type for the softmax layers. ENGINES AND LICENSE. The card recommends vLLM from the Upstage fork (v0.22.0-solar-open2) at tensor-parallel 8 with expert parallel, and says the minimum hardware is 4x H200. It does not publish a measured single-GPU expert-offload recipe; the offload case is the architecture (8 of 320 routed, 15B active of 250B stored, KV only on 12 of 48 layers). Official INT4 and NVFP4 quants exist under nota-ai but are separate repos, not this record. License is the bespoke Upstage Solar License (card license field other), not an OSI license. Weights repo created 2026-07-22 (HF API createdAt 2026-07-22T02:40:42Z). Technical report arXiv 2607.20062. Benchmark numbers are the card English benchmark table and are vendor claims.