Beam 1.0 501B-A23B

ADDED 2026-10-18. Released 2026-10-07 (HF API createdAt 2026-10-07T19:52:01Z); weights and the model card were public by the 2026-10-16 availability post. Card nameplate: 501.3B total, 23.4B active, 52 layers of which 3 are dense and 49 are MoE, 256 experts, gated GQA plus sliding-window attention, 1M context, Apache-2.0. config.json agrees: model_type beam, architectures BeamForCausalLM, hidden_size 5120, num_hidden_layers 52, num_attention_heads 48, num_key_value_heads 8, head_dim 128, num_experts 256, num_experts_per_tok 8, moe_intermediate_size 2560, intermediate_size 20480, vocab_size 151936, max_position_embeddings 1048576, hidden_act silu, sliding_window 512, norm_topk_prob true, n_group 8, topk_group 4, routed_scaling_factor 2.5, final_logit_softcapping 30. mlp_layer_types is dense, dense, dense, then sparse for the remaining 49 layers, matching the card's 3 dense + 49 MoE. No shared-expert key and no shared-expert intermediate size, so shared_expert is false. HF safetensors metadata: 501,203,374,528 BF16 parameters plus 64,237,824 F32 (501,267,612,352 total). The card's 501.3B is the rounded BF16 count. KV CACHE. The card does not publish a per-token cache size, so this is derived and counts only the 9 full-attention layers. Sliding layers hold a fixed window of 512 and do not grow with context, the same treatment the catalog gives hybrid linear-attention models. 9 x 2 (K and V) x num_key_value_heads 8 x head_dim 128 x 2 bytes (bf16) = 9 x 2 x 8 x 128 x 2 = 36,864 B/token from the full layers. The 43 sliding layers add a fixed resident window of 43 x 2 x 8 x 128 x 2 bytes x 512 tokens = 180,879,360 bytes, about 172.5 MiB, which does not grow with context and is not part of the per-token figure. kv_cache_bytes_per_token records the growing part only: 36,864. If every layer cached globally the figure would be 52 x 2 x 8 x 128 x 2 = 212,992 B/token, about 5.8x higher. active_params_b 23.4 is the vendor activated figure, not a count derived here. flops_per_token_active = 2 x 23.4e9 = 4.68e10 is the schema one-pass approximation off that vendor figure. OFFLOAD SHAPE. 8 of 256 experts routed per token and no shared expert, so expert weights can sit in system RAM while the attention stack, the 3 dense MLPs and the router stay resident. At Q4 the active footprint is about 23.4e9 x 0.5 = 11.7 GB of weights read per token, plus the fixed sliding-window state. The total Q4 weight file is about 250 GB, which is why this only makes sense with expert offload rather than as a dense-in-VRAM model. ENGINES AND QUANTS. The 2026-10-16 availability post says weights ship in BF16, FP8 and NVFP4, plus a speculative-decoding drafter, and names day-zero support in Hugging Face Transformers, vLLM (Inferact), SGLang (RadixArk), Unsloth, Axolotl, TokenSpeed (LightSeek) and NVIDIA Dynamo. LM Studio and Ollama are named as launch platforms. The model card itself has no serving recipe and does not say whether vLLM or SGLang can keep attention resident while offloading experts; that split is the architecture, not a measured engine flag. llama.cpp is not named. BENCHMARKS ARE VENDOR CLAIMS from the model card's own BF16 column, which the card says were run by Reflection. BEATS models/glm-5-3-flash, the catalog's best large-sparse record with a published SWE-bench Pro number, on SWE-bench Pro: 66.1 (card's SWE Bench Pro v1, BF16) vs 59.70. Same class: both are large-total MoE with a much smaller active footprint that stays runnable when experts are offloaded to RAM. Beam activates 23.4B of 501.3B with 8 of 256 experts; GLM-5.3-Flash activates 18B of 320B with 8 of 288. The same card column also reports SWE-bench Verified 81.6, which is above the 79.6 vendor claim on models/mimo-v2-6-flash-rl and the 79.10 on models/glm-5-3-flash, Terminal-Bench 2.1 80.1, GPQA Diamond 90.6 and AIME 2026 97.7. It does not beat models/mimo-v2-6-flash-rl on Terminal-Bench 2.1 (80.1 vs 87.6) or models/qwen3-8-flash-next's card-reported 85.80 on that benchmark as repeated on the Nemotron 3.5 Super card. The launch blog (updated 2026-10-08) prints a slightly different row for the same benchmarks (SWE-bench Verified 80.9, SWE Bench Pro v1 65.5, Terminal Bench v2.1 80.1, GPQA Diamond 90.5); the model-card BF16 column is the one cited here. 23.4B active is well above the 6B-active Flash-Next reference shape, so decode under expert offload is several times more expensive than the small-active class. It earns a record because no catalogued offload-friendly model has a published SWE-bench Pro number this high.