Kolibri-1

ADDED 2026-10-17. Open-weight MoE in the ~4B-active class, released 2026-10-03 (card Release Date). Card nameplate: 78,103,074,560 total and 3,457,573,120 active per token, so params_b is 78.103 and active_params_b is 3.458. flops_per_token_active = 2 x 3.458e9 = 6.916e9 is the schema one-pass approximation off the vendor active count, not a profile. BEATS models/gemma-4-26b-a4b on AIME 2026. The Kolibri card's own table, repeated by the HF eval-results widget, reports MathArena AIME 2026 at 96. The Xing4.0-29B-A4B card, which is the source of the catalog comparison, reports Gemma4-26B-A4B at 88.30 on AIME2026 under its own harness (and Xing itself at 90.00, Qwen3.6-35B-A3B at 92.70). Same class: gemma-4-26b-a4b is the catalog's ~4B-active MoE (25.81B total, 4.0B active) and Kolibri is 78.1B total with 3.46B active. Both the Kolibri 96 and the Gemma 88.30 are vendor claims from different cards and were not re-run on one harness here. The same Kolibri table reports Terminal-Bench 2.1 at 27.7, which does NOT beat xing4-0-29b-a4b (57.50) or gemma-4-26b-a4b (30.00) on that benchmark, so the class win is AIME only. GEOMETRY from config.json fetched 2026-10-17: model_type kolibri1, hidden 2560, 50 layers, heads 48 / kv heads 4 so gqa_ratio 12, head_dim 128, silu, vocab 128000, rope_theta 10000, no rope_scaling key. MoE: 384 experts, 6 per token, moe_intermediate_size 512, shared_expert_intermediate_size 512 so shared_expert is true. Attention is hybrid sliding-window, not linear attention: layer_types is 40 sliding_attention + 10 full_attention, full attention every 5th layer (indices 4, 9, 14, 19, 24, 29, 34, 39, 44, 49), sliding_window 513. The growing KV term is the 10 full-attention layers: 10 x 4 kv heads x 128 x 2 x 2 bytes = 20,480 B/token. The 40 sliding layers cap at 40 x 4 x 128 x 2 x 2 x 513 = 42,106,880 B (about 40.2 MiB) regardless of context, so they are not in kv_cache_bytes_per_token. CONTEXT CONFLICT: the card says 1,048,576 tokens and recommends at most 262,144 for serving; config.json max_position_embeddings is 262144 and there is no rope_scaling. max_position_embeddings follows the config. context_scaling_method is none-verified because the 1M claim is not in the file that was read. RESIDENT MEMORY AND ENGINES. The card's own hardware line says the FP8 checkpoint is ~78 GB and lists a minimum of 2x A100 80 GB or 1x H200, which is datacenter hardware and does not qualify on its own. What qualifies is the quant path, and it is third-party, not Aleph Alpha. velaia/Kolibri-1-MLX-2bit (community, not the vendor) reports a 4-bit MLX build at 41 GiB with peak memory 44 GB on a short prompt and 48 GB at a 4k prompt, which fits a 64 GB unified-memory machine, and a 2-bit build at 24 GiB with peak 26/30 GB. Hob-forge/Kolibri-1-GGUF publishes Q4_K_M at 47.5 GB and Q2_K at 28.6 GB (file size, not a measured RSS) and shows llama.cpp running Q4_K_M with cpu-moe so only attention and the shared expert sit on the GPU (6.4 GB VRAM at 262,144 context on a 12 GB card, experts in system RAM). Both quant cards say they did not run the vendor benchmark suite, so the AIME number above is the FP8 model's vendor claim, not a measurement of the 4-bit or 2-bit checkpoint. Engines: the GGUF README targets llama.cpp with a kolibri1 arch patch; the MLX README targets mlx-lm. The vendor card does not name vLLM, SGLang, or llama.cpp. Stock builds that only know common arch tags will not load model_type kolibri1 until that arch is registered. License Apache-2.0. Vendor is Aleph Alpha. English and German, with an explicit reasoning mode and tool calling. FP8 e4m3 weights, 128x128 block size, dynamic activation quant; embeddings, the LM head, norms and the MoE router stay bf16.