Kimi-K2.6

ADDED 2026-10-18. Open-weight native multimodal agent model, released 2026-04-14 (HF API createdAt), built on Kimi K2.5. The catalog had models/kimi-k2 (the 1T/32B text model) but not this later multimodal checkpoint. RUNNABLE WITH EXPERT OFFLOAD: 1026.879B total but the card activates 32B per token and routes only 8 of 384 experts plus one shared expert, so the per-token expert traffic is a small slice of the checkpoint and the expert bank can sit in system RAM. Same offload shape as models/kimi-k2. PARAMS: HF safetensors metadata total is 1,026,879,376,368. Of that, 1,014,686,023,680 parameters are stored as I32, which is the INT4 packing (int4 weights in int32 containers), and 12,193,329,648 are BF16. params_b is that total, 1026.879. active_params_b is the card 32B figure, not a count derived here. flops_per_token_active = 2 x 32e9 = 6.4e10 is the schema one-pass approximation. BEATS models/qwen3-6-35b-a3b, the catalog best open-weight record with a published LiveCodeBench v6 number, on LiveCodeBench v6: 89.6 vs 75.4. Both numbers are vendor claims and were not re-run on one harness here. The class is large-sparse MoE with a far smaller active footprint: this one routes 8 of 384 experts, and the Qwen record routes 8 of 256, so both stay runnable with experts offloaded. The same card column is also ahead of models/qwen3-6-35b-a3b on GPQA Diamond (90.5 vs 86.7) and on SWE-bench Verified (80.2 vs 76.0). It does NOT beat models/step-5-preview on GPQA Diamond (90.5 vs 93.5), and the card does not publish an MMLU-Pro number, so LiveCodeBench v6 is the qualifying comparison. The card K2.5 column is 85.0 on LiveCodeBench v6 and 76.8 on SWE-bench Verified; there is no models/kimi-k2-5 record, and models/kimi-k2 carries no benchmark numbers. GEOMETRY from config.json fetched 2026-10-18 (commit 7eb5002f6aad): top-level model_type kimi_k25, text_config model_type kimi_k2, hidden 7168, 61 layers, 64 attention heads, first layer dense (first_k_dense_replace 1) and the remaining 60 are MoE. MoE: 384 routed experts, 8 per token, n_shared_experts 1, moe_intermediate_size 2048, so shared_expert is true. MLA: kv_lora_rank 512, q_lora_rank 1536, qk_nope_head_dim 128, qk_rope_head_dim 64, v_head_dim 128. head_dim is recorded as the nope dim (128), matching models/kimi-k2; num_kv_heads is the attention-head count because MLA has no GQA group. KV (bf16, MLA compressed): 61 layers x (kv_lora_rank 512 + qk_rope_head_dim 64) x 2 bytes = 61 x 576 x 2 = 70,272 B/token, the same formula models/kimi-k2 uses. RoPE is YaRN: factor 64, original 4096, rope_theta 50000, declared max_position_embeddings 262144, which is the card 256K context. License is the Moonshot modified MIT linked from the repo LICENSE file, the same bespoke license as models/kimi-k2. ENGINES: the card names vLLM, SGLang and KTransformers and says the Kimi-K2.5 deployment method is reused; transformers must be >=4.57.1 and <5.0.0. QUANT: native INT4 via compressed-tensors (group size 32, pack-quantized, symmetric). The quant ignore list leaves attention, the shared expert, the dense MLP, lm_head and the vision tower in higher precision, so the routed experts are the INT4 bank. Expert offload while attention stays resident is the architecture (8 of 384 routed, attention and the shared expert are not in the expert bank) and is what KTransformers is built for; the card does not publish a measured single-GPU offload recipe, so that split is not recorded as a measured flag.