Ling-3.0-flash

CONTEXT SCALING: null is a VERIFIED value here, checked against config rather than left blank. config.json (inclusionAI/Ling-3.0-Flash) declares rope_scaling null with rope_theta 6000000, max_position_embeddings 262144. Verified native as far as config states. Vendor: 124B total / 5.1B active; safetensors total 127.49B. Card calls it a 'native hybrid linear attention' model, hybrid from the start of pretraining rather than retrofit. The pattern is KDA + MLA, which makes it architecturally a sibling of Kimi Linear: config (model_type bailing_hybrid) has kv_lora_rank 512, qk_nope_head_dim 128, qk_rope_head_dim 64, v_head_dim 128, qk_head_dim 192, plus max_window_layers 20 and layer_group_size 6, short_conv_kernel_size 4, kda_lower_bound -5.0, kda_safe_gate true. The card states 5:1 alternating stacking of KDA and MLA. kv_cache_bytes_per_token = 20 full-attention layers x (kv_lora_rank 512 + qk_rope_head_dim 64) x 2 bytes = 20 x 576 x 2 = 23,040 B/token (22.5 KiB). Note 20 is max_window_layers, and the config does NOT enumerate which layers are full vs linear the way Kimi Linear's linear_attn_config does - the 5:1 ratio over 42 layers is the source for the split, so this derivation is the least certain of the hybrid records. Treat the layer count as approximate. At 5.1B active and 22.5 KiB/token this is one of the cheapest long-context configurations in the slice: ~2.75 GiB of KV at 128k. MoE: 512 routed + 1 shared expert (moe_shared_expert_intermediate_size 768), 8 per token, first_k_dense_replace 2, moe_intermediate_size 768, topk_method noaux_tc with n_group 8 / topk_group 4. hidden 2560, intermediate 6144. Context 262144, rope_theta 6000000, partial_rotary_factor 0.5, rope_interleave true. MIT. SERVICEABILITY JOIN (added 2026-10-03; the architecture x engine matrix this record was missing, with a citation per cell in docs/13-model-compatibility.md): FAMILY=hybrid-attention-ssm. REQUIRED KERNEL CLASS: two kernel families in one graph - a chunked linear-attention / delta-rule kernel for the SSM or linear-attention layers, and an ordinary or MLA decode kernel for the full-attention minority - plus a fixed-size recurrent state allocator. Three independent records say the recurrent branch is the hard part: flop/ssm-recurrent-state-update (the state is a fixed per-sequence tax that never amortises, so it scales with concurrency and NOT with context); flop/ssm-sequential-scan-tensor-core-mismatch (the recurrence is a multiplicative chain that tensor cores cannot express, so it must run as a sequential scan; the published SSD-vs-fused-scan gap is 2-8x for identical mathematics); and llama.cpp's own dispatch table, which puts every hybrid and SSM architecture in its supports_sm_tensor == false list [[mdl2-llamacpp-arch-class-registry]] - the engine agreeing with the arithmetic. That same list also shows llama.cpp needed ONE shared hybrid code path (llm_arch_is_hybrid, 18 entries: JAMBA, FALCON_H1, PLAMO2, GRANITE_HYBRID, LFM2, LFM2MOE, NEMOTRON_H, NEMOTRON_H_MOE, QWEN3NEXT, KIMI_LINEAR, BAILINGMOE3, KIMI_K3, GLM5_NEXT, QWEN35, QWEN35MOE, QWEN4EXP, DEEPSEEK4, MINIMAX_01) covering exactly the families vLLM and SGLang each ship as separate rows, which is why hybrid support arrives per engine release rather than at once. SUPPORTED BY (cited): vLLM native rows MambaForCausalLM, Mamba2ForCausalLM, JambaForCausalLM, FalconMambaForCausalLM, FalconH1ForCausalLM, NemotronHForCausalLM, GraniteMoeHybridForCausalLM, Qwen3NextForCausalLM, KimiLinearForCausalLM, Lfm2ForCausalLM, Lfm2MoeForCausalLM, OlmoHybridForCausalLM, KimiK3ForConditionalGeneration, BailingMoeV3ForCausalLM, Qwen3_5 rows [[mdl2-vllm-supported-models-2026-10]]; SGLang generative table names Granite 4.0 micro as 'hybrid Mamba-MoE ... combining gated short convolutions with a small number of grouped query attention blocks', Kimi Linear as 'hybrid linear attention model ... Kimi Delta Attention (KDA) for up to 6x faster decoding and 75% KV cache reduction vs full attention', Nemotron Nano as 'hybrid Mamba-Transformer architecture', Ling as 'hybrid MoE model: per-layer mix of DeepSeek-style MLA full attention and Qwen3 Gated-Delta-Net (GDN) linear attention', LFM2 as 'hybrid backbone', Mamba-Codestral as 'Mamba2 ... holds no KV cache', and its multimodal page carries the descriptor 'hybrid KDA/MLA' [[mdl2-sglang-generative-models]] [[mdl2-sglang-multimodal-language-models]]. TensorRT-LLM has rows for NemotronHForCausalLM, Qwen3NextForCausalLM, Qwen3_5MoeForCausalLM, Qwen4ExpForCausalLM, KimiLinearForCausalLM and KimiK3ForConditionalGeneration but NO row at all for MambaForCausalLM, Mamba2ForCausalLM, FalconH1ForCausalLM, Lfm2ForCausalLM or GraniteMoeHybridForCausalLM [[mdl2-trtllm-supported-models]]. PER-FAMILY FEATURE TRAPS, all from the TRT-LLM key-models matrix, which is the only per-architecture feature matrix in this repo [[mdl2-trtllm-supported-models]]: Qwen3NextForCausalLM has Sliding Window Attention = No and Speculative Decoding = No; Kimi K3 has no MTP or EAGLE-3 head at all (spec method DSpark only) and needs Blackwell SM100; MiniMax-M3 sparse-attention path supports neither KV cache reuse nor MTP in this release; DeepSeek-V4 and Step-3.7 are 'Untested' for disaggregated serving. KERNEL BUGS THAT ARE ARCHITECTURE-SPECIFIC AND REPRODUCED: (1) vLLM's EAGLE last-block drop (drop_eagle_block) costs 6-10 points of prefix-cache hit rate, and on a hybrid GDN/Mamba layout the scheduler alignment unit is 1,648 tokens instead of a 16-token dense block, so the same protection costs ~100x more cache hit - 97.5% to 86.9-91.7% hit rate and 30-40% batch throughput on prefix-reusing workloads [[eagle-prefix-cache-last-block-drop]]. (2) llama.cpp decode collapses on hybrid gated-deltanet GGUF models at high KV position: 33 t/s at 68K to 1.4 t/s at 91K with the model fully resident, across KV quant types and with FA on or off, reproduced on Windows and WSL2 on master and release [[gated-deltanet-decode-collapse-long-context]]. (3) llama.cpp's SYCL path has a regression specific to hybrid linear-attention graphs - qwen3next/qwen3_35 architecture models crash in ggml_sycl_op_mul_mat or return empty output on 2x Arc Pro B60, bracketed between builds b8477 (works) and b9479 (fails), while dense-attention models on the same cards are unaffected [[sycl-hybrid-linear-attention-arch-crash-intel-arc]]. (4) FlashInfer rejects large head sizes at engine init for the head geometry these hybrids introduce [[flashinfer-rejects-large-head-dim]]. Hybrid MLA + Kimi Delta Attention: requires BOTH the MLA decode kernel and the chunked delta-rule kernel, so it is strictly harder than either. SGLang names it as 'hybrid MoE model: per-layer mix of DeepSeek-style MLA full attention and Qwen3 Gated-Delta-Net (GDN) linear attention' [[mdl2-sglang-generative-models]] and its multimodal page carries the descriptor 'hybrid KDA/MLA' [[mdl2-sglang-multimodal-language-models]]; llama.cpp's llm_arch_is_hybrid includes BAILINGMOE3, which is this model's model_type family (bailing_hybrid) [[mdl2-llamacpp-arch-class-registry]]. LAYER SPLIT CORRECTED + kv_cache_bytes_per_token FIXED (2026-10-05): the KV figure was 23040 B/token and was wrong by 2.86x (OVERSTATED), because it counted 20 attention layers. config.json (inclusionAI/Ling-3.0-Flash) does not enumerate the attention layers directly - that is why this record had none recorded - but it does publish the RULE, in two fields: num_hidden_layers 42 and layer_group_size 6. modeling_bailing_moe_v3.py implements exactly that rule at the layer-construction site: self.attention_layer_type = 'attention' if (layer_idx + 1) % config.layer_group_size == 0 or layer_idx >= config.num_hidden_layers // config.layer_group_size * config.layer_group_size else 'linear_attention'. Over 42 layers with group size 6 that yields attention at 0-based indices [5, 11, 17, 23, 29, 35, 41] = 7 layers, and the remaining 35 are KDA (Kimi Delta Attention) layers holding a fixed-size recurrent state. CORRECTED ARITHMETIC: 7 attention layers x (kv_lora_rank 512 + qk_rope_head_dim 64) x 2 bytes (bf16) = 7 x 576 x 2 = 8064 B/token (7.9 KiB/token). attention_layer_indices and layer_types are now recorded, which this record previously left null. The sibling record kimi-linear-48b-a3b is the cross-check that this was wrong: it is the same KDA+MLA design and its config DOES enumerate linear_attn_config.full_attn_layers [4,8,12,16,20,24,27] = 7 of 27 layers, giving 7 x 576 x 2 = 8,064 B/token. Ling at 7 of 42 giving 8,064 as well is the consistent answer; 23,040 was not derivable from config under any reading. WHY THE ERROR MATTERED: Ling-3.0-Flash advertises 262144 max_position_embeddings. At the old figure a single 256k sequence needs 256000 x 0.02 GiB = 5.5 GiB of KV; at the corrected figure it needs 1.92 GiB. That is the difference between a 256k request fitting alongside the weights on a single 80GB-class card and not fitting at all, so the error inverted the answer to the one question the field exists to answer. Also note max_window_layers 20 in config: that is a WINDOW cap on the attention layers, not a layer count, and must not be confused with the 20 the old figure implied.