Granite 4.0 H Tiny

CONTEXT SCALING: null is a VERIFIED value here, checked against config rather than left blank. config.json (ibm-granite/granite-4.0-h-tiny) declares rope_scaling null, position_embedding_type "nope" (NoPE - no positional embedding at all, hence no rope scaling is possible), max_position_embeddings 131072. Verified null, and structurally impossible by design. MoE + hybrid-Mamba, and both matter. config.json layer_types gives 40 layers of which 36 are 'mamba' and only 4 are 'attention' (positions 5, 14, 23, 32). num_local_experts 64 with num_experts_per_tok 6 and a shared_intermediate_size 1024 always-on expert alongside per-expert hidden 512. params_b 6.939 is the safetensors total; active_params_b 1.0 is IBM's own stated active-parameter figure for H-Tiny in the model card table, NOT derived from config - treat as vendor-stated. KV: only the 4 attention layers hold KV. bf16 KV bytes/token = 4 attention layers x 4 KV heads x 128 head_dim (1536 / 12) x 2 x 2 bytes = 8192 B/token = 8.0 KiB/token - 16x smaller than the 131,072 B/token of a 32-layer 8B dense GQA model (Llama-3.1-8B geometry), and 5.5x smaller than Falcon-H1-7B's 45,056 B/token, the only other hybrid in this slice. The ranking across this slice is: 8,192 B/token (Granite 4.0 H-Tiny) << 16,384 (Nemotron-H-8B) << 45,056 (Falcon-H1-7B) << 114,688 (Falcon3-7B) < 131,072 (Llama-3.1-8B) << 163,840 (Granite 3.3 8B) - i.e. the hybrid-Mamba models buy a 8-16x KV reduction over comparable dense GQA models, but only the interleaved 4-attention-layer variants get it; Falcon-H1's parallel-in-every-layer design gets barely 2.9x. The other 36 layers carry a constant Mamba-2 state instead: mamba_d_state 128, mamba_d_head 64, mamba_n_heads 48, so per sequence per layer the recurrent state is 48 x 64 x 128 = 393216 elements = 0.75 MiB at bf16, x 36 layers = 27 MiB, constant in context length. Net: KV does not grow with context, and resident state is a one-time 27 MiB per sequence. position_embedding_type 'nope' - NoPE on the attention layers, with Mamba carrying position implicitly. mamba_chunk_size 256, mamba_d_conv 4. Serving shape: 6.9B of weights resident (the memory bottleneck) for 1B of per-token compute (the speed bottleneck), 131072 native context with no rope_scaling. SERVICEABILITY JOIN (added 2026-10-03; the architecture x engine matrix this record was missing, with a citation per cell in docs/13-model-compatibility.md): FAMILY=hybrid-attention-ssm. REQUIRED KERNEL CLASS: two kernel families in one graph - a chunked linear-attention / delta-rule kernel for the SSM or linear-attention layers, and an ordinary or MLA decode kernel for the full-attention minority - plus a fixed-size recurrent state allocator. Three independent records say the recurrent branch is the hard part: flop/ssm-recurrent-state-update (the state is a fixed per-sequence tax that never amortises, so it scales with concurrency and NOT with context); flop/ssm-sequential-scan-tensor-core-mismatch (the recurrence is a multiplicative chain that tensor cores cannot express, so it must run as a sequential scan; the published SSD-vs-fused-scan gap is 2-8x for identical mathematics); and llama.cpp's own dispatch table, which puts every hybrid and SSM architecture in its supports_sm_tensor == false list [[mdl2-llamacpp-arch-class-registry]] - the engine agreeing with the arithmetic. That same list also shows llama.cpp needed ONE shared hybrid code path (llm_arch_is_hybrid, 18 entries: JAMBA, FALCON_H1, PLAMO2, GRANITE_HYBRID, LFM2, LFM2MOE, NEMOTRON_H, NEMOTRON_H_MOE, QWEN3NEXT, KIMI_LINEAR, BAILINGMOE3, KIMI_K3, GLM5_NEXT, QWEN35, QWEN35MOE, QWEN4EXP, DEEPSEEK4, MINIMAX_01) covering exactly the families vLLM and SGLang each ship as separate rows, which is why hybrid support arrives per engine release rather than at once. SUPPORTED BY (cited): vLLM native rows MambaForCausalLM, Mamba2ForCausalLM, JambaForCausalLM, FalconMambaForCausalLM, FalconH1ForCausalLM, NemotronHForCausalLM, GraniteMoeHybridForCausalLM, Qwen3NextForCausalLM, KimiLinearForCausalLM, Lfm2ForCausalLM, Lfm2MoeForCausalLM, OlmoHybridForCausalLM, KimiK3ForConditionalGeneration, BailingMoeV3ForCausalLM, Qwen3_5 rows [[mdl2-vllm-supported-models-2026-10]]; SGLang generative table names Granite 4.0 micro as 'hybrid Mamba-MoE ... combining gated short convolutions with a small number of grouped query attention blocks', Kimi Linear as 'hybrid linear attention model ... Kimi Delta Attention (KDA) for up to 6x faster decoding and 75% KV cache reduction vs full attention', Nemotron Nano as 'hybrid Mamba-Transformer architecture', Ling as 'hybrid MoE model: per-layer mix of DeepSeek-style MLA full attention and Qwen3 Gated-Delta-Net (GDN) linear attention', LFM2 as 'hybrid backbone', Mamba-Codestral as 'Mamba2 ... holds no KV cache', and its multimodal page carries the descriptor 'hybrid KDA/MLA' [[mdl2-sglang-generative-models]] [[mdl2-sglang-multimodal-language-models]]. TensorRT-LLM has rows for NemotronHForCausalLM, Qwen3NextForCausalLM, Qwen3_5MoeForCausalLM, Qwen4ExpForCausalLM, KimiLinearForCausalLM and KimiK3ForConditionalGeneration but NO row at all for MambaForCausalLM, Mamba2ForCausalLM, FalconH1ForCausalLM, Lfm2ForCausalLM or GraniteMoeHybridForCausalLM [[mdl2-trtllm-supported-models]]. PER-FAMILY FEATURE TRAPS, all from the TRT-LLM key-models matrix, which is the only per-architecture feature matrix in this repo [[mdl2-trtllm-supported-models]]: Qwen3NextForCausalLM has Sliding Window Attention = No and Speculative Decoding = No; Kimi K3 has no MTP or EAGLE-3 head at all (spec method DSpark only) and needs Blackwell SM100; MiniMax-M3 sparse-attention path supports neither KV cache reuse nor MTP in this release; DeepSeek-V4 and Step-3.7 are 'Untested' for disaggregated serving. KERNEL BUGS THAT ARE ARCHITECTURE-SPECIFIC AND REPRODUCED: (1) vLLM's EAGLE last-block drop (drop_eagle_block) costs 6-10 points of prefix-cache hit rate, and on a hybrid GDN/Mamba layout the scheduler alignment unit is 1,648 tokens instead of a 16-token dense block, so the same protection costs ~100x more cache hit - 97.5% to 86.9-91.7% hit rate and 30-40% batch throughput on prefix-reusing workloads [[eagle-prefix-cache-last-block-drop]]. (2) llama.cpp decode collapses on hybrid gated-deltanet GGUF models at high KV position: 33 t/s at 68K to 1.4 t/s at 91K with the model fully resident, across KV quant types and with FA on or off, reproduced on Windows and WSL2 on master and release [[gated-deltanet-decode-collapse-long-context]]. (3) llama.cpp's SYCL path has a regression specific to hybrid linear-attention graphs - qwen3next/qwen3_35 architecture models crash in ggml_sycl_op_mul_mat or return empty output on 2x Arc Pro B60, bracketed between builds b8477 (works) and b9479 (fails), while dense-attention models on the same cards are unaffected [[sycl-hybrid-linear-attention-arch-crash-intel-arc]]. (4) FlashInfer rejects large head sizes at engine init for the head geometry these hybrids introduce [[flashinfer-rejects-large-head-dim]]. Hybrid MoE + Mamba-2: needs the MoE grouped-GEMM path AND the SSD scan AND ordinary attention, three kernel classes. vLLM row GraniteMoeHybridForCausalLM exists and SGLang describes Granite 4.0 micro as 'hybrid Mamba-MoE' with gated short convolutions plus a small number of GQA blocks [[mdl2-vllm-supported-models-2026-10]] [[mdl2-sglang-generative-models]]. TRT-LLM has NO Granite or Granite-hybrid row in its architecture table, so that cell is unsupported-by-omission rather than an explicit no [[mdl2-trtllm-supported-models]]. llama.cpp puts GRANITE_HYBRID in llm_arch_is_hybrid and in the supports_sm_tensor == false list [[mdl2-llamacpp-arch-class-registry]]. EXL3 vendors Flash Linear Attention kernels (chunked linear-attention prefill under exllamav3/vendor/fla) and its README has the broadest HF architecture coverage of any engine here, which is the specific reason a hybrid lands early on consumer NVIDIA [[exllamav3]]. CONTEXT_SCALING_METHOD (native): SCHEME RECLASSIFIED (2026-10-04): was null, read as 'no data'. this is a VERIFIED value and structurally impossible otherwise: config declares position_embedding_type 'nope' (NoPE), so rope scaling cannot exist by construction. Null here means native, not missing. LAYER LAYOUT (granite-4-0-h-tiny): layer_types read verbatim from config, one entry per layer; attention_layer_indices are the 0-based positions of the full-attention entries.