Llama 4 Maverick (17B active / 400B total)

CONTEXT SCALING: null CONFIRMED against config, not left unfilled. The unsloth mirror's config.json declares rope_scaling absent (null), rope_theta 10000, max_position_embeddings 1048576. Meta markets 1M context on Maverick, but config declares no interpolation, no NTK adjustment and no YaRN - so whatever produced the 1M is not a config-level position-scaling method and this record will not guess at one. Contrast llama-4-scout, whose config DOES carry llama3 rope_scaling (factor 16 from an 8192 base): same family, same 1M-class window, and Scout's mechanism is documented while Maverick's is not. That asymmetry is a real finding rather than a gap. PARAMS: 401.6B is the safetensors weight count; Meta's card says 400B total. ACTIVE: 17B, same as Scout. KV: 48 * 8 * 128 * 2 * 2 bytes = 196,608 bytes/token = 192 KiB/token, identical to Scout because the text tower geometry is identical. This is the cleanest serving comparison in the repo: Maverick is 3.7x the weights of Scout for the same per-token FLOPs and the same KV per token. The whole difference is that 400B must be resident. Unlike Scout there is no rope_scaling block in config.json; max_position_embeddings is 1048576 with no declared scaling method, so the context extension mechanism is not documented in the config. SOURCE CAVEAT: config from the unsloth mirror; 17B/400B and 1M context are Meta's card figures. SERVICEABILITY JOIN (added 2026-10-03; the architecture x engine matrix this record was missing, with a citation per cell in docs/13-model-compatibility.md): FAMILY=dense-gqa. REQUIRED KERNEL CLASS: ordinary causal attention plus GEMM/GEMV. This is the baseline every engine in this repo implements, so the join here is a floor rather than a filter: no engine record in data/engines/ states that dense GQA is unsupported. SUPPORTED BY (cited): vLLM LlamaForCausalLM / Qwen2 / Qwen3 / Mistral / Phi3ForCausalLM rows [[mdl2-vllm-supported-models-2026-10]]; TensorRT-LLM LlamaForCausalLM, Qwen2ForCausalLM, Qwen3ForCausalLM, Phi3ForCausalLM rows [[mdl2-trtllm-supported-models]]; SGLang generative table (Llama, Qwen, Mistral families) [[mdl2-sglang-generative-models]]; llama.cpp LLM_ARCH_LLAMA [[pap-arch-llamacpp-arch-registry]]. HARD CONSTRAINTS THAT DO APPLY even to dense GQA, and they are model-shape rather than architecture-class: tensor-parallel size must divide the attention head count or the engine refuses to start [[mgpu-tp-size-must-divide-attention-heads]] (affects vllm and sglang); and a GQA KV cache is the thing paged-attention machinery exists for, so an engine with no paged KV (llama.cpp's slot-based pool [[llama-cpp]], exllamav2 [[exllamav2]]) loses the concurrency lever rather than the ability to run. ONE HARDWARE-DEPENDENT KERNEL GATE THAT IS NOT ABOUT GQA: on RDNA, vLLM's custom paged-attention kernel is gated on head_size == 128 AND block_size == 16, so a dense GQA model with head_dim 256 falls to the Triton 2D fallback and decode falls off a cliff - measured 12.1 tok/s at 518 tokens of context to 4.2 tok/s at 32k, with kernel_paged_attention_2d growing 28.3x [[rdna-triton-paged-attn-decode-cliff]]. Check head_dim before blaming the card.