Falcon3 7B Instruct

CONTEXT SCALING: null is a VERIFIED value, checked against config rather than left blank. config.json (tiiuae/falcon3-7B-Instruct) declares rope_scaling null, rope_theta 1000042, max_position_embeddings 32768. Verified native. Falcon3 is NOT a Mamba or hybrid model - config.json declares architectures ['LlamaForCausalLM'], model_type 'llama'. It is a plain dense GQA transformer. What is unusual is the head geometry, and it is worth recording for GEMM efficiency: hidden_size 3072 is narrow for a 7B model, depth is only 28 layers, and head_dim is 256 with just 12 query heads and 4 KV heads (gqa_ratio 3). That inverts the usual 8B recipe of 4096 hidden / 32 layers / 32 heads: the projection GEMMs are narrower but fewer, and head_dim 256 (vs the 128 standard for this size class) means 12 heads instead of 32, so the attention block carries 4 * 3072 * 3072 = 37.7M params/layer against the MLP's 3 * 3072 * 23040 = 212.3M - a 5.6:1 MLP-to-attention ratio. With intermediate_size 23040 the MLP is 7.5x hidden_size, so essentially all of this model's FLOPs and weight bytes sit in the MLP, and the narrow hidden_size makes it a poor fit for tensor-core tiles that prefer 4096-wide matrices. rope_theta 1000042, no rope_scaling, 32768 native context, vocab 131072 (untied embeddings = 2 x 131072 x 3072 = 805M params, 11% of the model). KV bytes/token at bf16 (2 bytes/element): 28 layers x 4 KV heads x 256 head_dim x 2 (K and V) x 2 bytes = 114,688 B/token = 112.0 KiB/token. The 256-wide heads make each KV head twice as wide as a 128-head_dim model's, but comparing against Llama-3.1-8B (32 layers x 8 KV heads x 128 x 2 x 2 = 131,072 B/token) Falcon3 is 12.5% SMALLER overall, because 4 fewer layers and 4 fewer KV heads outweigh the doubled head width. Falcon3 gets a slightly cheaper KV cache than its nominal size class suggests. SERVICEABILITY JOIN (added 2026-10-03; the architecture x engine matrix this record was missing, with a citation per cell in docs/13-model-compatibility.md): FAMILY=dense-gqa. REQUIRED KERNEL CLASS: ordinary causal attention plus GEMM/GEMV. This is the baseline every engine in this repo implements, so the join here is a floor rather than a filter: no engine record in data/engines/ states that dense GQA is unsupported. SUPPORTED BY (cited): vLLM LlamaForCausalLM / Qwen2 / Qwen3 / Mistral / Phi3ForCausalLM rows [[mdl2-vllm-supported-models-2026-10]]; TensorRT-LLM LlamaForCausalLM, Qwen2ForCausalLM, Qwen3ForCausalLM, Phi3ForCausalLM rows [[mdl2-trtllm-supported-models]]; SGLang generative table (Llama, Qwen, Mistral families) [[mdl2-sglang-generative-models]]; llama.cpp LLM_ARCH_LLAMA [[pap-arch-llamacpp-arch-registry]]. HARD CONSTRAINTS THAT DO APPLY even to dense GQA, and they are model-shape rather than architecture-class: tensor-parallel size must divide the attention head count or the engine refuses to start [[mgpu-tp-size-must-divide-attention-heads]] (affects vllm and sglang); and a GQA KV cache is the thing paged-attention machinery exists for, so an engine with no paged KV (llama.cpp's slot-based pool [[llama-cpp]], exllamav2 [[exllamav2]]) loses the concurrency lever rather than the ability to run. ONE HARDWARE-DEPENDENT KERNEL GATE THAT IS NOT ABOUT GQA: on RDNA, vLLM's custom paged-attention kernel is gated on head_size == 128 AND block_size == 16, so a dense GQA model with head_dim 256 falls to the Triton 2D fallback and decode falls off a cliff - measured 12.1 tok/s at 518 tokens of context to 4.2 tok/s at 32k, with kernel_paged_attention_2d growing 28.3x [[rdna-triton-paged-attn-decode-cliff]]. Check head_dim before blaming the card.