GLM-4.1V-9B-Thinking

TWO-PART MODEL, and one of only three records here using mrope. (1) Vision: glm4v_vision, 24 layers, hidden 1536, 12 heads, patch 14, image_size 336, spatial_merge_size 2, temporal_patch_size 2, out_hidden_size 4096, silu. VISION TOKENS: each 336x336 crop = (336/14)^2 = 576 patches, 2x2 spatial_merge -> 144 LLM tokens per crop. This is a deliberately SMALL per-crop budget (compare Phi-3.5-vision's 144 with no tiling, and Gemma 3's 4,096 uncropped) because GLM-4.1V leans on cropping rather than on a huge single view. DOMINANCE: VISION DOMINATES as tiling engages -- several hundred tokens per image against a ~100-token prompt -- but the per-crop 144 keeps the total bounded and predictable. (2) LLM: glm4v_text, 40 layers, hidden 4096, 32 heads / only 2 KV heads, head_dim 128, intermediate 13696, vocab 151,552, max_position 65,536, partial_rotary_factor 0.5 (half of head dims get RoPE), attention_bias true, rope_theta 1e4, tie_word_embeddings false. ARCHITECTURE = dense. CONTEXT_SCALING = mrope: rope_scaling {rope_type 'default', mrope_section [8,12,12]}. rope_type 'default' means no frequency rescaling -- the 65,536 context is native and mrope only partitions the 32 rotary halves into temporal/height/width groups of 8/12/12. KV CACHE: 40 * 2 * 128 * 2 * 2 = 40,960 B/token (40 KiB). Only 2 KV heads over 40 layers, i.e. gqa_ratio 16 -- the second-smallest KV per token of any VLM here, 3.5x better than MiniCPM-V-4_5 and 3.6x better than Phi-3.5-vision, at comparable hidden size. At 65,536 ctx that is 2.5 GiB per sequence at bf16, which makes the declared context genuinely servable. This is the KV-efficiency mirror image of Qwen3-VL-8B (147,456 B/token) at nearly the same width. params_b 10.293 = HF API safetensors total (10,292,777,472); the '9B' name refers to the LLM, with the vision tower adding ~1.3B. GLM-4V-9B (the 2024 predecessor) was NOT recorded: its config is a much older ChatGLM-era layout (num_layers/kv_channels at top level, multi_query_attention true, and a 63-layer vision tower with 6401 learned positions at image_size 1120), whose HF API tensor total of 13,906,320,928 contradicts the '9b' name by 54%. That contradiction is unresolved and GLM-4.1V supersedes it. SERVICEABILITY JOIN (added 2026-10-03; the architecture x engine matrix this record was missing, with a citation per cell in docs/13-model-compatibility.md): FAMILY=dense-gqa. REQUIRED KERNEL CLASS: ordinary causal attention plus GEMM/GEMV. This is the baseline every engine in this repo implements, so the join here is a floor rather than a filter: no engine record in data/engines/ states that dense GQA is unsupported. SUPPORTED BY (cited): vLLM LlamaForCausalLM / Qwen2 / Qwen3 / Mistral / Phi3ForCausalLM rows [[mdl2-vllm-supported-models-2026-10]]; TensorRT-LLM LlamaForCausalLM, Qwen2ForCausalLM, Qwen3ForCausalLM, Phi3ForCausalLM rows [[mdl2-trtllm-supported-models]]; SGLang generative table (Llama, Qwen, Mistral families) [[mdl2-sglang-generative-models]]; llama.cpp LLM_ARCH_LLAMA [[pap-arch-llamacpp-arch-registry]]. HARD CONSTRAINTS THAT DO APPLY even to dense GQA, and they are model-shape rather than architecture-class: tensor-parallel size must divide the attention head count or the engine refuses to start [[mgpu-tp-size-must-divide-attention-heads]] (affects vllm and sglang); and a GQA KV cache is the thing paged-attention machinery exists for, so an engine with no paged KV (llama.cpp's slot-based pool [[llama-cpp]], exllamav2 [[exllamav2]]) loses the concurrency lever rather than the ability to run. ONE HARDWARE-DEPENDENT KERNEL GATE THAT IS NOT ABOUT GQA: on RDNA, vLLM's custom paged-attention kernel is gated on head_size == 128 AND block_size == 16, so a dense GQA model with head_dim 256 falls to the Triton 2D fallback and decode falls off a cliff - measured 12.1 tok/s at 518 tokens of context to 4.2 tok/s at 32k, with kernel_paged_attention_2d growing 28.3x [[rdna-triton-paged-attn-decode-cliff]]. Check head_dim before blaming the card. CONTEXT_SCALING_METHOD (mrope): rope_scaling {rope_type: 'default', mrope_section: [8,12,12]} - the mrope_section is the whole point of the declaration.