GLM-4.5

CONTEXT SCALING: null is a VERIFIED value here, checked against config rather than left blank. config.json (zai-org/GLM-4.5) declares rope_scaling null, rope_theta 1000000, max_position_embeddings 131072. Verified null. CONTRADICTS A COMMON ASSUMPTION: GLM-4.5 does NOT use MLA. Its config has no kv_lora_rank; it has num_key_value_heads 8 and the HF reference implementation Glm4MoeAttention is plain GQA (kv_a_proj_with_mqa is absent). MLA arrived back in the GLM line only at GLM-4.7. Worth stating plainly because GLM is usually assumed to be DeepSeek-derived. Consequence for serving, and this is the worst KV economics of any 32B-active model in this slice: kv_cache_bytes_per_token = 92 layers x 8 kv_heads x head_dim 128 x 2 (K and V) x 2 bytes (bf16) = 92 x 8 x 128 x 2 x 2 = 376,832 B/token (368 KiB). DeepSeek-V3 does 37B active in 68.6 KiB/token. GLM-4.5 does 32B active in 368 KiB/token - 5.4x the KV for slightly less compute. At 128k context that is 46 GiB of KV per sequence versus 8.58 GiB. On long-context batch serving this is the deciding constraint, not FLOPs. Vendor: 355B total / 32B active (safetensors total 358.34B). Our config arithmetic gives 352.7B / 33.6B, matching to ~1%. MoE: 160 routed + 1 shared, 8 per token, first_k_dense_replace 3, n_group 1 / topk_group 1 (no grouped routing, unlike DeepSeek's 8 groups). routed_scaling_factor 2.5. hidden 5120, 92 layers, 96 heads / 8 kv heads (gqa_ratio 12), head_dim 128, intermediate 12288, moe_intermediate_size 1536. partial_rotary_factor 0.5. Context 131072 native, rope_theta 1000000, rope_scaling null. MIT. num_nextn_predict_layers 1. SERVICEABILITY JOIN (added 2026-10-03; the architecture x engine matrix this record was missing, with a citation per cell in docs/13-model-compatibility.md): FAMILY=dense-gqa. REQUIRED KERNEL CLASS: ordinary causal attention plus GEMM/GEMV. This is the baseline every engine in this repo implements, so the join here is a floor rather than a filter: no engine record in data/engines/ states that dense GQA is unsupported. SUPPORTED BY (cited): vLLM LlamaForCausalLM / Qwen2 / Qwen3 / Mistral / Phi3ForCausalLM rows [[mdl2-vllm-supported-models-2026-10]]; TensorRT-LLM LlamaForCausalLM, Qwen2ForCausalLM, Qwen3ForCausalLM, Phi3ForCausalLM rows [[mdl2-trtllm-supported-models]]; SGLang generative table (Llama, Qwen, Mistral families) [[mdl2-sglang-generative-models]]; llama.cpp LLM_ARCH_LLAMA [[pap-arch-llamacpp-arch-registry]]. HARD CONSTRAINTS THAT DO APPLY even to dense GQA, and they are model-shape rather than architecture-class: tensor-parallel size must divide the attention head count or the engine refuses to start [[mgpu-tp-size-must-divide-attention-heads]] (affects vllm and sglang); and a GQA KV cache is the thing paged-attention machinery exists for, so an engine with no paged KV (llama.cpp's slot-based pool [[llama-cpp]], exllamav2 [[exllamav2]]) loses the concurrency lever rather than the ability to run. ONE HARDWARE-DEPENDENT KERNEL GATE THAT IS NOT ABOUT GQA: on RDNA, vLLM's custom paged-attention kernel is gated on head_size == 128 AND block_size == 16, so a dense GQA model with head_dim 256 falls to the Triton 2D fallback and decode falls off a cliff - measured 12.1 tok/s at 518 tokens of context to 4.2 tok/s at 32k, with kernel_paged_attention_2d growing 28.3x [[rdna-triton-paged-attn-decode-cliff]]. Check head_dim before blaming the card. CLASSIFICATION CORRECTION: this record was initially filed in the MLA family on the strength of its MoE design alone. its own attention_variant field reads 'GQA (standard) - NOT MLA despite the DeepSeek-derived MoE design', so the DeepSeek-shaped MoE design does NOT make it an MLA checkpoint. It belongs in the dense-GQA family for kernel purposes, which is the more useful answer anyway: it needs no latent-attention decode kernel, so the FlashMLA and FlashInfer MLA page-block constraints do not apply to it. vLLM's native row for this family is Glm4MoeForCausalLM (GLM-4.5, 4.6, 4.7) with guided decoding noted as xgrammar-only, llguidance not working [[mdl2-trtllm-supported-models]] [[mdl2-vllm-supported-models-2026-10]]; llama.cpp's dispatch table puts DEEPSEEK2/DEEPSEEK32 in supports_sm_tensor == false but has no such exclusion for this architecture, which is consistent with it being ordinary GQA MoE [[mdl2-llamacpp-arch-class-registry]].