DeepSeek-R1-Distill-Qwen-32B

CONTEXT SCALING: left null, and this is now a VERIFIED null rather than an unfilled one - rope_scaling is ABSENT from config (Qwen2ForCausalLM, rope_theta 1000000, max_position_embeddings 131072, sliding_window 131072 = the full window so the sliding window is disabled, use_sliding_window false). The 131072 is therefore NOT explained by any config-level position interpolation. The model card advertises 131072 for this distill; where the extension comes from when the config declares no interpolation is a question the config cannot answer, and null plus this note is the honest record. Note the distinguishability from the Llama-based distills above, which DO carry llama3 rope_scaling in config: same R1 brand, same advertised 131072, different mechanism. The important contrast case for this repo: an R1-family model with plain dense GQA and no MLA at all. kv_cache_bytes_per_token = 64 layers x 8 kv_heads x head_dim 128 x 2 (K and V) x 2 bytes (bf16) = 64 x 8 x 128 x 2 x 2 = 262,144 B/token (256 KiB). head_dim is NOT in config.json; it is 5120/40 = 128 (hidden_size / num_attention_heads), which is the standard Qwen2 derivation. At 3.7x DeepSeek-V3's KV per token (256 KiB vs 68.6 KiB) despite roughly comparable active params, this is why distills run out of KV memory long before they run out of FLOPs. Architecture Qwen2ForCausalLM (model_type qwen2): hidden 5120, 64 layers, 40 heads / 8 kv heads (gqa_ratio 5), intermediate 27648. sliding_window is set to 131072, equal to max_position_embeddings, i.e. the sliding window is disabled in practice. rope_theta 1000000. Context 131072. SERVICEABILITY JOIN (added 2026-10-03; the architecture x engine matrix this record was missing, with a citation per cell in docs/13-model-compatibility.md): FAMILY=dense-gqa. REQUIRED KERNEL CLASS: ordinary causal attention plus GEMM/GEMV. This is the baseline every engine in this repo implements, so the join here is a floor rather than a filter: no engine record in data/engines/ states that dense GQA is unsupported. SUPPORTED BY (cited): vLLM LlamaForCausalLM / Qwen2 / Qwen3 / Mistral / Phi3ForCausalLM rows [[mdl2-vllm-supported-models-2026-10]]; TensorRT-LLM LlamaForCausalLM, Qwen2ForCausalLM, Qwen3ForCausalLM, Phi3ForCausalLM rows [[mdl2-trtllm-supported-models]]; SGLang generative table (Llama, Qwen, Mistral families) [[mdl2-sglang-generative-models]]; llama.cpp LLM_ARCH_LLAMA [[pap-arch-llamacpp-arch-registry]]. HARD CONSTRAINTS THAT DO APPLY even to dense GQA, and they are model-shape rather than architecture-class: tensor-parallel size must divide the attention head count or the engine refuses to start [[mgpu-tp-size-must-divide-attention-heads]] (affects vllm and sglang); and a GQA KV cache is the thing paged-attention machinery exists for, so an engine with no paged KV (llama.cpp's slot-based pool [[llama-cpp]], exllamav2 [[exllamav2]]) loses the concurrency lever rather than the ability to run. ONE HARDWARE-DEPENDENT KERNEL GATE THAT IS NOT ABOUT GQA: on RDNA, vLLM's custom paged-attention kernel is gated on head_size == 128 AND block_size == 16, so a dense GQA model with head_dim 256 falls to the Triton 2D fallback and decode falls off a cliff - measured 12.1 tok/s at 518 tokens of context to 4.2 tok/s at 32k, with kernel_paged_attention_2d growing 28.3x [[rdna-triton-paged-attn-decode-cliff]]. Check head_dim before blaming the card. WINDOW-INERT CAVEAT: its sliding_window is set to the full 131072 = max_position_embeddings, so the window is INERT and this model is served as plain GQA; it appears in the sliding-window family only because the config carries the field. Do NOT apply the KV overpay table to it. CONTEXT_SCALING_METHOD (native): rope_scaling null at theta 1e6 over 131072 - the Qwen2.5-32B base already trains at 128K, so the distill is native rather than extended.