DeepSeek-R1-Distill-Qwen-1.5B

CONTEXT SCALING: left null, and this is now a VERIFIED null rather than an unfilled one - rope_scaling is ABSENT from config (rope_theta 10000, sliding_window 4096, max_position_embeddings 131072). The model card advertises 131072 for this distill; where the extension comes from when the config declares no interpolation is a question the config cannot answer, and null plus this note is the honest record. Note the distinguishability from the Llama-based distills above, which DO carry llama3 rope_scaling in config: same R1 brand, same advertised 131072, different mechanism. Architecture is the parent's, not DeepSeek's: architectures=['Qwen2ForCausalLM'], model_type 'qwen2', i.e. identical to Qwen2.5-1.5B. Only the weights differ - DeepSeek-R1 reasoning traces were distilled into the Qwen dense architecture. head_dim is not in config.json; derived as hidden_size / num_attention_heads = 1536 / 12 = 128. gqa_ratio = 12 / 2 = 6. rope_theta 10000 with no rope_scaling; max_position_embeddings 131072 is declared natively. config declares sliding_window 4096 but use_sliding_window is false, so it is inert in this checkpoint. vocab_size 151936 with tie_word_embeddings false means untied embed_tokens plus lm_head at hidden 1536, 2 x 151936 x 1536 = 466.7M params, 26% of the whole model - a 1.5B model's largest single block is its embedding table, not its transformer stack. KV bytes/token at bf16 (2 bytes/element): 28 layers x 2 KV heads x 128 head_dim x 2 (K and V) x 2 bytes = 28672 B/token = 28.0 KiB/token. At 128k context that is 3.50 GiB of KV for one sequence at bf16. SERVICEABILITY JOIN (added 2026-10-03; the architecture x engine matrix this record was missing, with a citation per cell in docs/13-model-compatibility.md): FAMILY=dense-gqa. REQUIRED KERNEL CLASS: ordinary causal attention plus GEMM/GEMV. This is the baseline every engine in this repo implements, so the join here is a floor rather than a filter: no engine record in data/engines/ states that dense GQA is unsupported. SUPPORTED BY (cited): vLLM LlamaForCausalLM / Qwen2 / Qwen3 / Mistral / Phi3ForCausalLM rows [[mdl2-vllm-supported-models-2026-10]]; TensorRT-LLM LlamaForCausalLM, Qwen2ForCausalLM, Qwen3ForCausalLM, Phi3ForCausalLM rows [[mdl2-trtllm-supported-models]]; SGLang generative table (Llama, Qwen, Mistral families) [[mdl2-sglang-generative-models]]; llama.cpp LLM_ARCH_LLAMA [[pap-arch-llamacpp-arch-registry]]. HARD CONSTRAINTS THAT DO APPLY even to dense GQA, and they are model-shape rather than architecture-class: tensor-parallel size must divide the attention head count or the engine refuses to start [[mgpu-tp-size-must-divide-attention-heads]] (affects vllm and sglang); and a GQA KV cache is the thing paged-attention machinery exists for, so an engine with no paged KV (llama.cpp's slot-based pool [[llama-cpp]], exllamav2 [[exllamav2]]) loses the concurrency lever rather than the ability to run. ONE HARDWARE-DEPENDENT KERNEL GATE THAT IS NOT ABOUT GQA: on RDNA, vLLM's custom paged-attention kernel is gated on head_size == 128 AND block_size == 16, so a dense GQA model with head_dim 256 falls to the Triton 2D fallback and decode falls off a cliff - measured 12.1 tok/s at 518 tokens of context to 4.2 tok/s at 32k, with kernel_paged_attention_2d growing 28.3x [[rdna-triton-paged-attn-decode-cliff]]. Check head_dim before blaming the card.