Phi-3.5-mini-instruct

KV CACHE CORRECTED 2026-10-04 (parent agent): was 24576.0, which is wrong by 16x. Correct value 393216 = 32 layers * 32 kv_heads * 96 head_dim * 2 (K+V) * 2 (bf16). The record's own note showed the formula but its arithmetic did not match it. Param count 3821079552 from safetensors metadata is IDENTICAL to Phi-3 mini, and every geometry field matches (hidden 3072, 32 layers, 32 KV heads, intermediate 8192, vocab 32064) — verified field-by-field against both config.json files. head_dim derived = 3072 / 32 = 96. kv_cache_bytes_per_token = 32 * 32 * 96 * 2 * 2 = 24576 bytes/token at bf16, same as Phi-3 mini. The one change is context: max_position_embeddings 131072 with original_max_position_embeddings 4096 and a LongRoPE-style extension, plus sliding_window raised from 2047 to 262144 which effectively disables block sparsity. That matters for serving: at the advertised 128k context this model holds 128k * 24576 B = 3.0 GiB of KV per sequence, which is more than a 4-bit copy of the weights. Anyone quoting the 128k window without quoting the KV cost is quoting half the model. Phi-3.5-mini has NO separate rope_scaling object in config.json (LongRoPE is baked into the released weights); context_scaling 'LongRoPE' is stated by Microsoft's model card, not by config.json — treat that one field as card-derived. Official ONNX and GGUF ports exist, including onnx-community/Phi-3.5-mini-instruct-ONNX-GQA. SERVICEABILITY JOIN (added 2026-10-03; the architecture x engine matrix this record was missing, with a citation per cell in docs/13-model-compatibility.md): FAMILY=dense-mha. REQUIRED KERNEL CLASS: ordinary causal attention plus GEMM/GEMV. This is the baseline every engine in this repo implements, so the join here is a floor rather than a filter: no engine record in data/engines/ states that dense GQA is unsupported. SUPPORTED BY (cited): vLLM LlamaForCausalLM / Qwen2 / Qwen3 / Mistral / Phi3ForCausalLM rows [[mdl2-vllm-supported-models-2026-10]]; TensorRT-LLM LlamaForCausalLM, Qwen2ForCausalLM, Qwen3ForCausalLM, Phi3ForCausalLM rows [[mdl2-trtllm-supported-models]]; SGLang generative table (Llama, Qwen, Mistral families) [[mdl2-sglang-generative-models]]; llama.cpp LLM_ARCH_LLAMA [[pap-arch-llamacpp-arch-registry]]. HARD CONSTRAINTS THAT DO APPLY even to dense GQA, and they are model-shape rather than architecture-class: tensor-parallel size must divide the attention head count or the engine refuses to start [[mgpu-tp-size-must-divide-attention-heads]] (affects vllm and sglang); and a GQA KV cache is the thing paged-attention machinery exists for, so an engine with no paged KV (llama.cpp's slot-based pool [[llama-cpp]], exllamav2 [[exllamav2]]) loses the concurrency lever rather than the ability to run. ONE HARDWARE-DEPENDENT KERNEL GATE THAT IS NOT ABOUT GQA: on RDNA, vLLM's custom paged-attention kernel is gated on head_size == 128 AND block_size == 16, so a dense GQA model with head_dim 256 falls to the Triton 2D fallback and decode falls off a cliff - measured 12.1 tok/s at 518 tokens of context to 4.2 tok/s at 32k, with kernel_paged_attention_2d growing 28.3x [[rdna-triton-paged-attn-decode-cliff]]. Check head_dim before blaming the card. MHA SPECIFIC: gqa_ratio is 1.0, so the KV cache is num_layers x num_kv_heads x head_dim x 2 x 2 with num_kv_heads == num_attention_heads. There is no engine-side special case: MHA is the shape every paged-attention kernel was written for, so the join is strictly easier than for GQA and the only cost is the KV bytes themselves. Tensor parallelism is unchanged - the constraint is on the QUERY head count, so MHA with many heads is the most permissive case for head divisibility [[mgpu-tp-size-must-divide-attention-heads]]. WINDOW-INERT CAVEAT: its sliding_window 262144 exceeds any usable context, so the window is INERT. Do NOT apply the KV overpay table to it. CONTEXT_SCALING_METHOD (other): SCHEME RECLASSIFIED (2026-10-04): was free text 'LongRoPE' had no enum value. config rope_scaling type 'longrope' with long_factor/short_factor per-dimension arrays: LongRoPE is a per-dimension rescale on an NTK-aware base, which is neither 'linear' nor plain 'ntk-aware' in this enum. Recorded as 'other' rather than forced.