Gemma 1 7B

CONTRAST RECORD, kept because Gemma 1's serving pathology is the reason Gemma 2 exists. Reading the config against Gemma 2: model_type 'gemma' not 'gemma2'; num_key_value_heads 16 equals num_attention_heads 16, i.e. plain MHA with no GQA at all; NO attn_logit_softcapping key; NO sliding_window key; query_pre_attn_scalar absent. head_dim 256 is stated explicitly. intermediate 24576. vocab 256000. kv_cache_bytes_per_token = 28 * 16 * 256 * 2 * 2 = 458752 bytes/token at bf16 — 4.3x the Gemma 2 9B figure of 344064 on almost identical parameters. That ungated, unsoftcapped, un-windowed attention is the mechanism behind Gemma 1's well-documented tendency to emit enormous logits that destabilize greedy decode: without softcapping, attention logits grow with sequence length and a small numerical difference flips the argmax. Serve Gemma 2 instead. params_b = 8.538 is the safetensors total from the unsloth mirror (8537680896); google/gemma-1-7b is gated and returned 401 so its own metadata could not be read directly. This is the one field in this record not confirmed against the vendor repo. SERVICEABILITY JOIN (added 2026-10-03; the architecture x engine matrix this record was missing, with a citation per cell in docs/13-model-compatibility.md): FAMILY=dense-mha. REQUIRED KERNEL CLASS: ordinary causal attention plus GEMM/GEMV. This is the baseline every engine in this repo implements, so the join here is a floor rather than a filter: no engine record in data/engines/ states that dense GQA is unsupported. SUPPORTED BY (cited): vLLM LlamaForCausalLM / Qwen2 / Qwen3 / Mistral / Phi3ForCausalLM rows [[mdl2-vllm-supported-models-2026-10]]; TensorRT-LLM LlamaForCausalLM, Qwen2ForCausalLM, Qwen3ForCausalLM, Phi3ForCausalLM rows [[mdl2-trtllm-supported-models]]; SGLang generative table (Llama, Qwen, Mistral families) [[mdl2-sglang-generative-models]]; llama.cpp LLM_ARCH_LLAMA [[pap-arch-llamacpp-arch-registry]]. HARD CONSTRAINTS THAT DO APPLY even to dense GQA, and they are model-shape rather than architecture-class: tensor-parallel size must divide the attention head count or the engine refuses to start [[mgpu-tp-size-must-divide-attention-heads]] (affects vllm and sglang); and a GQA KV cache is the thing paged-attention machinery exists for, so an engine with no paged KV (llama.cpp's slot-based pool [[llama-cpp]], exllamav2 [[exllamav2]]) loses the concurrency lever rather than the ability to run. ONE HARDWARE-DEPENDENT KERNEL GATE THAT IS NOT ABOUT GQA: on RDNA, vLLM's custom paged-attention kernel is gated on head_size == 128 AND block_size == 16, so a dense GQA model with head_dim 256 falls to the Triton 2D fallback and decode falls off a cliff - measured 12.1 tok/s at 518 tokens of context to 4.2 tok/s at 32k, with kernel_paged_attention_2d growing 28.3x [[rdna-triton-paged-attn-decode-cliff]]. Check head_dim before blaming the card. MHA SPECIFIC: gqa_ratio is 1.0, so the KV cache is num_layers x num_kv_heads x head_dim x 2 x 2 with num_kv_heads == num_attention_heads. There is no engine-side special case: MHA is the shape every paged-attention kernel was written for, so the join is strictly easier than for GQA and the only cost is the KV bytes themselves. Tensor parallelism is unchanged - the constraint is on the QUERY head count, so MHA with many heads is the most permissive case for head divisibility [[mgpu-tp-size-must-divide-attention-heads]]. CONTEXT_SCALING_METHOD (native): rope_scaling null at theta 1e4 over 8192.