OLMo 2 1B

CONTEXT SCALING: null is a VERIFIED value, checked against config rather than left blank. config.json declares rope_scaling null, rope_theta 500000, max_position_embeddings 4096. ARCHITECTURE CORRECTION worth recording: OLMo-2 1B and 7B are DENSE, not MoE. config.json is model_type 'olmo2' with 16 layers, hidden 2048, num_attention_heads 16 = num_key_value_heads 16, and NO num_experts / num_local_experts / experts_per_token keys and no MoE architecture string. The MoE variants of OLMo-2 are the 32B and 64B (allenai/OLMo-2-0325-32B: 64 layers, hidden 5120, 40 query / 8 KV heads) — the 1B and 7B are not. Anyone budgeting VRAM from a 'OLMo-2 is MoE' assumption is wrong on exactly the models that fit on a laptop. head_dim derived = 2048 / 16 = 128. kv_cache_bytes_per_token = 16 * 16 * 128 * 2 * 2 = 131072 bytes/token at bf16 — identical to Phi-4-mini-instruct despite this model being less than half its size, because MHA at 16 heads costs as much as GQA at 8 heads with a double head_dim. At OLMo-2's native 4096 context that is 512 MiB of KV per sequence, comfortable; the same shape at 32k would be 4 GiB. 4096 context is genuinely the limit here (rope_scaling null, no extension), unlike the 128k Phi-4 mini. vocab 100352, intermediate 8192, rope_theta 500000, untied embeddings. Repo default torch_dtype is float32 — quantized at load, not bf16. Apache-2.0, fully open: GGUF and ONNX ports both exist (allenai/OLMo-2-0425-1B-Instruct-GGUF, liodon-ai/OLMo-2-0425-1B-ONNX). SERVICEABILITY JOIN (added 2026-10-03; the architecture x engine matrix this record was missing, with a citation per cell in docs/13-model-compatibility.md): FAMILY=dense-mha. REQUIRED KERNEL CLASS: ordinary causal attention plus GEMM/GEMV. This is the baseline every engine in this repo implements, so the join here is a floor rather than a filter: no engine record in data/engines/ states that dense GQA is unsupported. SUPPORTED BY (cited): vLLM LlamaForCausalLM / Qwen2 / Qwen3 / Mistral / Phi3ForCausalLM rows [[mdl2-vllm-supported-models-2026-10]]; TensorRT-LLM LlamaForCausalLM, Qwen2ForCausalLM, Qwen3ForCausalLM, Phi3ForCausalLM rows [[mdl2-trtllm-supported-models]]; SGLang generative table (Llama, Qwen, Mistral families) [[mdl2-sglang-generative-models]]; llama.cpp LLM_ARCH_LLAMA [[pap-arch-llamacpp-arch-registry]]. HARD CONSTRAINTS THAT DO APPLY even to dense GQA, and they are model-shape rather than architecture-class: tensor-parallel size must divide the attention head count or the engine refuses to start [[mgpu-tp-size-must-divide-attention-heads]] (affects vllm and sglang); and a GQA KV cache is the thing paged-attention machinery exists for, so an engine with no paged KV (llama.cpp's slot-based pool [[llama-cpp]], exllamav2 [[exllamav2]]) loses the concurrency lever rather than the ability to run. ONE HARDWARE-DEPENDENT KERNEL GATE THAT IS NOT ABOUT GQA: on RDNA, vLLM's custom paged-attention kernel is gated on head_size == 128 AND block_size == 16, so a dense GQA model with head_dim 256 falls to the Triton 2D fallback and decode falls off a cliff - measured 12.1 tok/s at 518 tokens of context to 4.2 tok/s at 32k, with kernel_paged_attention_2d growing 28.3x [[rdna-triton-paged-attn-decode-cliff]]. Check head_dim before blaming the card. MHA SPECIFIC: gqa_ratio is 1.0, so the KV cache is num_layers x num_kv_heads x head_dim x 2 x 2 with num_kv_heads == num_attention_heads. There is no engine-side special case: MHA is the shape every paged-attention kernel was written for, so the join is strictly easier than for GQA and the only cost is the KV bytes themselves. Tensor parallelism is unchanged - the constraint is on the QUERY head count, so MHA with many heads is the most permissive case for head divisibility [[mgpu-tp-size-must-divide-attention-heads]]. CONTEXT_SCALING_METHOD (native): rope_scaling null at theta 500000 over 4096.