Gemma 3 Technical Report (Gemma Team, Google DeepMind, arXiv:2503.19786)

v1, 25 Mar 2025. Section 2: 5 local sliding-window layers per 1 global layer, local span 1024 tokens. Table 3 reports memory footprints in GB for raw bf16 and quantized checkpoints with and without a KV cache for a sequence of 32,768 tokens, KV cache quantized in 8 bits: 4B raw bf16 8.0 GB, +KV 12.7 GB. Section 5.2: a 'global only' 2B ablation has a 60% KV memory overhead at 32k, reduced to less than 15% with a 1:3 local:global ratio and sliding window 1024; perplexity is described as minimally affected by the ratio. HTML and PDF both read 2026-10-18.