Mastering LLM Techniques: Inference Optimization

Vendor technical overview stating the two-phase picture directly: prefill processes input tokens in parallel, decode generates autoregressively and is memory-bound. Also documents KV-cache footprint growing linearly with batch x context and why PagedAttention enables larger batch sizes. Vendor blog: treat magnitudes as directional, the prefill/decode bound split is the transferable claim.