Efficient Memory Management for Large Language Model Serving with PagedAttention

Fields

id

sources/flop-pagedattention-vllm

URL

https://arxiv.org/abs/2309.06180

Publisher

arXiv (Kwon et al.; vLLM / UC Berkeley)

Kind

paper

Published

2023-09-12

Accessed

2026-10-03

notes

Paging-inspired KV cache: fixed-size blocks, near-zero waste, flexible sharing within and across requests. Reports 2-4x throughput improvement at same latency vs FasterTransformer and Orca, more pronounced at longer sequences and larger models.

Notes

Paging-inspired KV cache: fixed-size blocks, near-zero waste, flexible sharing within and across requests. Reports 2-4x throughput improvement at same latency vs FasterTransformer and Orca, more pronounced at longer sequences and larger models.

Cross-references

This record references no other records.