inference/research
verifiedhigh · 0.90updated 2026-10-04

vLLM SharedAdmissionStats source (lock-free cross-process admission counters)

repo

Fields

id
sources/ops-vllm-shared-admission-stats
Publisher
vLLM project
Kind
repo
Published
main branch as of 2026-10-04
Accessed
2026-10-04
notes
SharedAdmissionStats is documented as 'Lock-free admission stats shared by API server processes. Each process writes exclusively to its own cache-line-sized slot. Readers aggregate the slots, avoiding both inter-process locks and cache-line contention on the request path.' Constants CACHE_LINE_BYTES=64, COUNTER_BYTES=8, COUNTERS_PER_SLOT=8, with the comment that spacing is 'to avoid false sharing on CPUs with 64-byte cache lines: one process writes would otherwise invalidate the cache line containing another process counter.' Relevant to multi-process API server deployments: the admission counter lives in shared memory as one slot per process, so it is process-count dependent and never a durable or cluster-wide quantity.

Notes

SharedAdmissionStats is documented as 'Lock-free admission stats shared by API server processes. Each process writes exclusively to its own cache-line-sized slot. Readers aggregate the slots, avoiding both inter-process locks and cache-line contention on the request path.' Constants CACHE_LINE_BYTES=64, COUNTER_BYTES=8, COUNTERS_PER_SLOT=8, with the comment that spacing is 'to avoid false sharing on CPUs with 64-byte cache lines: one process writes would otherwise invalidate the cache line containing another process counter.' Relevant to multi-process API server deployments: the admission counter lives in shared memory as one slot per process, so it is process-count dependent and never a durable or cluster-wide quantity.

Cross-references

This record references no other records.