inference/research
verifiedhigh · 0.95updated 2026-10-04

vLLM v1 metrics loggers source (Prometheus metric names and documentation strings)

repo

Fields

id
sources/ops-vllm-metrics-loggers
Publisher
vLLM project
Kind
repo
Published
main branch as of 2026-10-04
Accessed
2026-10-04
notes
Verbatim documentation strings for the operationally relevant signals: vllm:num_requests_running = 'Number of requests in model execution batches.' vllm:num_requests_waiting = 'Number of requests waiting to be processed.' vllm:num_requests_waiting_by_reason = 'Number of waiting requests by reason. Reason labels: capacity = waiting for scheduling capacity; deferred = deferred by transient constraints (LoRA budget, KV transfer, blocked status). Sum of all reasons equals vllm:num_requests_waiting.' vllm:kv_cache_usage_perc = 'KV-cache usage. 1 means 100 percent usage.' vllm:num_preemptions = 'Cumulative number of preemption from the engine.' vllm:engine_sleep_state = 'Engine sleep state; awake = 0 means engine is sleeping; awake = 1 means engine is awake; weights_offloaded = 1 means sleep level 1; discard_all = 1 means sleep level 2.' vllm:corrupted_requests = 'Corrupted requests, in terms of total number of requests with NaNs in logits', registered only when VLLM_COMPUTE_NANS_IN_LOGITS is set. vllm:num_requests_kv_fetch_by_stage carries stage labels waiting_to_start, in_progress, completed_waiting. The scheduler-state gauges use multiprocess_mode='mostrecent'. The periodic operator-facing log line prints Running and Waiting (waiting plus skipped), Deferred, KV fetch stages, a Preemptions count when non-zero, and 'GPU KV cache usage: %.1f%%'. Prefill KV computed tokens are logged as num_prompt_tokens minus the cached prefix.

Notes

Verbatim documentation strings for the operationally relevant signals: vllm:num_requests_running = 'Number of requests in model execution batches.' vllm:num_requests_waiting = 'Number of requests waiting to be processed.' vllm:num_requests_waiting_by_reason = 'Number of waiting requests by reason. Reason labels: capacity = waiting for scheduling capacity; deferred = deferred by transient constraints (LoRA budget, KV transfer, blocked status). Sum of all reasons equals vllm:num_requests_waiting.' vllm:kv_cache_usage_perc = 'KV-cache usage. 1 means 100 percent usage.' vllm:num_preemptions = 'Cumulative number of preemption from the engine.' vllm:engine_sleep_state = 'Engine sleep state; awake = 0 means engine is sleeping; awake = 1 means engine is awake; weights_offloaded = 1 means sleep level 1; discard_all = 1 means sleep level 2.' vllm:corrupted_requests = 'Corrupted requests, in terms of total number of requests with NaNs in logits', registered only when VLLM_COMPUTE_NANS_IN_LOGITS is set. vllm:num_requests_kv_fetch_by_stage carries stage labels waiting_to_start, in_progress, completed_waiting. The scheduler-state gauges use multiprocess_mode='mostrecent'. The periodic operator-facing log line prints Running and Waiting (waiting plus skipped), Deferred, KV fetch stages, a Preemptions count when non-zero, and 'GPU KV cache usage: %.1f%%'. Prefill KV computed tokens are logged as num_prompt_tokens minus the cached prefix.

Cross-references

This record references no other records.