vLLM per-request time-per-output-token (TPOT) histogram
vllm · histogram
Fields
id
metric-exposures/w3m-vllm-vllm-request-time-per-output-token-secondsMetric name
vllm:request_time_per_output_token_seconds
Metric type
histogram
Unit
seconds
Exposure
prometheus
Endpoint
:8000/metrics
Histogram buckets
0.010.0250.050.0750.10.150.20.30.40.50.7512.557.510204080
Labels
model_nameengine
Enabled by default
yes
Phase scoped
yes
notes
vLLM main, PrometheusStatLogger in vllm/v1/metrics/loggers.py, read 2026-10-04. On by default (EngineArgs.disable_log_stats defaults False); served at :8000/metrics. Bucket family name in --custom-histogram-buckets: inter_token_latency (shared ladder). One sample per finished request, so it is the right basis for a per-user TPOT SLO where inter_token_latency is the right basis for a fleet stall detector.
Notes
vLLM main, PrometheusStatLogger in vllm/v1/metrics/loggers.py, read 2026-10-04. On by default (EngineArgs.disable_log_stats defaults False); served at :8000/metrics. Bucket family name in --custom-histogram-buckets: inter_token_latency (shared ladder). One sample per finished request, so it is the right basis for a per-user TPOT SLO where inter_token_latency is the right basis for a fleet stall detector.