obs-vllm-metrics-design-doc · inference-research

Fields

id

sources/obs-vllm-metrics-design-doc

URL

https://github.com/vllm-project/vllm/blob/main/docs/design/metrics.md

Publisher

vllm-project/vllm

Kind

repo

Accessed

2026-10-04

notes

The project's own metrics design note. Contains the explicit ITL-vs-per-request-TPOT difference, the deprecation policy (hidden one version, removed two), the list of removed legacy metrics (vllm:num_requests_swapped, vllm:cpu_cache_usage_perc, vllm:avg_prompt_throughput_toks_per_s), and the argument that colons in metric names are contrary to Prometheus convention.

Notes

The project's own metrics design note. Contains the explicit ITL-vs-per-request-TPOT difference, the deprecation policy (hidden one version, removed two), the list of removed legacy metrics (vllm:num_requests_swapped, vllm:cpu_cache_usage_perc, vllm:avg_prompt_throughput_toks_per_s), and the argument that colons in metric names are contrary to Prometheus convention.

Cross-references

This record references no other records.