obs-vllm-per-request-metrics-doc · inference-research

Fields

id

sources/obs-vllm-per-request-metrics-doc

URL

https://github.com/vllm-project/vllm/blob/main/docs/features/per_request_metrics.md

Publisher

vllm-project/vllm

Kind

repo

Accessed

2026-10-04

notes

--enable-per-request-metrics returns a metrics object in the response body with time_to_first_token_ms, generation_time_ms, queue_time_ms, mean_itl_ms, tokens_per_second. Documents that TTFT is measured from scheduling (not arrival) and that the object is null for n>1.

Notes

--enable-per-request-metrics returns a metrics object in the response body with time_to_first_token_ms, generation_time_ms, queue_time_ms, mean_itl_ms, tokens_per_second. Documents that TTFT is measured from scheduling (not arrival) and that the object is null for n>1.

Cross-references

This record references no other records.