inference/research
verifiedhigh · 0.95updated 2026-10-05

vLLM v1/metrics/buckets.py - default Prometheus histogram bucket boundaries

repo

Fields

id
sources/w3m-vllm-buckets-module
Publisher
vllm-project/vllm
Kind
repo
Accessed
2026-10-05
notes
The single source of truth for every core vLLM histogram's default bucket bounds, read 2026-10-04. Seven static families with literal tuples: REQUEST_LATENCY_BUCKETS (21 bounds, 0.3 .. 7680.0), TIME_TO_FIRST_TOKEN_BUCKETS (22 bounds, 0.001 .. 2560.0), INTER_TOKEN_LATENCY_BUCKETS (19 bounds, 0.01 .. 80.0), ITERATION_TOKENS_BUCKETS (13 bounds, 1 .. 16384), REQUEST_PARAMS_N_BUCKETS (5 bounds), REQUEST_NUM_PREEMPTIONS_BUCKETS (7 bounds, 1 .. 20), KV_CACHE_RESIDENCY_BUCKETS (21 bounds, 0.001 .. 1800). The request_tokens family is NOT static: histogram_buckets() builds it with build_1_2_5_buckets(max_model_len), so its bounds depend on the served model's context length. BUCKET_FAMILY_KEYS is the set of eight keys --custom-histogram-buckets accepts. The module's own docstrings state the design intent of each family, which is why the boundaries are quotable rather than arbitrary. | 2026-10-05 PATH SWEEP: PATH SWEEP RESULT: NO REPAIR NEEDED. The same agent report that cast doubt on vllm/v1/metrics/loggers.py also implied buckets.py had moved. It has not: vllm/v1/metrics/buckets.py is present in the authenticated recursive tree for vllm@main. The bucket-boundary values quoted in this note are therefore still quotable from the file this record cites.

Notes

The single source of truth for every core vLLM histogram's default bucket bounds, read 2026-10-04. Seven static families with literal tuples: REQUEST_LATENCY_BUCKETS (21 bounds, 0.3 .. 7680.0), TIME_TO_FIRST_TOKEN_BUCKETS (22 bounds, 0.001 .. 2560.0), INTER_TOKEN_LATENCY_BUCKETS (19 bounds, 0.01 .. 80.0), ITERATION_TOKENS_BUCKETS (13 bounds, 1 .. 16384), REQUEST_PARAMS_N_BUCKETS (5 bounds), REQUEST_NUM_PREEMPTIONS_BUCKETS (7 bounds, 1 .. 20), KV_CACHE_RESIDENCY_BUCKETS (21 bounds, 0.001 .. 1800). The request_tokens family is NOT static: histogram_buckets() builds it with build_1_2_5_buckets(max_model_len), so its bounds depend on the served model's context length. BUCKET_FAMILY_KEYS is the set of eight keys --custom-histogram-buckets accepts. The module's own docstrings state the design intent of each family, which is why the boundaries are quotable rather than arbitrary. | 2026-10-05 PATH SWEEP: PATH SWEEP RESULT: NO REPAIR NEEDED. The same agent report that cast doubt on vllm/v1/metrics/loggers.py also implied buckets.py had moved. It has not: vllm/v1/metrics/buckets.py is present in the authenticated recursive tree for vllm@main. The bucket-boundary values quoted in this note are therefore still quotable from the file this record cites.

Cross-references

This record references no other records.