inference/research
verifiedhigh · 0.88updated 2026-10-05

Xinference: an ITL histogram under the vLLM/SGLang name - VERIFIED ABSENT

xinference · unknown

Fields

id
metric-exposures/w5e-xinference-no-xinference-inter-token-latency-seconds
Metric name
xinference:inter_token_latency_seconds
Metric type
unknown
Exposure
unknown
Verified negatively
yes
notes
VERIFIED NEGATIVE. Xinference does not emit an inter-token-latency histogram. xinference:inter_token_latency_seconds is NOT a metric this engine declares - it is recorded here under the conventional name a reader migrating from vLLM or SGLang would go looking for, so the absence is queryable rather than implied by a gap in a list. Of the 73 metric names declared in xinference/core/metrics.py, exactly two are histograms (time_to_first_token_seconds and model_request_duration_seconds) and neither is a per-token or per-step measure. The consequence is concrete: on Xinference you cannot tell an ITL regression from a TTFT regression, because only TTFT is measured. What was searched: all 73 declarations parsed from core/metrics.py; the string 'token' appears only in generate_tokens_total, input_tokens_total_counter and output_tokens_total_counter, all counters. Do not read this record's metric_name as something the engine exports; it is the name that is missing.

Notes

VERIFIED NEGATIVE. Xinference does not emit an inter-token-latency histogram. xinference:inter_token_latency_seconds is NOT a metric this engine declares - it is recorded here under the conventional name a reader migrating from vLLM or SGLang would go looking for, so the absence is queryable rather than implied by a gap in a list. Of the 73 metric names declared in xinference/core/metrics.py, exactly two are histograms (time_to_first_token_seconds and model_request_duration_seconds) and neither is a per-token or per-step measure. The consequence is concrete: on Xinference you cannot tell an ITL regression from a TTFT regression, because only TTFT is measured. What was searched: all 73 declarations parsed from core/metrics.py; the string 'token' appears only in generate_tokens_total, input_tokens_total_counter and output_tokens_total_counter, all counters. Do not read this record's metric_name as something the engine exports; it is the name that is missing.

Cross-references