inference/research
verifiedhigh · 0.92updated 2026-10-04

TGI: a preemption or eviction counter - VERIFIED ABSENT

text-generation-inference · unknown

Fields

id
metric-exposures/w3m-text-generation-inference-no-tgi-preemption-or-eviction-counter
Metric name
tgi_preemption_or_eviction_counter
Metric type
unknown
Exposure
unknown
Verified negatively
yes
notes
VERIFIED NEGATIVE. TGI does not emit a preemption or eviction counter. tgi_preemption_or_eviction_counter is NOT a metric this engine registers - it is recorded here under the conventional name a reader would go looking for so the absence is queryable rather than implied by a gap in a list. The v3 backend can drop a request when its queue overflows and can shrink batches under backpressure, and both are observable - tgi_request_failure{err="dropped"} and tgi_batch_concat{reason="backpressure"|"chunking"|"wait_exceeded"} - but those are ADMISSION outcomes. The recompute cost of a preempted sequence is invisible, so the saturation story stops at 'the system refused work' with no measure of what that work would have cost. What was searched: the 27-name metric inventory across the router and v3 backend contains no preemption/eviction metric; grep for 'preempt' across the same files returns 0 hits.. Do not read this record's metric_name as something the engine exports; it is the name that is missing.

Notes

VERIFIED NEGATIVE. TGI does not emit a preemption or eviction counter. tgi_preemption_or_eviction_counter is NOT a metric this engine registers - it is recorded here under the conventional name a reader would go looking for so the absence is queryable rather than implied by a gap in a list. The v3 backend can drop a request when its queue overflows and can shrink batches under backpressure, and both are observable - tgi_request_failure{err="dropped"} and tgi_batch_concat{reason="backpressure"|"chunking"|"wait_exceeded"} - but those are ADMISSION outcomes. The recompute cost of a preempted sequence is invisible, so the saturation story stops at 'the system refused work' with no measure of what that work would have cost. What was searched: the 27-name metric inventory across the router and v3 backend contains no preemption/eviction metric; grep for 'preempt' across the same files returns 0 hits.. Do not read this record's metric_name as something the engine exports; it is the name that is missing.

Cross-references