inference/research
verifiedhigh · 0.90updated 2026-10-04

Triton Inference Server: a preemption metric - VERIFIED ABSENT

triton-inference-server · unknown

Fields

id
metric-exposures/w3m-triton-inference-server-no-nv-inference-preemptions
Metric name
nv_inference_preemptions
Metric type
unknown
Exposure
unknown
Verified negatively
yes
notes
VERIFIED NEGATIVE. Triton Inference Server does not emit a preemption metric. nv_inference_preemptions is NOT a metric this engine registers - it is recorded here under the conventional name a reader would go looking for so the absence is queryable rather than implied by a gap in a list. Nothing in Triton core schedules sequences, so it cannot observe preemption; and the TRT-LLM backend's custom families (request_metrics, kv_cache_block_metrics, inflight_batcher_metrics, general_metrics, disaggregated_serving_metrics, runtime_memory_metrics) do not include one either. What was searched: grep for preempt across the docs metric tables and the tensorrtllm_backend README's sample exposition output returns no series.. Do not read this record's metric_name as something the engine exports; it is the name that is missing.

Notes

VERIFIED NEGATIVE. Triton Inference Server does not emit a preemption metric. nv_inference_preemptions is NOT a metric this engine registers - it is recorded here under the conventional name a reader would go looking for so the absence is queryable rather than implied by a gap in a list. Nothing in Triton core schedules sequences, so it cannot observe preemption; and the TRT-LLM backend's custom families (request_metrics, kv_cache_block_metrics, inflight_batcher_metrics, general_metrics, disaggregated_serving_metrics, runtime_memory_metrics) do not include one either. What was searched: grep for preempt across the docs metric tables and the tensorrtllm_backend README's sample exposition output returns no series.. Do not read this record's metric_name as something the engine exports; it is the name that is missing.

Cross-references