inference/research
verifiedhigh · 0.90updated 2026-10-04

TensorRT-LLM metrics/collector.py - zero preemption metrics, confirmed by search

repo

Fields

id
sources/w3m-trtllm-metrics-no-preempt-search
Publisher
NVIDIA/TensorRT-LLM
Kind
repo
Accessed
2026-10-04
notes
tensorrt_llm/metrics/collector.py on main, read 2026-10-04 (979 lines). A case-insensitive count of 'preempt' in this file is 0. The collector's own class docstring (lines 44-112) enumerates every metric it creates, and the enumeration contains trtllm_num_paused_requests but no preemption, preemption-counter or context-removed metric - so the absence is confirmed both by a zero-hit search over the implementation and by the implementation's own self-declared inventory. trtllm_num_paused_requests is the closest available signal and it is a level, not a count: it tells you how many requests are paused right now, not how many have been preempted and recomputed.

Notes

tensorrt_llm/metrics/collector.py on main, read 2026-10-04 (979 lines). A case-insensitive count of 'preempt' in this file is 0. The collector's own class docstring (lines 44-112) enumerates every metric it creates, and the enumeration contains trtllm_num_paused_requests but no preemption, preemption-counter or context-removed metric - so the absence is confirmed both by a zero-hit search over the implementation and by the implementation's own self-declared inventory. trtllm_num_paused_requests is the closest available signal and it is a level, not a count: it tells you how many requests are paused right now, not how many have been preempted and recomputed.

Cross-references

This record references no other records.