TensorRT-LLM: a waiting-request breakdown by cause - VERIFIED ABSENT
Fields
metric-exposures/w3m-tensorrt-llm-no-trtllm-num-requests-waiting-by-reasonNotes
VERIFIED NEGATIVE. TensorRT-LLM does not emit a waiting-request breakdown by cause. trtllm_num_requests_waiting_by_reason is NOT a metric this engine registers - it is recorded here under the conventional name a reader would go looking for so the absence is queryable rather than implied by a gap in a list. trtllm_num_requests_waiting is undifferentiated. There is no reason label, so a waiting queue cannot be attributed to capacity versus some transient constraint. The Triton backend's nv_trt_llm_request_metrics{request_type="waiting"} is equally undifferentiated. What was searched: the collector's docstring enumeration contains trtllm_num_requests_running and trtllm_num_requests_waiting with no reason-labelled variant; grep for 'reason' in the collector yields only finished_reason, cache_level, cache_tier, token_position and http_code.. Do not read this record's metric_name as something the engine exports; it is the name that is missing.