Triton Inference Server: a time-per-output-token / inter-token latency metric - VERIFIED ABSENT
Fields
metric-exposures/w3m-triton-inference-server-no-nv-inference-time-per-output-tokeNotes
VERIFIED NEGATIVE. Triton Inference Server does not emit a time-per-output-token / inter-token latency metric. nv_inference_time_per_output_token is NOT a metric this engine registers - it is recorded here under the conventional name a reader would go looking for so the absence is queryable rather than implied by a gap in a list. No decode-rate metric of any kind exists in Triton core. Token-shaped latency has to come from the backend's custom metrics API, which for the TRT-LLM backend means a label-switched gauge family rather than a named metric. What was searched: the docs' metric tables contain no token-rate or inter-token family; the custom-metrics section points at the TensorRT-LLM and vLLM backends for LLM-specific metrics.. Do not read this record's metric_name as something the engine exports; it is the name that is missing.