Triton Inference Server: a time-to-first-token metric - VERIFIED ABSENT
Fields
metric-exposures/w3m-triton-inference-server-no-nv-inference-time-to-first-tokenNotes
VERIFIED NEGATIVE. Triton Inference Server does not emit a time-to-first-token metric. nv_inference_time_to_first_token is NOT a metric this engine registers - it is recorded here under the conventional name a reader would go looking for so the absence is queryable rather than implied by a gap in a list. The core server is request-shaped, not token-shaped. There is nv_inference_first_response_histogram_ms, which is close enough in name to be a trap: for a streaming LLM backend it measures time to the first delivered CHUNK, not the first decoded token, and for a non-streaming backend it is essentially the whole request. A reader who assumes it is TTFT builds a dashboard that silently measures something else. What was searched: the full metric list transcribed from the docs (31 distinct nv_* names) contains no first-token family; the only first-response family is nv_inference_first_response_histogram_ms.. Do not read this record's metric_name as something the engine exports; it is the name that is missing.