obs-triton-metrics-doc · inference-research

Fields

id

sources/obs-triton-metrics-doc

URL

https://github.com/triton-inference-server/server/blob/main/docs/user_guide/metrics.md

Publisher

triton-inference-server

Kind

repo

Accessed

2026-10-04

notes

The authoritative Triton metric list. Key finding for LLM work: the core server is request-shaped, not token-shaped. There is nv_inference_first_response_histogram_ms (first response, not first token) but no TTFT, no TPOT, no KV-cache-usage metric - those arrive only if a backend registers them through the custom metrics API.

Notes

The authoritative Triton metric list. Key finding for LLM work: the core server is request-shaped, not token-shaped. There is nv_inference_first_response_histogram_ms (first response, not first token) but no TTFT, no TPOT, no KV-cache-usage metric - those arrive only if a backend registers them through the custom metrics API.

Cross-references

This record references no other records.