Triton request-duration summary
triton-inference-server · summary
Fields
id
metric-exposures/w3m-triton-inference-server-nv-inference-request-summary-usMetric name
nv_inference_request_summary_us
Metric type
summary
Unit
microseconds
Exposure
prometheus
Endpoint
:8002/metrics
Labels
modelversion
Enabled by default
no
notes
Triton main, docs/user_guide/metrics.md, read 2026-10-04. On by default: 'By default, these metrics are available at http://localhost:8002/metrics', and the endpoint is pure pull - 'not pushed or published to any remote server'. tritonserver --allow-metrics=false turns the whole thing off. Labels are model and version, and crucially the LATENCY families are per MODEL, per request-update - not per-request observations as in vLLM/SGLang/TGI. OFF BY DEFAULT - requires --metrics-config. Opt-in via --metrics-config summary_latencies=true. Gives quantiles the default counter form cannot, at the cost of the bucket ladder.
Notes
Triton main, docs/user_guide/metrics.md, read 2026-10-04. On by default: 'By default, these metrics are available at http://localhost:8002/metrics', and the endpoint is pure pull - 'not pushed or published to any remote server'. tritonserver --allow-metrics=false turns the whole thing off. Labels are model and version, and crucially the LATENCY families are per MODEL, per request-update - not per-request observations as in vLLM/SGLang/TGI. OFF BY DEFAULT - requires --metrics-config. Opt-in via --metrics-config summary_latencies=true. Gives quantiles the default counter form cannot, at the cost of the bucket ladder.