TGI failed-request counter by error class
text-generation-inference · counter
Fields
id
metric-exposures/w3m-text-generation-inference-tgi-request-failureMetric name
tgi_request_failure
Metric type
counter
Unit
requests
Exposure
prometheus
Endpoint
:9000/metrics
Labels
err
Enabled by default
yes
notes
TGI main, router/src/server.rs plus backends/v3/src/{backend,queue,lib}.rs, read 2026-10-04. On by default: the Prometheus recorder is built unconditionally with PrometheusBuilder and an HTTP listener on prometheus_port, default 9000, so nothing has to be opted into. Label keys are method (prefill|decode) on the batch families, err on tgi_request_failure, and reason on tgi_batch_concat - there are no model or engine labels, so every series is fleet-wide. err in {validation, incomplete, dropped}. This is also the closest TGI comes to a preemption signal - a dropped request is an admission failure, and the recompute cost is never counted.
Notes
TGI main, router/src/server.rs plus backends/v3/src/{backend,queue,lib}.rs, read 2026-10-04. On by default: the Prometheus recorder is built unconditionally with PrometheusBuilder and an HTTP listener on prometheus_port, default 9000, so nothing has to be opted into. Label keys are method (prefill|decode) on the batch families, err on tgi_request_failure, and reason on tgi_batch_concat - there are no model or engine labels, so every series is fleet-wide. err in {validation, incomplete, dropped}. This is also the closest TGI comes to a preemption signal - a dropped request is an admission failure, and the recompute cost is never counted.