inference/research
verifiedhigh · 0.90updated 2026-10-05

Xinference model request duration histogram

xinference · histogram

Fields

id
metric-exposures/w5e-xinference-xinference-model-request-duration-seconds
Metric name
xinference:model_request_duration_seconds
Metric type
histogram
Unit
seconds
Exposure
prometheus
Endpoint
:9090/metrics (only when a metrics exporter host/port is set)
Histogram buckets
0.010.0250.050.10.250.512.55103060120
Enabled by default
no
Phase scoped
no
notes
Whole-request wall time (queue + inference), not a phase breakdown. 13 finite buckets plus float('inf'). Because the tail is dense in the sub-second range and the finite ladder stops at 120 s, a slow-but-not-pathological request lands in the overflow bucket and the histogram cannot distinguish a 3-minute request from a 30-minute one - a percentile above 120 s is not computable from this ladder. Xinference main (xorbitsai/inference), read 2026-10-05. NOT ON BY DEFAULT on the worker: the /metrics endpoint is only mounted when XINFERENCE_DISABLE_METRICS is unset AND a metrics exporter host or port is supplied; otherwise launch_metrics_export_server() is never called and the endpoint does not exist. Supervisor-side metrics are deregistered from the worker's registry on purpose (launch_metrics_export_server removes _SUPERVISOR_ONLY_METRICS), so the worker endpoint carries only worker metrics and the supervisor endpoint only supervisor metrics. aioprometheus, not prometheus_client: counters do not auto-suffix _total the way prometheus_client does, so two of these names end in _counter and three do not - match on the literal name.

Notes

Whole-request wall time (queue + inference), not a phase breakdown. 13 finite buckets plus float('inf'). Because the tail is dense in the sub-second range and the finite ladder stops at 120 s, a slow-but-not-pathological request lands in the overflow bucket and the histogram cannot distinguish a 3-minute request from a 30-minute one - a percentile above 120 s is not computable from this ladder. Xinference main (xorbitsai/inference), read 2026-10-05. NOT ON BY DEFAULT on the worker: the /metrics endpoint is only mounted when XINFERENCE_DISABLE_METRICS is unset AND a metrics exporter host or port is supplied; otherwise launch_metrics_export_server() is never called and the endpoint does not exist. Supervisor-side metrics are deregistered from the worker's registry on purpose (launch_metrics_export_server removes _SUPERVISOR_ONLY_METRICS), so the worker endpoint carries only worker metrics and the supervisor endpoint only supervisor metrics. aioprometheus, not prometheus_client: counters do not auto-suffix _total the way prometheus_client does, so two of these names end in _counter and three do not - match on the literal name.

Cross-references