inference/research
verifiedhigh · 0.90updated 2026-10-05

Xinference in-flight request gauge

xinference · gauge

Fields

id
metric-exposures/w5e-xinference-xinference-model-serve-count
Metric name
xinference:model_serve_count
Metric type
gauge
Unit
requests
Exposure
prometheus
Endpoint
:9090/metrics (only when a metrics exporter host/port is set)
Enabled by default
no
notes
Level, not a rate: number of requests currently being served by the model. This is the denominator for concurrency, and the signal that distinguishes 'the box is saturated' from 'the box is idle and slow'. Xinference main (xorbitsai/inference), read 2026-10-05. NOT ON BY DEFAULT on the worker: the /metrics endpoint is only mounted when XINFERENCE_DISABLE_METRICS is unset AND a metrics exporter host or port is supplied; otherwise launch_metrics_export_server() is never called and the endpoint does not exist. Supervisor-side metrics are deregistered from the worker's registry on purpose (launch_metrics_export_server removes _SUPERVISOR_ONLY_METRICS), so the worker endpoint carries only worker metrics and the supervisor endpoint only supervisor metrics. aioprometheus, not prometheus_client: counters do not auto-suffix _total the way prometheus_client does, so two of these names end in _counter and three do not - match on the literal name.

Notes

Level, not a rate: number of requests currently being served by the model. This is the denominator for concurrency, and the signal that distinguishes 'the box is saturated' from 'the box is idle and slow'. Xinference main (xorbitsai/inference), read 2026-10-05. NOT ON BY DEFAULT on the worker: the /metrics endpoint is only mounted when XINFERENCE_DISABLE_METRICS is unset AND a metrics exporter host or port is supplied; otherwise launch_metrics_export_server() is never called and the endpoint does not exist. Supervisor-side metrics are deregistered from the worker's registry on purpose (launch_metrics_export_server removes _SUPERVISOR_ONLY_METRICS), so the worker endpoint carries only worker metrics and the supervisor endpoint only supervisor metrics. aioprometheus, not prometheus_client: counters do not auto-suffix _total the way prometheus_client does, so two of these names end in _counter and three do not - match on the literal name.

Cross-references