inference/research
verifiedhigh · 0.94updated 2026-10-04

SGLang utilization gauge - DOCUMENTED BUT PERMANENTLY ZERO ON MAIN

sglang · gauge

Fields

id
metric-exposures/w3m-sglang-sglang-utilization
Metric name
sglang:utilization
Metric type
gauge
Unit
1
Exposure
prometheus
Endpoint
:30000/metrics
Labels
model_nameengine_typetp_rankpp_rankmoe_ep_rank
Enabled by default
no
notes
Structurally dead on current main and this is a verified reading of the source, not an inference. metrics_reporter._calculate_utilization reads 'TODO: max_running_requests_under_SLO has no setter - sglang:utilization stuck at 0 (regressed #22713)' and only assigns when max_under_slo is not None and > 0, which it never is. Under --disaggregation-mode prefill the same method assigns -1 instead. So the series exists, is exported, and carries either 0 or -1. DO NOT build an autoscaling or saturation rule on it: on a prefill worker 'utilization above 1' would fire on the sentinel value -1 inverted, and elsewhere it never fires at all.

Notes

Structurally dead on current main and this is a verified reading of the source, not an inference. metrics_reporter._calculate_utilization reads 'TODO: max_running_requests_under_SLO has no setter - sglang:utilization stuck at 0 (regressed #22713)' and only assigns when max_under_slo is not None and > 0, which it never is. Under --disaggregation-mode prefill the same method assigns -1 instead. So the series exists, is exported, and carries either 0 or -1. DO NOT build an autoscaling or saturation rule on it: on a prefill worker 'utilization above 1' would fire on the sentinel value -1 inverted, and elsewhere it never fires at all.

Cross-references