SGLang queue-time histogram
sglang · histogram
Fields
id
metric-exposures/w3m-sglang-sglang-queue-time-secondsMetric name
sglang:queue_time_seconds
Metric type
histogram
Unit
seconds
Exposure
prometheus
Endpoint
:30000/metrics
Histogram buckets
00.0010.0050.010.050.10.20.5123451015203040506070809010020030040050060070080090010001200140016001800200025003000
Labels
model_nameengine_typetp_rankpp_rankmoe_ep_rank
Enabled by default
no
Phase scoped
yes
notes
SGLang main, python/sglang/srt/observability/metrics_collector.py, read 2026-10-04. NOT ON BY DEFAULT: nothing is exported unless --enable-metrics (ServerArgs default False), which is also the only thing that mounts /metrics on the server port. Label keys are model_name, engine_type, tp_rank, pp_rank, moe_ep_rank (plus dp_rank under data parallelism, priority under priority scheduling, and anything in extra_metric_labels). Bucket ladders are overridable per histogram via get_histogram_conf_from_env. The finest-grained ladder in the file: 40 explicit bounds from 0.000 to 3000, dense below 1 s. Named queue_time_seconds, not request_queue_time_seconds - a vLLM-shaped scraper rule written against vLLM's name matches nothing here.
Notes
SGLang main, python/sglang/srt/observability/metrics_collector.py, read 2026-10-04. NOT ON BY DEFAULT: nothing is exported unless --enable-metrics (ServerArgs default False), which is also the only thing that mounts /metrics on the server port. Label keys are model_name, engine_type, tp_rank, pp_rank, moe_ep_rank (plus dp_rank under data parallelism, priority under priority scheduling, and anything in extra_metric_labels). Bucket ladders are overridable per histogram via get_histogram_conf_from_env. The finest-grained ladder in the file: 40 explicit bounds from 0.000 to 3000, dense below 1 s. Named queue_time_seconds, not request_queue_time_seconds - a vLLM-shaped scraper rule written against vLLM's name matches nothing here.