inference/research
verifiedhigh · 0.94updated 2026-10-04

SGLang per-stage request latency histogram (stage label)

sglang · histogram

Fields

id
metric-exposures/w3m-sglang-sglang-per-stage-req-latency-seconds
Metric name
sglang:per_stage_req_latency_seconds
Metric type
histogram
Unit
seconds
Exposure
prometheus
Endpoint
:30000/metrics
Histogram buckets
0.0010.001620.0026240.0042520.0068870.0111580.0180750.0292820.0474370.0768480.1244940.2016810.3267230.5292920.8574531.3890732.2502993.6454845.9056859.56720915.49887925.10818440.67525865.893919106.748148172.932280.14984453.84274735.2252391191.064887
Labels
model_nameengine_typetp_rankpp_rankmoe_ep_rankstage
Enabled by default
no
Phase scoped
yes
notes
SGLang main, python/sglang/srt/observability/metrics_collector.py, read 2026-10-04. NOT ON BY DEFAULT: nothing is exported unless --enable-metrics (ServerArgs default False), which is also the only thing that mounts /metrics on the server port. Label keys are model_name, engine_type, tp_rank, pp_rank, moe_ep_rank (plus dp_rank under data parallelism, priority under priority scheduling, and anything in extra_metric_labels). Bucket ladders are overridable per histogram via get_histogram_conf_from_env. exponential_buckets(start=0.001, width=1.62, length=30), covering roughly 1 ms to 1191 s. stage names come from the SAME RequestStage enum that names the OTel trace slices, so metrics and traces share one vocabulary - the property no other engine here has.

Notes

SGLang main, python/sglang/srt/observability/metrics_collector.py, read 2026-10-04. NOT ON BY DEFAULT: nothing is exported unless --enable-metrics (ServerArgs default False), which is also the only thing that mounts /metrics on the server port. Label keys are model_name, engine_type, tp_rank, pp_rank, moe_ep_rank (plus dp_rank under data parallelism, priority under priority scheduling, and anything in extra_metric_labels). Bucket ladders are overridable per histogram via get_histogram_conf_from_env. exponential_buckets(start=0.001, width=1.62, length=30), covering roughly 1 ms to 1191 s. stage names come from the SAME RequestStage enum that names the OTel trace slices, so metrics and traces share one vocabulary - the property no other engine here has.

Cross-references