SGLang per-request uncached prompt-token histogram
sglang · histogram
Fields
id
metric-exposures/w3m-sglang-sglang-uncached-prompt-tokens-histogramMetric name
sglang:uncached_prompt_tokens_histogram
Metric type
histogram
Unit
tokens
Exposure
prometheus
Endpoint
:30000/metrics
Histogram buckets
10030050070010001500200030004000500060007000800090001000012500150001750020000225002500027500300003500040000600008000010000020000030000040000060000080000010000001100000
Labels
model_nameengine_typetp_rankpp_rankmoe_ep_rank
Enabled by default
no
notes
SGLang main, python/sglang/srt/observability/metrics_collector.py, read 2026-10-04. NOT ON BY DEFAULT: nothing is exported unless --enable-metrics (ServerArgs default False), which is also the only thing that mounts /metrics on the server port. Label keys are model_name, engine_type, tp_rank, pp_rank, moe_ep_rank (plus dp_rank under data parallelism, priority under priority scheduling, and anything in extra_metric_labels). Bucket ladders are overridable per histogram via get_histogram_conf_from_env. All three per-request token histograms share ONE 35-bound ladder (default_bucket_prompt_tokens, 100 .. 1100000) and are overridable independently via --prompt-tokens-buckets / --generation-tokens-buckets.
Notes
SGLang main, python/sglang/srt/observability/metrics_collector.py, read 2026-10-04. NOT ON BY DEFAULT: nothing is exported unless --enable-metrics (ServerArgs default False), which is also the only thing that mounts /metrics on the server port. Label keys are model_name, engine_type, tp_rank, pp_rank, moe_ep_rank (plus dp_rank under data parallelism, priority under priority scheduling, and anything in extra_metric_labels). Bucket ladders are overridable per histogram via get_histogram_conf_from_env. All three per-request token histograms share ONE 35-bound ladder (default_bucket_prompt_tokens, 100 .. 1100000) and are overridable independently via --prompt-tokens-buckets / --generation-tokens-buckets.