LMDeploy spec per-position accepted-token counter
niche-lmdeploy-turbomind · counter
Fields
id
metric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-spec-decode-num-accepted-tMetric name
lmdeploy:spec_decode_num_accepted_tokens_per_pos_total
Metric type
counter
Unit
tokens
Exposure
prometheus
Endpoint
:23333/metrics
Labels
model_nameengineposition
Enabled by default
yes
notes
InternLM/lmdeploy main, PrometheusStatLogger in lmdeploy/metrics/loggers.py, read 2026-10-04. On by default and requires NO flag: the endpoint is simply available at the API server port (23333 by default). Label keys are model_name and engine on every metric, where engine carries the dp_rank. Bucket construction reuses vLLM's build_buckets / build_1_2_5_buckets idiom. Unusual: LMDeploy ships the DERIVED acceptance ratios as first-class gauges (mean_accept_rate = accepted/draft_tokens, mean_accept_length = 1 + accepted/drafts) rather than leaving the division to PromQL as vLLM does. The raw counters are present too, so the ratios are auditable.
Notes
InternLM/lmdeploy main, PrometheusStatLogger in lmdeploy/metrics/loggers.py, read 2026-10-04. On by default and requires NO flag: the endpoint is simply available at the API server port (23333 by default). Label keys are model_name and engine on every metric, where engine carries the dp_rank. Bucket construction reuses vLLM's build_buckets / build_1_2_5_buckets idiom. Unusual: LMDeploy ships the DERIVED acceptance ratios as first-class gauges (mean_accept_rate = accepted/draft_tokens, mean_accept_length = 1 + accepted/drafts) rather than leaving the division to PromQL as vLLM does. The raw counters are present too, so the ratios are auditable.