llama.cpp busy slots per decode call gauge
llama-cpp · gauge
Fields
id
metric-exposures/w3m-llama-cpp-llamacpp-n-busy-slots-per-decodeMetric name
llamacpp:n_busy_slots_per_decode
Metric type
gauge
Unit
slots
Exposure
prometheus
Endpoint
/metrics (--metrics; router mode requires ?model={model_id})
Enabled by default
no
notes
llama.cpp master, server_task_result_metrics::to_metrics() in tools/server/server-task.cpp, read 2026-10-04. Off by default: '--metrics enable prometheus compatible metrics endpoint (default: disabled)' (env LLAMA_ARG_ENDPOINT_METRICS), and 'This endpoint is only accessible if --metrics is set'. The body is assembled by string concatenation with no Prometheus client library, so there are NO labels at all and no multiprocess mode - every series is a bare name-plus-value line. In router mode /metrics requires ?model={model_id} or returns HTTP 400. n_busy_slots / max(n_decode, 1) - a BATCHING-EFFICIENCY signal, not a latency signal. Worth knowing about precisely because it is easy to misread as one.
Notes
llama.cpp master, server_task_result_metrics::to_metrics() in tools/server/server-task.cpp, read 2026-10-04. Off by default: '--metrics enable prometheus compatible metrics endpoint (default: disabled)' (env LLAMA_ARG_ENDPOINT_METRICS), and 'This endpoint is only accessible if --metrics is set'. The body is assembled by string concatenation with no Prometheus client library, so there are NO labels at all and no multiprocess mode - every series is a bare name-plus-value line. In router mode /metrics requires ?model={model_id} or returns HTTP 400. n_busy_slots / max(n_decode, 1) - a BATCHING-EFFICIENCY signal, not a latency signal. Worth knowing about precisely because it is easy to misread as one.