inference/research
verifiedhigh · 0.95updated 2026-10-04

llama.cpp server README - --metrics default disabled, /metrics behaviour, and the /slots alternative

repo

Fields

id
sources/w3m-llamacpp-metrics-flag-default-off
Publisher
ggml-org
Kind
repo
Accessed
2026-10-04
notes
tools/server/README.md on master, read 2026-10-04. Verbatim option row: '| --metrics | enable prometheus compatible metrics endpoint (default: disabled)<br/>(env: LLAMA_ARG_ENDPOINT_METRICS) |' next to '| --slots, --no-slots | expose slots monitoring endpoint (default: enabled)<br/>(env: LLAMA_ARG_ENDPOINT_SLOTS) |'. Section 'GET /metrics: Prometheus compatible metrics exporter' states 'This endpoint is only accessible if --metrics is set' and 'In router mode the query param ?model={model_id} has to be set. This endpoint will respond with status code 400 model name is missing from the request if not set.' The asymmetry matters for this record set: the slot-level state endpoint is on by default while the Prometheus endpoint is off by default, so a reader arriving at a stock server finds per-slot JSON and no time series. Same README documents the Available metrics table, which omits prompt_tokens_cached_total that the source does emit.

Notes

tools/server/README.md on master, read 2026-10-04. Verbatim option row: '| --metrics | enable prometheus compatible metrics endpoint (default: disabled)<br/>(env: LLAMA_ARG_ENDPOINT_METRICS) |' next to '| --slots, --no-slots | expose slots monitoring endpoint (default: enabled)<br/>(env: LLAMA_ARG_ENDPOINT_SLOTS) |'. Section 'GET /metrics: Prometheus compatible metrics exporter' states 'This endpoint is only accessible if --metrics is set' and 'In router mode the query param ?model={model_id} has to be set. This endpoint will respond with status code 400 model name is missing from the request if not set.' The asymmetry matters for this record set: the slot-level state endpoint is on by default while the Prometheus endpoint is off by default, so a reader arriving at a stock server finds per-slot JSON and no time series. Same README documents the Available metrics table, which omits prompt_tokens_cached_total that the source does emit.

Cross-references

This record references no other records.