inference/research
verifiedhigh · 0.95updated 2026-10-04

llama.cpp server: any latency metric of any kind on /metrics - VERIFIED ABSENT

llama-cpp · unknown

Fields

id
metric-exposures/w3m-llama-cpp-no-llamacpp-time-to-first-token-seconds
Metric name
llamacpp_time_to_first_token_seconds
Metric type
unknown
Exposure
unknown
Verified negatively
yes
notes
VERIFIED NEGATIVE. llama.cpp server does not emit any latency metric of any kind on /metrics. llamacpp_time_to_first_token_seconds is NOT a metric this engine registers - it is recorded here under the conventional name a reader would go looking for so the absence is queryable rather than implied by a gap in a list. This is the broad negative and it is worth stating in full: the whole metric set is throughput and slot occupancy. There is no TTFT, no TPOT, no ITL, no queue time, no KV-cache utilization, no preemption. Per-request timing DOES exist - the timings object on the chat/completions response carries prompt_ms, prompt_per_second, predicted_ms, predicted_per_token_ms, predicted_per_second and cache_n - but that is response data returned to the client, not an exported series, so it cannot be aggregated by a scraper and TTFT has to be measured client-side. The /slots JSON endpoint (on by default, unlike /metrics) carries a per-slot timings object, which is the nearest thing to a per-request latency surface. What was searched: the counters and gauges tables in to_metrics() were read in full - 10 counters and 5 gauges - and contain no latency name; the README's Available metrics table agrees.. Do not read this record's metric_name as something the engine exports; it is the name that is missing.

Notes

VERIFIED NEGATIVE. llama.cpp server does not emit any latency metric of any kind on /metrics. llamacpp_time_to_first_token_seconds is NOT a metric this engine registers - it is recorded here under the conventional name a reader would go looking for so the absence is queryable rather than implied by a gap in a list. This is the broad negative and it is worth stating in full: the whole metric set is throughput and slot occupancy. There is no TTFT, no TPOT, no ITL, no queue time, no KV-cache utilization, no preemption. Per-request timing DOES exist - the timings object on the chat/completions response carries prompt_ms, prompt_per_second, predicted_ms, predicted_per_token_ms, predicted_per_second and cache_n - but that is response data returned to the client, not an exported series, so it cannot be aggregated by a scraper and TTFT has to be measured client-side. The /slots JSON endpoint (on by default, unlike /metrics) carries a per-slot timings object, which is the nearest thing to a per-request latency surface. What was searched: the counters and gauges tables in to_metrics() were read in full - 10 counters and 5 gauges - and contain no latency name; the README's Available metrics table agrees.. Do not read this record's metric_name as something the engine exports; it is the name that is missing.

Cross-references