TensorRT-LLM KV-cache utilization gauge
Fields
metric-exposures/w3m-tensorrt-llm-trtllm-kv-cache-utilizationNotes
TensorRT-LLM main, MetricsCollector in tensorrt_llm/metrics/collector.py, read 2026-10-04. collector.py line 32 carries the header comment 'Adapted from https://github.com/vllm-project/vllm/blob/v0.10.0rc1/vllm/engine/metrics.py#L30', so this is a vLLM fork with the prefix swapped to trtllm_. NOT ON BY DEFAULT: the collector is only constructed and /prometheus/metrics only mounted when --return_perf-metrics is set (llm_args default False). Endpoint is :8000/prometheus/metrics - NOT /metrics, and the server's own GET /metrics serves a JSON iteration-stats snapshot, so a scraper pointed at /metrics gets JSON rather than exposition text. Labels are model_name and engine_type. usedNumBlocks / maxNumBlocks. The in-repo unit test _test_openai_prometheus.py polls /prometheus/metrics specifically for this series, which is a useful sanity check that the flag was set at all. Note the name is kv_cache_utilization, not vllm:kv_cache_usage_perc - a dashboard shared with vLLM needs both spelling changes.