obs-triton-trtllm-backend-metrics

Fields

id

sources/obs-triton-trtllm-backend-metrics

URL

https://github.com/triton-inference-server/tensorrtllm_backend/blob/main/README.md

Publisher

triton-inference-server

Kind

repo

Accessed

2026-10-04

notes

The Triton Metrics section: TRT-LLM's LLM-specific metrics are registered into the Triton endpoint as a small number of label-switched gauges - nv_trt_llm_request_metrics{request_type="waiting|context|scheduled|max|active"}, nv_trt_llm_kv_cache_block_metrics{kv_cache_block_type="fraction|tokens_per|used|free|max"}, nv_trt_llm_general_metrics, nv_trt_llm_disaggregated_serving_metrics. Only available from Triton 23.11+.

Notes

The Triton Metrics section: TRT-LLM's LLM-specific metrics are registered into the Triton endpoint as a small number of label-switched gauges - nv_trt_llm_request_metrics{request_type="waiting|context|scheduled|max|active"}, nv_trt_llm_kv_cache_block_metrics{kv_cache_block_type="fraction|tokens_per|used|free|max"}, nv_trt_llm_general_metrics, nv_trt_llm_disaggregated_serving_metrics. Only available from Triton 23.11+.

Cross-references

This record references no other records.