Metric exposures (316)
One record per (engine, metric): how each engine exposes a metric and how to turn it on.
Dynamo frontend embedding end-to-end latency histogram
nvidia-dynamo · histogramdynamo_frontend_embedding_latency_secondsprometheussecondsmetric-exposures/w5e-dynamo-dynamo-frontend-embedding-latency-seconds
verifiedhigh · 0.92
Dynamo frontend in-flight request gauge
nvidia-dynamo · gaugedynamo_frontend_inflight_requestsprometheusrequestsmetric-exposures/w5e-dynamo-dynamo-frontend-inflight-requests
verifiedhigh · 0.92
Dynamo frontend inter-token-latency histogram
nvidia-dynamo · histogramdynamo_frontend_inter_token_latency_secondsprometheussecondsphase-scopedmetric-exposures/w5e-dynamo-dynamo-frontend-inter-token-latency-seconds
verifiedhigh · 0.92
Dynamo frontend queued-request gauge
nvidia-dynamo · gaugedynamo_frontend_queued_requestsprometheusrequestsmetric-exposures/w5e-dynamo-dynamo-frontend-queued-requests
verifiedhigh · 0.92
Dynamo frontend request duration histogram
nvidia-dynamo · histogramdynamo_frontend_request_duration_secondsprometheussecondsmetric-exposures/w5e-dynamo-dynamo-frontend-request-duration-seconds
verifiedhigh · 0.92
Dynamo frontend terminal failure counter
nvidia-dynamo · counterdynamo_frontend_failures_totalprometheusrequestsmetric-exposures/w5e-dynamo-dynamo-frontend-failures-total
verifiedhigh · 0.92
Dynamo frontend time-to-first-token histogram
nvidia-dynamo · histogramdynamo_frontend_time_to_first_token_secondsprometheussecondsphase-scopedmetric-exposures/w5e-dynamo-dynamo-frontend-time-to-first-token-seconds
verifiedhigh · 0.92
Dynamo router total per-request routing overhead histogram
nvidia-dynamo · histogramdynamo_router_overhead_total_msprometheusmillisecondsphase-scopedmetric-exposures/w5e-dynamo-dynamo-router-overhead-total-ms
verifiedhigh · 0.93
Envoy AI Gateway MCP request duration histogram
envoy-ai-gateway · histogrammcp.request.durationprometheusmetric-exposures/w5e-envoy-ai-gateway-mcp-request-duration
verifiedhigh · 0.90
Envoy AI Gateway server request duration histogram (OTel gen_ai semconv)
envoy-ai-gateway · histogramgen_ai.server.request.durationprometheussecondsmetric-exposures/w5e-envoy-ai-gateway-gen-ai-server-request-duration
verifiedhigh · 0.90
Envoy AI Gateway time-per-output-token histogram (OTel gen_ai semconv)
envoy-ai-gateway · histogramgen_ai.server.time_per_output_tokenprometheussecondsphase-scopedmetric-exposures/w5e-envoy-ai-gateway-gen-ai-server-time-per-output-token
verifiedhigh · 0.90
Envoy AI Gateway time-to-first-token histogram (OTel gen_ai semconv)
envoy-ai-gateway · histogramgen_ai.server.time_to_first_tokenprometheussecondsphase-scopedmetric-exposures/w5e-envoy-ai-gateway-gen-ai-server-time-to-first-token
verifiedhigh · 0.90
Envoy AI Gateway token usage histogram (OTel gen_ai semconv)
envoy-ai-gateway · histogramgen_ai.client.token.usageprometheustokensmetric-exposures/w5e-envoy-ai-gateway-gen-ai-client-token-usage
verifiedhigh · 0.90
llama.cpp average generation throughput gauge
llama-cpp · gaugellamacpp:predicted_tokens_secondsprometheustokens/sopt-inmetric-exposures/w3m-llama-cpp-llamacpp-predicted-tokens-seconds
verifiedhigh · 0.94
llama.cpp average prompt throughput gauge
llama-cpp · gaugellamacpp:prompt_tokens_secondsprometheustokens/sopt-inmetric-exposures/w3m-llama-cpp-llamacpp-prompt-tokens-seconds
verifiedhigh · 0.94
llama.cpp busy slots per decode call gauge
llama-cpp · gaugellamacpp:n_busy_slots_per_decodeprometheusslotsopt-inmetric-exposures/w3m-llama-cpp-llamacpp-n-busy-slots-per-decode
verifiedhigh · 0.94
llama.cpp cached prompt-token counter
llama-cpp · counterllamacpp:prompt_tokens_cached_totalprometheustokensopt-inmetric-exposures/w3m-llama-cpp-llamacpp-prompt-tokens-cached-total
verifiedhigh · 0.94
llama.cpp cumulative generated-token counter
llama-cpp · counterllamacpp:tokens_predicted_totalprometheustokensopt-inmetric-exposures/w3m-llama-cpp-llamacpp-tokens-predicted-total
verifiedhigh · 0.94
llama.cpp cumulative generation seconds
llama-cpp · counterllamacpp:tokens_predicted_seconds_totalprometheussecondsopt-inmetric-exposures/w3m-llama-cpp-llamacpp-tokens-predicted-seconds-total
verifiedhigh · 0.94
llama.cpp cumulative prompt-processing seconds
llama-cpp · counterllamacpp:prompt_seconds_totalprometheussecondsopt-inmetric-exposures/w3m-llama-cpp-llamacpp-prompt-seconds-total
verifiedhigh · 0.94
llama.cpp cumulative prompt-token counter
llama-cpp · counterllamacpp:prompt_tokens_totalprometheustokensopt-inmetric-exposures/w3m-llama-cpp-llamacpp-prompt-tokens-total
verifiedhigh · 0.94
llama.cpp deferred-task gauge
llama-cpp · gaugellamacpp:requests_deferredprometheusrequestsopt-inmetric-exposures/w3m-llama-cpp-llamacpp-requests-deferred
verifiedhigh · 0.94
llama.cpp largest observed sequence length
llama-cpp · counterllamacpp:n_tokens_maxprometheustokensopt-inmetric-exposures/w3m-llama-cpp-llamacpp-n-tokens-max
verifiedhigh · 0.94
llama.cpp llama_decode() call counter
llama-cpp · counterllamacpp:n_decode_totalprometheuscallsopt-inmetric-exposures/w3m-llama-cpp-llamacpp-n-decode-total
verifiedhigh · 0.94
llama.cpp per-position accepted-token counter
llama-cpp · counterllamacpp:spec_decode_num_accepted_tokens_per_pos_totalprometheustokensopt-inmetric-exposures/w3m-llama-cpp-llamacpp-spec-decode-num-accepted-tokens-per-pos-t
verifiedhigh · 0.94
llama.cpp per-position accepted-token counter
llama-cpp · counterllamacpp:spec_decode_num_accepted_tokens_per_pos_totalprometheustokensopt-inmetric-exposures/w3m-llama-cpp-llamacpp-spec-decode-num-accepted-tokens-per-pos-total
verifiedhigh · 0.94
llama.cpp processing-request gauge
llama-cpp · gaugellamacpp:requests_processingprometheusrequestsopt-inmetric-exposures/w3m-llama-cpp-llamacpp-requests-processing
verifiedhigh · 0.94
llama.cpp server: a KV-cache utilization metric - VERIFIED ABSENT
llama-cpp · unknownllamacpp_kv_cache_usage_percunknownmetric-exposures/w3m-llama-cpp-no-llamacpp-kv-cache-usage-perc
verifiedhigh · 0.94
llama.cpp server: any latency metric of any kind on /metrics - VERIFIED ABSENT
llama-cpp · unknownllamacpp_time_to_first_token_secondsunknownmetric-exposures/w3m-llama-cpp-no-llamacpp-time-to-first-token-seconds
verifiedhigh · 0.95
llama.cpp spec accepted-token counter
llama-cpp · counterllamacpp:spec_decode_num_accepted_tokens_totalprometheustokensopt-inmetric-exposures/w3m-llama-cpp-llamacpp-spec-decode-num-accepted-tokens-total
verifiedhigh · 0.94
llama.cpp spec draft-token counter
llama-cpp · counterllamacpp:spec_decode_num_draft_tokens_totalprometheustokensopt-inmetric-exposures/w3m-llama-cpp-llamacpp-spec-decode-num-draft-tokens-total
verifiedhigh · 0.94
llama.cpp spec verification-step counter
llama-cpp · counterllamacpp:spec_decode_num_drafts_totalprometheusstepsopt-inmetric-exposures/w3m-llama-cpp-llamacpp-spec-decode-num-drafts-total
verifiedhigh · 0.94
LMDeploy API-routed-request gauge
niche-lmdeploy-turbomind · gaugelmdeploy:num_api_requests_routedprometheusrequestsmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-num-api-requests-routed
verifiedhigh · 0.94
LMDeploy API-waiting-request gauge
niche-lmdeploy-turbomind · gaugelmdeploy:num_api_requests_waitingprometheusrequestsmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-num-api-requests-waiting
verifiedhigh · 0.94
LMDeploy backend-config info metric
niche-lmdeploy-turbomind · unknownlmdeploy:backend_configprometheus1metric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-backend-config
verifiedhigh · 0.94
LMDeploy cumulative cached-token counter
niche-lmdeploy-turbomind · counterlmdeploy:cached_tokens_totalprometheustokensmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-cached-tokens-total
verifiedhigh · 0.94
LMDeploy cumulative generation-token counter
niche-lmdeploy-turbomind · counterlmdeploy:generation_tokens_totalprometheustokensmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-generation-tokens-total
verifiedhigh · 0.94
LMDeploy cumulative prompt-token counter
niche-lmdeploy-turbomind · counterlmdeploy:prompt_tokens_totalprometheustokensmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-prompt-tokens-total
verifiedhigh · 0.94
LMDeploy end-to-end request latency histogram
niche-lmdeploy-turbomind · histogramlmdeploy:e2e_request_latency_secondsprometheussecondsphase-scopedmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-e2e-request-latency-second
verifiedhigh · 0.94
LMDeploy end-to-end request latency histogram
niche-lmdeploy-turbomind · histogramlmdeploy:e2e_request_latency_secondsprometheussecondsphase-scopedmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-e2e-request-latency-seconds
verifiedhigh · 0.94
LMDeploy failed-request gauge
niche-lmdeploy-turbomind · gaugelmdeploy:num_requests_failedprometheusrequestsmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-num-requests-failed
verifiedhigh · 0.94
LMDeploy GPU KV-cache usage gauge (1.0 = 100%)
niche-lmdeploy-turbomind · gaugelmdeploy:gpu_cache_usage_percprometheus1metric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-gpu-cache-usage-perc
verifiedhigh · 0.94
LMDeploy inter-token latency histogram
niche-lmdeploy-turbomind · histogramlmdeploy:iter_token_latencyprometheussecondsphase-scopedmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-iter-token-latency
verifiedhigh · 0.94
LMDeploy per-request cache-hit-ratio histogram
niche-lmdeploy-turbomind · histogramlmdeploy:request_cache_hit_ratioprometheus1metric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-request-cache-hit-ratio
verifiedhigh · 0.94
LMDeploy per-request cached-token histogram
niche-lmdeploy-turbomind · histogramlmdeploy:request_cached_tokensprometheustokensmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-request-cached-tokens
verifiedhigh · 0.94
LMDeploy per-request generation-token histogram
niche-lmdeploy-turbomind · histogramlmdeploy:request_generation_tokensprometheustokensmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-request-generation-tokens
verifiedhigh · 0.94
LMDeploy per-request prompt-token histogram
niche-lmdeploy-turbomind · histogramlmdeploy:request_prompt_tokensprometheustokensmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-request-prompt-tokens
verifiedhigh · 0.94
LMDeploy prefix-cache hit-rate gauge
niche-lmdeploy-turbomind · gaugelmdeploy:prefix_cache_hit_rateprometheus1metric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-prefix-cache-hit-rate
verifiedhigh · 0.94
LMDeploy queue-time histogram
niche-lmdeploy-turbomind · histogramlmdeploy:request_queue_time_secondsprometheussecondsphase-scopedmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-request-queue-time-seconds
verifiedhigh · 0.94
LMDeploy request decode time histogram
niche-lmdeploy-turbomind · histogramlmdeploy:request_decode_time_secondsprometheussecondsphase-scopedmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-request-decode-time-second
verifiedhigh · 0.94
LMDeploy request decode time histogram
niche-lmdeploy-turbomind · histogramlmdeploy:request_decode_time_secondsprometheussecondsphase-scopedmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-request-decode-time-seconds
verifiedhigh · 0.94
LMDeploy request inference time histogram
niche-lmdeploy-turbomind · histogramlmdeploy:request_inference_time_secondsprometheussecondsphase-scopedmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-request-inference-time-sec
verifiedhigh · 0.94
LMDeploy request inference time histogram
niche-lmdeploy-turbomind · histogramlmdeploy:request_inference_time_secondsprometheussecondsphase-scopedmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-request-inference-time-seconds
verifiedhigh · 0.94
LMDeploy request prefill time histogram
niche-lmdeploy-turbomind · histogramlmdeploy:request_prefill_time_secondsprometheussecondsphase-scopedmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-request-prefill-time-secon
verifiedhigh · 0.94
LMDeploy request prefill time histogram
niche-lmdeploy-turbomind · histogramlmdeploy:request_prefill_time_secondsprometheussecondsphase-scopedmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-request-prefill-time-seconds
verifiedhigh · 0.94
LMDeploy request-success counter by finish reason
niche-lmdeploy-turbomind · counterlmdeploy:request_success_totalprometheusrequestsmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-request-success-total
verifiedhigh · 0.94
LMDeploy running-request gauge
niche-lmdeploy-turbomind · gaugelmdeploy:num_requests_runningprometheusrequestsmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-num-requests-running
verifiedhigh · 0.94
LMDeploy spec accepted-token counter
niche-lmdeploy-turbomind · counterlmdeploy:spec_decode_num_accepted_tokens_totalprometheustokensmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-spec-decode-num-accepted-tokens-total
verifiedhigh · 0.94
LMDeploy spec draft counter
niche-lmdeploy-turbomind · counterlmdeploy:spec_decode_num_drafts_totalprometheusdraftsmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-spec-decode-num-drafts-tot
verifiedhigh · 0.94
LMDeploy spec draft counter
niche-lmdeploy-turbomind · counterlmdeploy:spec_decode_num_drafts_totalprometheusdraftsmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-spec-decode-num-drafts-total
verifiedhigh · 0.94
LMDeploy spec draft-token counter
niche-lmdeploy-turbomind · counterlmdeploy:spec_decode_num_draft_tokens_totalprometheustokensmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-spec-decode-num-draft-toke
verifiedhigh · 0.94
LMDeploy spec draft-token counter
niche-lmdeploy-turbomind · counterlmdeploy:spec_decode_num_draft_tokens_totalprometheustokensmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-spec-decode-num-draft-tokens-total
verifiedhigh · 0.94
LMDeploy spec mean accept-length gauge
niche-lmdeploy-turbomind · gaugelmdeploy:spec_decode_mean_accept_lengthprometheustokensmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-spec-decode-mean-accept-le
verifiedhigh · 0.94
LMDeploy spec mean accept-length gauge
niche-lmdeploy-turbomind · gaugelmdeploy:spec_decode_mean_accept_lengthprometheustokensmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-spec-decode-mean-accept-length
verifiedhigh · 0.94
LMDeploy spec mean accept-rate gauge
niche-lmdeploy-turbomind · gaugelmdeploy:spec_decode_mean_accept_rateprometheus1metric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-spec-decode-mean-accept-ra
verifiedhigh · 0.94
LMDeploy spec mean accept-rate gauge
niche-lmdeploy-turbomind · gaugelmdeploy:spec_decode_mean_accept_rateprometheus1metric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-spec-decode-mean-accept-rate
verifiedhigh · 0.94
LMDeploy spec per-position accept-rate gauge
niche-lmdeploy-turbomind · gaugelmdeploy:spec_decode_per_position_accept_rateprometheus1metric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-spec-decode-per-position-a
verifiedhigh · 0.94
LMDeploy spec per-position accept-rate gauge
niche-lmdeploy-turbomind · gaugelmdeploy:spec_decode_per_position_accept_rateprometheus1metric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-spec-decode-per-position-accept-rate
verifiedhigh · 0.94
LMDeploy spec per-position accepted-token counter
niche-lmdeploy-turbomind · counterlmdeploy:spec_decode_num_accepted_tokens_per_pos_totalprometheustokensmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-spec-decode-num-accepted-t
verifiedhigh · 0.94
LMDeploy spec per-position accepted-token counter
niche-lmdeploy-turbomind · counterlmdeploy:spec_decode_num_accepted_tokens_per_pos_totalprometheustokensmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-spec-decode-num-accepted-tokens-per-pos-total
verifiedhigh · 0.94
LMDeploy succeeded-request gauge
niche-lmdeploy-turbomind · gaugelmdeploy:num_requests_succeededprometheusrequestsmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-num-requests-succeeded
verifiedhigh · 0.94
LMDeploy time-per-output-token (TPOT) histogram
niche-lmdeploy-turbomind · histogramlmdeploy:time_per_output_token_secondsprometheussecondsphase-scopedmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-time-per-output-token-seco
verifiedhigh · 0.94
LMDeploy time-per-output-token (TPOT) histogram
niche-lmdeploy-turbomind · histogramlmdeploy:time_per_output_token_secondsprometheussecondsphase-scopedmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-time-per-output-token-seconds
verifiedhigh · 0.94
LMDeploy time-to-first-token histogram
niche-lmdeploy-turbomind · histogramlmdeploy:time_to_first_token_secondsprometheussecondsphase-scopedmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-time-to-first-token-second
verifiedhigh · 0.94
LMDeploy time-to-first-token histogram
niche-lmdeploy-turbomind · histogramlmdeploy:time_to_first_token_secondsprometheussecondsphase-scopedmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-time-to-first-token-seconds
verifiedhigh · 0.94
LMDeploy tokens-per-engine-step histogram
niche-lmdeploy-turbomind · histogramlmdeploy:iteration_tokens_totalprometheustokensmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-iteration-tokens-total
verifiedhigh · 0.94
LMDeploy waiting-request gauge
niche-lmdeploy-turbomind · gaugelmdeploy:num_requests_waitingprometheusrequestsmetric-exposures/w3m-niche-lmdeploy-turbomind-lmdeploy-num-requests-waiting
verifiedhigh · 0.94
LMDeploy: any preemption metric - VERIFIED ABSENT
niche-lmdeploy-turbomind · unknownlmdeploy_preemptionsunknownmetric-exposures/w3m-niche-lmdeploy-turbomind-no-lmdeploy-preemptions
verifiedhigh · 0.93
Ollama: a KV-cache or occupancy state metric on the server side - VERIFIED ABSENT
ollama · unknownollama_api_ps_kv_stateunknownmetric-exposures/w3m-ollama-no-ollama-api-ps-kv-state
verifiedhigh · 0.92
Ollama: a Prometheus exporter, any metric, or a /metrics endpoint - VERIFIED ABSENT
ollama · unknownollama_prometheus_any_metricunknownmetric-exposures/w3m-ollama-no-ollama-prometheus-any-metric
verifiedhigh · 0.93
SGLang available token slots
sglang · gaugesglang:kv_available_tokensprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-kv-available-tokens
verifiedhigh · 0.94
SGLang block acceptance-length gauge
sglang · gaugesglang:spec_block_accept_lengthprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-spec-block-accept-length
verifiedhigh · 0.94
SGLang cached prompt-token counter
sglang · countersglang:cached_tokens_totalprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-cached-tokens-total
verifiedhigh · 0.94
SGLang context-length gauge
sglang · gaugesglang:context_lenprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-context-len
verifiedhigh · 0.94
SGLang cumulative aborted-request counter
sglang · countersglang:num_aborted_requests_totalprometheusrequestsopt-inmetric-exposures/w3m-sglang-sglang-num-aborted-requests-total
verifiedhigh · 0.94
SGLang cumulative generation-token counter
sglang · countersglang:generation_tokens_totalprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-generation-tokens-total
verifiedhigh · 0.94
SGLang cumulative prompt-token counter
sglang · countersglang:prompt_tokens_totalprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-prompt-tokens-total
verifiedhigh · 0.94
SGLang cumulative request counter
sglang · countersglang:num_requests_totalprometheusrequestsopt-inmetric-exposures/w3m-sglang-sglang-num-requests-total
verifiedhigh · 0.94
SGLang cumulative retracted output-token counter
sglang · countersglang:num_retracted_output_tokens_totalprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-num-retracted-output-tokens-total
verifiedhigh · 0.94
SGLang cumulative retracted prompt-token counter
sglang · countersglang:num_retracted_input_tokens_totalprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-num-retracted-input-tokens-total
verifiedhigh · 0.94
SGLang cumulative retracted-request counter
sglang · countersglang:num_retracted_requests_totalprometheusrequestsopt-inmetric-exposures/w3m-sglang-sglang-num-retracted-requests-total
verifiedhigh · 0.94
SGLang decode host-receive queue gauge
sglang · gaugesglang:num_decode_host_receive_queue_reqsprometheusrequestsopt-inmetric-exposures/w3m-sglang-sglang-num-decode-host-receive-queue-reqs
verifiedhigh · 0.94
SGLang decode prealloc queue gauge
sglang · gaugesglang:num_decode_prealloc_queue_reqsprometheusrequestsopt-inmetric-exposures/w3m-sglang-sglang-num-decode-prealloc-queue-reqs
verifiedhigh · 0.94
SGLang decode transfer queue gauge
sglang · gaugesglang:num_decode_transfer_queue_reqsprometheusrequestsopt-inmetric-exposures/w3m-sglang-sglang-num-decode-transfer-queue-reqs
verifiedhigh · 0.94
SGLang end-to-end request latency histogram
sglang · histogramsglang:e2e_request_latency_secondsprometheussecondsphase-scopedopt-inmetric-exposures/w3m-sglang-sglang-e2e-request-latency-seconds
verifiedhigh · 0.94
SGLang evictable token slots
sglang · gaugesglang:kv_evictable_tokensprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-kv-evictable-tokens
verifiedhigh · 0.94
SGLang forward-pass occupancy gauge
sglang · gaugesglang:fwd_occupancyprometheus1opt-inmetric-exposures/w3m-sglang-sglang-fwd-occupancy
verifiedhigh · 0.94
SGLang full-attention pool token usage
sglang · gaugesglang:full_token_usageprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-full-token-usage
verifiedhigh · 0.94
SGLang generation throughput gauge (token/s)
sglang · gaugesglang:gen_throughputprometheustokens/sopt-inmetric-exposures/w3m-sglang-sglang-gen-throughput
verifiedhigh · 0.94
SGLang grammar-constrained waiting-queue gauge
sglang · gaugesglang:num_grammar_queue_reqsprometheusrequestsopt-inmetric-exposures/w3m-sglang-sglang-num-grammar-queue-reqs
verifiedhigh · 0.94
SGLang inter-token latency histogram
sglang · histogramsglang:inter_token_latency_secondsprometheussecondsphase-scopedopt-inmetric-exposures/w3m-sglang-sglang-inter-token-latency-seconds
verifiedhigh · 0.94
SGLang KV-cache memory usage gauge
sglang · gaugesglang:kv_cache_memory_usage_gbprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-kv-cache-memory-usage-gb
verifiedhigh · 0.94
SGLang KV-pool token usage gauge (max across pools)
sglang · gaugesglang:token_usageprometheus1opt-inmetric-exposures/w3m-sglang-sglang-token-usage
verifiedhigh · 0.94
SGLang KV-transfer allocation histogram
sglang · histogramsglang:kv_transfer_alloc_msprometheusmillisecondsphase-scopedopt-inmetric-exposures/w3m-sglang-sglang-kv-transfer-alloc-ms
verifiedhigh · 0.94
SGLang KV-transfer bootstrap histogram
sglang · histogramsglang:kv_transfer_bootstrap_msprometheusmillisecondsphase-scopedopt-inmetric-exposures/w3m-sglang-sglang-kv-transfer-bootstrap-ms
verifiedhigh · 0.94
SGLang KV-transfer latency histogram
sglang · histogramsglang:kv_transfer_latency_msprometheusmillisecondsphase-scopedopt-inmetric-exposures/w3m-sglang-sglang-kv-transfer-latency-ms
verifiedhigh · 0.94
SGLang KV-transfer speed histogram
sglang · histogramsglang:kv_transfer_speed_gb_sprometheusGB/sphase-scopedopt-inmetric-exposures/w3m-sglang-sglang-kv-transfer-speed-gb-s
verifiedhigh · 0.94
SGLang KV-transfer volume histogram
sglang · histogramsglang:kv_transfer_total_mbprometheusMBopt-inmetric-exposures/w3m-sglang-sglang-kv-transfer-total-mb
verifiedhigh · 0.95
SGLang Mamba/SSM pool usage
sglang · gaugesglang:mamba_usageprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-mamba-usage
verifiedhigh · 0.94
SGLang max-running-requests-under-SLO gauge - DEAD, NO SETTER
sglang · gaugesglang:max_running_requests_under_SLOprometheusrequestsopt-inmetric-exposures/w3m-sglang-sglang-max-running-requests-under-slo
verifiedhigh · 0.94
SGLang new-token ratio gauge
sglang · gaugesglang:new_token_ratioprometheus1opt-inmetric-exposures/w3m-sglang-sglang-new-token-ratio
verifiedhigh · 0.94
SGLang page count gauge
sglang · gaugesglang:num_pagesprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-num-pages
verifiedhigh · 0.94
SGLang page size gauge
sglang · gaugesglang:page_sizeprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-page-size
verifiedhigh · 0.94
SGLang paused-request gauge
sglang · gaugesglang:num_paused_reqsprometheusrequestsopt-inmetric-exposures/w3m-sglang-sglang-num-paused-reqs
verifiedhigh · 0.94
SGLang per-request generation-token histogram
sglang · histogramsglang:generation_tokens_histogramprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-generation-tokens-histogram
verifiedhigh · 0.94
SGLang per-request prompt-token histogram
sglang · histogramsglang:prompt_tokens_histogramprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-prompt-tokens-histogram
verifiedhigh · 0.94
SGLang per-request time-per-output-token (TPOT) histogram
sglang · histogramsglang:request_time_per_output_token_secondsprometheussecondsphase-scopedopt-inmetric-exposures/w3m-sglang-sglang-request-time-per-output-token-seconds
verifiedhigh · 0.94
SGLang per-request uncached prompt-token histogram
sglang · histogramsglang:uncached_prompt_tokens_histogramprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-uncached-prompt-tokens-histogram
verifiedhigh · 0.94
SGLang per-stage request latency histogram (stage label)
sglang · histogramsglang:per_stage_req_latency_secondsprometheussecondsphase-scopedopt-inmetric-exposures/w3m-sglang-sglang-per-stage-req-latency-seconds
verifiedhigh · 0.94
SGLang pool capacity gauge
sglang · gaugesglang:max_total_num_tokensprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-max-total-num-tokens
verifiedhigh · 0.94
SGLang prefill bootstrap queue gauge
sglang · gaugesglang:num_prefill_bootstrap_queue_reqsprometheusrequestsopt-inmetric-exposures/w3m-sglang-sglang-num-prefill-bootstrap-queue-reqs
verifiedhigh · 0.94
SGLang prefill in-flight queue gauge
sglang · gaugesglang:num_prefill_inflight_queue_reqsprometheusrequestsopt-inmetric-exposures/w3m-sglang-sglang-num-prefill-inflight-queue-reqs
verifiedhigh · 0.94
SGLang prefix-cache hit-rate gauge
sglang · gaugesglang:cache_hit_rateprometheus1opt-inmetric-exposures/w3m-sglang-sglang-cache-hit-rate
verifiedhigh · 0.94
SGLang processed-token counter by mode
sglang · countersglang:realtime_tokens_totalprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-realtime-tokens-total
verifiedhigh · 0.94
SGLang queue-time histogram
sglang · histogramsglang:queue_time_secondsprometheussecondsphase-scopedopt-inmetric-exposures/w3m-sglang-sglang-queue-time-seconds
verifiedhigh · 0.94
SGLang retracted-request gauge (preemption, instantaneous)
sglang · gaugesglang:num_retracted_reqsprometheusrequestsopt-inmetric-exposures/w3m-sglang-sglang-num-retracted-reqs
verifiedhigh · 0.94
SGLang running-request gauge
sglang · gaugesglang:num_running_reqsprometheusrequestsopt-inmetric-exposures/w3m-sglang-sglang-num-running-reqs
verifiedhigh · 0.94
SGLang sliding-window-attention pool token usage
sglang · gaugesglang:swa_token_usageprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-swa-token-usage
verifiedhigh · 0.94
SGLang spec-decode acceptance-length gauge
sglang · gaugesglang:spec_accept_lengthprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-spec-accept-length
verifiedhigh · 0.94
SGLang spec-decode acceptance-rate gauge
sglang · gaugesglang:spec_accept_rateprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-spec-accept-rate
verifiedhigh · 0.94
SGLang spec-decode cap-length gauge
sglang · gaugesglang:spec_cap_lengthprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-spec-cap-length
verifiedhigh · 0.94
SGLang spec-decode draft-token gauge
sglang · gaugesglang:spec_num_draft_tokensprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-spec-num-draft-tokens
verifiedhigh · 0.94
SGLang spec-decode step gauge
sglang · gaugesglang:spec_num_stepsprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-spec-num-steps
verifiedhigh · 0.94
SGLang summed decode sequence lengths
sglang · gaugesglang:decode_sum_seq_lensprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-decode-sum-seq-lens
verifiedhigh · 0.94
SGLang SWA pool capacity gauge
sglang · gaugesglang:max_total_num_tokens_swaprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-max-total-num-tokens-swa
verifiedhigh · 0.94
SGLang time-to-first-token histogram (is_streaming split)
sglang · histogramsglang:time_to_first_token_secondsprometheussecondsphase-scopedopt-inmetric-exposures/w3m-sglang-sglang-time-to-first-token-seconds
verifiedhigh · 0.94
SGLang used token count (pool view)
sglang · gaugesglang:kv_used_tokensprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-kv-used-tokens
verifiedhigh · 0.94
SGLang used-token count
sglang · gaugesglang:num_used_tokensprometheustokensopt-inmetric-exposures/w3m-sglang-sglang-num-used-tokens
verifiedhigh · 0.94
SGLang utilization gauge - DOCUMENTED BUT PERMANENTLY ZERO ON MAIN
sglang · gaugesglang:utilizationprometheus1opt-inmetric-exposures/w3m-sglang-sglang-utilization
verifiedhigh · 0.94
SGLang waiting-queue gauge
sglang · gaugesglang:num_queue_reqsprometheusrequestsopt-inmetric-exposures/w3m-sglang-sglang-num-queue-reqs
verifiedhigh · 0.94
SGLang: a KV-cache usage gauge under vLLM's name - VERIFIED ABSENT
sglang · unknownsglang:kv_cache_usage_percunknownmetric-exposures/w3m-sglang-no-sglang-kv-cache-usage-perc
verifiedhigh · 0.93
SGLang: a preemption metric under vLLM's name - VERIFIED ABSENT
sglang · unknownsglang:num_preemptionsunknownmetric-exposures/w3m-sglang-no-sglang-num-preemptions
verifiedhigh · 0.93
SGLang: a request-queue-time histogram under vLLM's name - VERIFIED ABSENT
sglang · unknownsglang:request_queue_time_secondsunknownmetric-exposures/w3m-sglang-no-sglang-request-queue-time-seconds
verifiedhigh · 0.93
TEI failed-request counter
niche-text-embeddings-inference · counterte_request_failureprometheusrequestsmetric-exposures/w5e-tei-te-request-failure
verifiedhigh · 0.90
TEI inference-duration histogram
niche-text-embeddings-inference · histogramte_request_inference_durationprometheussecondsphase-scopedmetric-exposures/w5e-tei-te-request-inference-duration
verifiedhigh · 0.90
TEI input-length histogram
niche-text-embeddings-inference · histogramte_request_input_lengthprometheustokensmetric-exposures/w5e-tei-te-request-input-length
verifiedhigh · 0.90
TEI next-batch-size histogram
niche-text-embeddings-inference · histogramte_batch_next_sizeprometheusrequestsmetric-exposures/w5e-tei-te-batch-next-size
verifiedhigh · 0.90
TEI next-batch-token histogram
niche-text-embeddings-inference · histogramte_batch_next_tokensprometheustokensmetric-exposures/w5e-tei-te-batch-next-tokens
verifiedhigh · 0.90
TEI queue-duration histogram
niche-text-embeddings-inference · histogramte_request_queue_durationprometheussecondsphase-scopedmetric-exposures/w5e-tei-te-request-queue-duration
verifiedhigh · 0.90
TEI request counter
niche-text-embeddings-inference · counterte_request_countprometheusrequestsmetric-exposures/w5e-tei-te-request-count
verifiedhigh · 0.90
TEI request duration histogram
niche-text-embeddings-inference · histogramte_request_durationprometheussecondsmetric-exposures/w5e-tei-te-request-duration
verifiedhigh · 0.90
TEI successful-request counter
niche-text-embeddings-inference · counterte_request_successprometheusrequestsmetric-exposures/w5e-tei-te-request-success
verifiedhigh · 0.90
TEI tokenization-duration histogram
niche-text-embeddings-inference · histogramte_request_tokenization_durationprometheussecondsphase-scopedmetric-exposures/w5e-tei-te-request-tokenization-duration
verifiedhigh · 0.90
TEI: a queue-depth gauge under that conventional name - VERIFIED ABSENT
niche-text-embeddings-inference · unknownte_queue_sizeunknownmetric-exposures/w5e-tei-no-te-queue-size-gauge
verifiedhigh · 0.86
TensorRT-LLM average decoded tokens per iteration
tensorrt-llm · gaugetrtllm_avg_decoded_tokens_per_iterprometheustokensopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-avg-decoded-tokens-per-iter
verifiedhigh · 0.94
TensorRT-LLM cached prompt-token counter
tensorrt-llm · countertrtllm_prompt_cached_tokens_totalprometheustokensopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-prompt-cached-tokens-total
verifiedhigh · 0.94
TensorRT-LLM cached-prompt-tokens-per-request histogram
tensorrt-llm · histogramtrtllm_prompt_cached_tokens_per_requestprometheustokensopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-prompt-cached-tokens-per-request
verifiedhigh · 0.94
TensorRT-LLM completed-request counter
tensorrt-llm · countertrtllm_num_requests_completed_totalprometheusrequestsopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-num-requests-completed-total
verifiedhigh · 0.94
TensorRT-LLM context tokens per iteration histogram
tensorrt-llm · histogramtrtllm_prefill_batch_tokensprometheus1opt-inmetric-exposures/w3m-tensorrt-llm-trtllm-prefill-batch-tokens
verifiedhigh · 0.94
TensorRT-LLM cumulative generation-token counter
tensorrt-llm · countertrtllm_generation_tokens_totalprometheustokensopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-generation-tokens-total
verifiedhigh · 0.94
TensorRT-LLM cumulative prompt-token counter
tensorrt-llm · countertrtllm_prompt_tokens_totalprometheustokensopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-prompt-tokens-total
verifiedhigh · 0.94
TensorRT-LLM end-to-end request latency histogram
tensorrt-llm · histogramtrtllm_e2e_request_latency_secondsprometheussecondsphase-scopedopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-e2e-request-latency-seconds
verifiedhigh · 0.94
TensorRT-LLM GPU memory usage gauge
tensorrt-llm · gaugetrtllm_gpu_memory_usage_bytesprometheusbytesopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-gpu-memory-usage-bytes
verifiedhigh · 0.94
TensorRT-LLM host KV-pool utilization gauge
tensorrt-llm · gaugetrtllm_kv_cache_host_utilizationprometheus1opt-inmetric-exposures/w3m-tensorrt-llm-trtllm-kv-cache-host-utilization
verifiedhigh · 0.94
TensorRT-LLM inflight context-request gauge
tensorrt-llm · gaugetrtllm_num_context_requestsprometheusrequestsopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-num-context-requests
verifiedhigh · 0.94
TensorRT-LLM inflight generation-request gauge
tensorrt-llm · gaugetrtllm_num_generation_requestsprometheusrequestsopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-num-generation-requests
verifiedhigh · 0.94
TensorRT-LLM iteration-latency gauge
tensorrt-llm · gaugetrtllm_iteration_latency_secondsprometheussecondsopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-iteration-latency-seconds
verifiedhigh · 0.94
TensorRT-LLM KV-cache free-blocks gauge
tensorrt-llm · gaugetrtllm_kv_cache_free_blocksprometheusblocksopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-kv-cache-free-blocks
verifiedhigh · 0.94
TensorRT-LLM KV-cache hit-rate gauge
tensorrt-llm · gaugetrtllm_kv_cache_hit_rateprometheus1opt-inmetric-exposures/w3m-tensorrt-llm-trtllm-kv-cache-hit-rate
verifiedhigh · 0.94
TensorRT-LLM KV-cache max-blocks gauge
tensorrt-llm · gaugetrtllm_kv_cache_max_blocksprometheusblocksopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-kv-cache-max-blocks
verifiedhigh · 0.94
TensorRT-LLM KV-cache used-blocks gauge
tensorrt-llm · gaugetrtllm_kv_cache_used_blocksprometheusblocksopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-kv-cache-used-blocks
verifiedhigh · 0.94
TensorRT-LLM KV-cache utilization gauge
tensorrt-llm · gaugetrtllm_kv_cache_utilizationprometheus1opt-inmetric-exposures/w3m-tensorrt-llm-trtllm-kv-cache-utilization
verifiedhigh · 0.94
TensorRT-LLM KV-cache-config info gauge
tensorrt-llm · gaugetrtllm_kv_cache_config_infoprometheus1opt-inmetric-exposures/w3m-tensorrt-llm-trtllm-kv-cache-config-info
verifiedhigh · 0.94
TensorRT-LLM max-active-requests gauge
tensorrt-llm · gaugetrtllm_max_num_active_requestsprometheusrequestsopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-max-num-active-requests
verifiedhigh · 0.94
TensorRT-LLM model-config info gauge
tensorrt-llm · gaugetrtllm_model_config_infoprometheus1opt-inmetric-exposures/w3m-tensorrt-llm-trtllm-model-config-info
verifiedhigh · 0.94
TensorRT-LLM parallel-config info gauge
tensorrt-llm · gaugetrtllm_parallel_config_infoprometheus1opt-inmetric-exposures/w3m-tensorrt-llm-trtllm-parallel-config-info
verifiedhigh · 0.94
TensorRT-LLM paused-request gauge
tensorrt-llm · gaugetrtllm_num_paused_requestsprometheusrequestsopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-num-paused-requests
verifiedhigh · 0.94
TensorRT-LLM per-request generation perplexity histogram
tensorrt-llm · histogramtrtllm_generation_perplexityprometheus1opt-inmetric-exposures/w3m-tensorrt-llm-trtllm-generation-perplexity
verifiedhigh · 0.94
TensorRT-LLM per-request prefill perplexity histogram
tensorrt-llm · histogramtrtllm_prefill_perplexityprometheus1opt-inmetric-exposures/w3m-tensorrt-llm-trtllm-prefill-perplexity
verifiedhigh · 0.94
TensorRT-LLM prefill-batch occupancy gauge
tensorrt-llm · gaugetrtllm_prefill_batch_occupancyprometheus1opt-inmetric-exposures/w3m-tensorrt-llm-trtllm-prefill-batch-occupancy
verifiedhigh · 0.94
TensorRT-LLM queue-time histogram
tensorrt-llm · histogramtrtllm_request_queue_time_secondsprometheussecondsphase-scopedopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-request-queue-time-seconds
verifiedhigh · 0.94
TensorRT-LLM request decode-phase duration histogram
tensorrt-llm · histogramtrtllm_request_decode_time_secondsprometheussecondsphase-scopedopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-request-decode-time-seconds
verifiedhigh · 0.94
TensorRT-LLM request prefill-phase duration histogram
tensorrt-llm · histogramtrtllm_request_prefill_time_secondsprometheussecondsphase-scopedopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-request-prefill-time-seconds
verifiedhigh · 0.94
TensorRT-LLM request-error counter by HTTP status
tensorrt-llm · countertrtllm_request_error_totalprometheusrequestsopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-request-error-total
verifiedhigh · 0.94
TensorRT-LLM request-success counter by finish reason
tensorrt-llm · countertrtllm_request_success_totalprometheusrequestsopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-request-success-total
verifiedhigh · 0.94
TensorRT-LLM running-request gauge
tensorrt-llm · gaugetrtllm_num_requests_runningprometheusrequestsopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-num-requests-running
verifiedhigh · 0.94
TensorRT-LLM scheduled-request gauge
tensorrt-llm · gaugetrtllm_num_scheduled_requestsprometheusrequestsopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-num-scheduled-requests
verifiedhigh · 0.94
TensorRT-LLM speculative-config info gauge
tensorrt-llm · gaugetrtllm_speculative_config_infoprometheus1opt-inmetric-exposures/w3m-tensorrt-llm-trtllm-speculative-config-info
verifiedhigh · 0.94
TensorRT-LLM time-per-output-token (TPOT) histogram
tensorrt-llm · histogramtrtllm_time_per_output_token_secondsprometheussecondsphase-scopedopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-time-per-output-token-seconds
verifiedhigh · 0.94
TensorRT-LLM time-to-first-token histogram
tensorrt-llm · histogramtrtllm_time_to_first_token_secondsprometheussecondsphase-scopedopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-time-to-first-token-seconds
verifiedhigh · 0.94
TensorRT-LLM total context tokens gauge
tensorrt-llm · gaugetrtllm_total_context_tokensprometheustokensopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-total-context-tokens
verifiedhigh · 0.94
TensorRT-LLM total inference duration histogram
tensorrt-llm · histogramtrtllm_request_inference_time_secondsprometheussecondsphase-scopedopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-request-inference-time-seconds
verifiedhigh · 0.94
TensorRT-LLM waiting-request gauge
tensorrt-llm · gaugetrtllm_num_requests_waitingprometheusrequestsopt-inmetric-exposures/w3m-tensorrt-llm-trtllm-num-requests-waiting
verifiedhigh · 0.94
TensorRT-LLM: a waiting-request breakdown by cause - VERIFIED ABSENT
tensorrt-llm · unknowntrtllm_num_requests_waiting_by_reasonunknownmetric-exposures/w3m-tensorrt-llm-no-trtllm-num-requests-waiting-by-reason
verifiedhigh · 0.92
TensorRT-LLM: an inter-token-latency histogram - VERIFIED ABSENT
tensorrt-llm · unknowntrtllm_inter_token_latency_secondsunknownmetric-exposures/w3m-tensorrt-llm-no-trtllm-inter-token-latency-seconds
verifiedhigh · 0.92
TensorRT-LLM: any preemption metric (counter or gauge) - VERIFIED ABSENT
tensorrt-llm · unknowntrtllm_num_preemptionsunknownmetric-exposures/w3m-tensorrt-llm-no-trtllm-num-preemptions
verifiedhigh · 0.93
TGI batch decode-duration histogram (prefill/decode split)
text-generation-inference · histogramtgi_batch_decode_durationprometheussecondsphase-scopedmetric-exposures/w3m-text-generation-inference-tgi-batch-decode-duration
verifiedhigh · 0.94
TGI batch filter-duration histogram (prefill/decode split)
text-generation-inference · histogramtgi_batch_filter_durationprometheussecondsphase-scopedmetric-exposures/w3m-text-generation-inference-tgi-batch-filter-duration
verifiedhigh · 0.94
TGI batch forward-duration histogram (prefill/decode split)
text-generation-inference · histogramtgi_batch_forward_durationprometheussecondsphase-scopedmetric-exposures/w3m-text-generation-inference-tgi-batch-forward-duration
verifiedhigh · 0.94
TGI batch inference-duration histogram (prefill/decode split)
text-generation-inference · histogramtgi_batch_inference_durationprometheussecondsphase-scopedmetric-exposures/w3m-text-generation-inference-tgi-batch-inference-duration
verifiedhigh · 0.94
TGI batch max-total-tokens gauge
text-generation-inference · gaugetgi_batch_max_total_tokensprometheustokensmetric-exposures/w3m-text-generation-inference-tgi-batch-max-total-tokens
verifiedhigh · 0.94
TGI batch total-tokens gauge
text-generation-inference · gaugetgi_batch_total_tokensprometheustokensmetric-exposures/w3m-text-generation-inference-tgi-batch-total-tokens
verifiedhigh · 0.94
TGI batch-concatenation counter by reason
text-generation-inference · countertgi_batch_concatprometheusrequestsmetric-exposures/w3m-text-generation-inference-tgi-batch-concat
verifiedhigh · 0.94
TGI batch-concatenation duration histogram
text-generation-inference · histogramtgi_batch_concat_durationprometheussecondsphase-scopedmetric-exposures/w3m-text-generation-inference-tgi-batch-concat-duration
verifiedhigh · 0.94
TGI cumulative request counter
text-generation-inference · countertgi_request_countprometheusrequestsmetric-exposures/w3m-text-generation-inference-tgi-request-count
verifiedhigh · 0.94
TGI current batch max-tokens gauge
text-generation-inference · gaugetgi_batch_current_max_tokensprometheustokensmetric-exposures/w3m-text-generation-inference-tgi-batch-current-max-tokens
verifiedhigh · 0.94
TGI current batch-size gauge
text-generation-inference · gaugetgi_batch_current_sizeprometheusrequestsmetric-exposures/w3m-text-generation-inference-tgi-batch-current-size
verifiedhigh · 0.94
TGI end-to-end request duration histogram
text-generation-inference · histogramtgi_request_durationprometheussecondsphase-scopedmetric-exposures/w3m-text-generation-inference-tgi-request-duration
verifiedhigh · 0.94
TGI failed-inference counter by method
text-generation-inference · countertgi_batch_inference_failureprometheusrequestsmetric-exposures/w3m-text-generation-inference-tgi-batch-inference-failure
verifiedhigh · 0.94
TGI failed-request counter by error class
text-generation-inference · countertgi_request_failureprometheusrequestsmetric-exposures/w3m-text-generation-inference-tgi-request-failure
verifiedhigh · 0.94
TGI inference-call counter by method
text-generation-inference · countertgi_batch_inference_countprometheusrequestsmetric-exposures/w3m-text-generation-inference-tgi-batch-inference-count
verifiedhigh · 0.94
TGI next batch-size histogram
text-generation-inference · histogramtgi_batch_next_sizeprometheusrequestsmetric-exposures/w3m-text-generation-inference-tgi-batch-next-size
verifiedhigh · 0.94
TGI per-request generated-token histogram
text-generation-inference · histogramtgi_request_generated_tokensprometheustokensmetric-exposures/w3m-text-generation-inference-tgi-request-generated-tokens
verifiedhigh · 0.94
TGI per-request inference-duration histogram
text-generation-inference · histogramtgi_request_inference_durationprometheussecondsphase-scopedmetric-exposures/w3m-text-generation-inference-tgi-request-inference-duration
verifiedhigh · 0.94
TGI per-request input-length histogram
text-generation-inference · histogramtgi_request_input_lengthprometheustokensmetric-exposures/w3m-text-generation-inference-tgi-request-input-length
verifiedhigh · 0.94
TGI per-request max-new-tokens histogram
text-generation-inference · histogramtgi_request_max_new_tokensprometheustokensmetric-exposures/w3m-text-generation-inference-tgi-request-max-new-tokens
verifiedhigh · 0.94
TGI per-request mean-time-per-token histogram
text-generation-inference · histogramtgi_request_mean_time_per_token_durationprometheussecondsphase-scopedmetric-exposures/w3m-text-generation-inference-tgi-request-mean-time-per-token-du
verifiedhigh · 0.94
TGI per-request mean-time-per-token histogram
text-generation-inference · histogramtgi_request_mean_time_per_token_durationprometheussecondsphase-scopedmetric-exposures/w3m-text-generation-inference-tgi-request-mean-time-per-token-duration
verifiedhigh · 0.94
TGI per-request queue-duration histogram
text-generation-inference · histogramtgi_request_queue_durationprometheussecondsphase-scopedmetric-exposures/w3m-text-generation-inference-tgi-request-queue-duration
verifiedhigh · 0.94
TGI per-request speculated-token histogram
text-generation-inference · histogramtgi_request_skipped_tokensprometheustokensmetric-exposures/w3m-text-generation-inference-tgi-request-skipped-tokens
verifiedhigh · 0.94
TGI per-request validation-duration histogram
text-generation-inference · histogramtgi_request_validation_durationprometheussecondsphase-scopedmetric-exposures/w3m-text-generation-inference-tgi-request-validation-duration
verifiedhigh · 0.94
TGI queue-size gauge
text-generation-inference · gaugetgi_queue_sizeprometheusrequestsmetric-exposures/w3m-text-generation-inference-tgi-queue-size
verifiedhigh · 0.94
TGI successful-inference counter by method
text-generation-inference · countertgi_batch_inference_successprometheusrequestsmetric-exposures/w3m-text-generation-inference-tgi-batch-inference-success
verifiedhigh · 0.94
TGI successful-request counter
text-generation-inference · countertgi_request_successprometheusrequestsmetric-exposures/w3m-text-generation-inference-tgi-request-success
verifiedhigh · 0.94
TGI: a preemption or eviction counter - VERIFIED ABSENT
text-generation-inference · unknowntgi_preemption_or_eviction_counterunknownmetric-exposures/w3m-text-generation-inference-no-tgi-preemption-or-eviction-coun
verifiedhigh · 0.92
TGI: a preemption or eviction counter - VERIFIED ABSENT
text-generation-inference · unknowntgi_preemption_or_eviction_counterunknownmetric-exposures/w3m-text-generation-inference-no-tgi-preemption-or-eviction-counter
verifiedhigh · 0.92
TGI: a time-to-first-token metric of any kind - VERIFIED ABSENT
text-generation-inference · unknowntgi_time_to_first_tokenunknownmetric-exposures/w3m-text-generation-inference-no-tgi-time-to-first-token
verifiedhigh · 0.93
TGI: any KV-cache occupancy metric - VERIFIED ABSENT
text-generation-inference · unknowntgi_kv_cache_usageunknownmetric-exposures/w3m-text-generation-inference-no-tgi-kv-cache-usage
verifiedhigh · 0.93
Triton batch-execution counter
triton-inference-server · counternv_inference_exec_countprometheusexecutionsmetric-exposures/w3m-triton-inference-server-nv-inference-exec-count
verifiedhigh · 0.94
Triton CPU memory used gauge
triton-inference-server · gaugenv_cpu_memory_used_bytesprometheusbytesmetric-exposures/w3m-triton-inference-server-nv-cpu-memory-used-bytes
verifiedhigh · 0.94
Triton cumulative compute-infer duration
triton-inference-server · counternv_inference_compute_infer_duration_usprometheusmicrosecondsmetric-exposures/w3m-triton-inference-server-nv-inference-compute-infer-duration
verifiedhigh · 0.94
Triton cumulative compute-infer duration
triton-inference-server · counternv_inference_compute_infer_duration_usprometheusmicrosecondsmetric-exposures/w3m-triton-inference-server-nv-inference-compute-infer-duration-us
verifiedhigh · 0.94
Triton cumulative compute-input duration
triton-inference-server · counternv_inference_compute_input_duration_usprometheusmicrosecondsmetric-exposures/w3m-triton-inference-server-nv-inference-compute-input-duration
verifiedhigh · 0.94
Triton cumulative compute-input duration
triton-inference-server · counternv_inference_compute_input_duration_usprometheusmicrosecondsmetric-exposures/w3m-triton-inference-server-nv-inference-compute-input-duration-us
verifiedhigh · 0.94
Triton cumulative compute-output duration
triton-inference-server · counternv_inference_compute_output_duration_usprometheusmicrosecondsmetric-exposures/w3m-triton-inference-server-nv-inference-compute-output-duration
verifiedhigh · 0.94
Triton cumulative compute-output duration
triton-inference-server · counternv_inference_compute_output_duration_usprometheusmicrosecondsmetric-exposures/w3m-triton-inference-server-nv-inference-compute-output-duration-us
verifiedhigh · 0.94
Triton cumulative e2e request duration
triton-inference-server · counternv_inference_request_duration_usprometheusmicrosecondsmetric-exposures/w3m-triton-inference-server-nv-inference-request-duration-us
verifiedhigh · 0.94
Triton cumulative queue duration
triton-inference-server · counternv_inference_queue_duration_usprometheusmicrosecondsmetric-exposures/w3m-triton-inference-server-nv-inference-queue-duration-us
verifiedhigh · 0.94
Triton failed-request counter by reason
triton-inference-server · counternv_inference_request_failureprometheusrequestsmetric-exposures/w3m-triton-inference-server-nv-inference-request-failure
verifiedhigh · 0.94
Triton GPU memory total gauge
triton-inference-server · gaugenv_gpu_memory_total_bytesprometheusbytesmetric-exposures/w3m-triton-inference-server-nv-gpu-memory-total-bytes
verifiedhigh · 0.94
Triton GPU memory used gauge
triton-inference-server · gaugenv_gpu_memory_used_bytesprometheusbytesmetric-exposures/w3m-triton-inference-server-nv-gpu-memory-used-bytes
verifiedhigh · 0.94
Triton GPU power gauge
triton-inference-server · gaugenv_gpu_power_usageprometheuswattsmetric-exposures/w3m-triton-inference-server-nv-gpu-power-usage
verifiedhigh · 0.94
Triton GPU utilization gauge
triton-inference-server · gaugenv_gpu_utilizationprometheus1metric-exposures/w3m-triton-inference-server-nv-gpu-utilization
verifiedhigh · 0.94
Triton inference counter
triton-inference-server · counternv_inference_countprometheusinferencesmetric-exposures/w3m-triton-inference-server-nv-inference-count
verifiedhigh · 0.94
Triton Inference Server: a KV-cache occupancy metric - VERIFIED ABSENT
triton-inference-server · unknownnv_inference_kv_cache_usageunknownmetric-exposures/w3m-triton-inference-server-no-nv-inference-kv-cache-usage
verifiedhigh · 0.93
Triton Inference Server: a preemption metric - VERIFIED ABSENT
triton-inference-server · unknownnv_inference_preemptionsunknownmetric-exposures/w3m-triton-inference-server-no-nv-inference-preemptions
verifiedhigh · 0.90
Triton Inference Server: a time-per-output-token / inter-token latency metric - VERIFIED ABSENT
triton-inference-server · unknownnv_inference_time_per_output_tokenunknownmetric-exposures/w3m-triton-inference-server-no-nv-inference-time-per-output-toke
verifiedhigh · 0.93
Triton Inference Server: a time-per-output-token / inter-token latency metric - VERIFIED ABSENT
triton-inference-server · unknownnv_inference_time_per_output_tokenunknownmetric-exposures/w3m-triton-inference-server-no-nv-inference-time-per-output-token
verifiedhigh · 0.93
Triton Inference Server: a time-to-first-token metric - VERIFIED ABSENT
triton-inference-server · unknownnv_inference_time_to_first_tokenunknownmetric-exposures/w3m-triton-inference-server-no-nv-inference-time-to-first-token
verifiedhigh · 0.93
Triton Inference Server: a token-throughput metric - VERIFIED ABSENT
triton-inference-server · unknownnv_inference_token_throughputunknownmetric-exposures/w3m-triton-inference-server-no-nv-inference-token-throughput
verifiedhigh · 0.93
Triton pending-request gauge
triton-inference-server · gaugenv_inference_pending_request_countprometheusrequestsmetric-exposures/w3m-triton-inference-server-nv-inference-pending-request-count
verifiedhigh · 0.94
Triton queue-duration summary
triton-inference-server · summarynv_inference_queue_summary_usprometheusmicrosecondsopt-inmetric-exposures/w3m-triton-inference-server-nv-inference-queue-summary-us
verifiedhigh · 0.94
Triton request-duration summary
triton-inference-server · summarynv_inference_request_summary_usprometheusmicrosecondsopt-inmetric-exposures/w3m-triton-inference-server-nv-inference-request-summary-us
verifiedhigh · 0.94
Triton successful-request counter
triton-inference-server · counternv_inference_request_successprometheusrequestsmetric-exposures/w3m-triton-inference-server-nv-inference-request-success
verifiedhigh · 0.94
Triton time-to-first-RESPONSE histogram
triton-inference-server · histogramnv_inference_first_response_histogram_msprometheusmillisecondsopt-inmetric-exposures/w3m-triton-inference-server-nv-inference-first-response-histogra
verifiedhigh · 0.94
Triton time-to-first-RESPONSE histogram
triton-inference-server · histogramnv_inference_first_response_histogram_msprometheusmillisecondsopt-inmetric-exposures/w3m-triton-inference-server-nv-inference-first-response-histogram-ms
verifiedhigh · 0.94
Triton+TRT-LLM TRT-LLM disaggregated-serving gauge, label-switched
triton-inference-server · gaugenv_trt_llm_disaggregated_serving_metricsprometheus1metric-exposures/w3m-triton-inference-server-nv-trt-llm-disaggregated-serving-met
verifiedhigh · 0.94
Triton+TRT-LLM TRT-LLM disaggregated-serving gauge, label-switched
triton-inference-server · gaugenv_trt_llm_disaggregated_serving_metricsprometheus1metric-exposures/w3m-triton-inference-server-nv-trt-llm-disaggregated-serving-metrics
verifiedhigh · 0.94
Triton+TRT-LLM TRT-LLM general gauge, label-switched
triton-inference-server · gaugenv_trt_llm_general_metricsprometheus1metric-exposures/w3m-triton-inference-server-nv-trt-llm-general-metrics
verifiedhigh · 0.94
Triton+TRT-LLM TRT-LLM inflight-batcher gauge, label-switched
triton-inference-server · gaugenv_trt_llm_inflight_batcher_metricsprometheus1metric-exposures/w3m-triton-inference-server-nv-trt-llm-inflight-batcher-metrics
verifiedhigh · 0.94
Triton+TRT-LLM TRT-LLM KV-cache block gauge, label-switched
triton-inference-server · gaugenv_trt_llm_kv_cache_block_metricsprometheus1metric-exposures/w3m-triton-inference-server-nv-trt-llm-kv-cache-block-metrics
verifiedhigh · 0.94
Triton+TRT-LLM TRT-LLM request gauge, label-switched
triton-inference-server · gaugenv_trt_llm_request_metricsprometheus1metric-exposures/w3m-triton-inference-server-nv-trt-llm-request-metrics
verifiedhigh · 0.94
Triton+TRT-LLM TRT-LLM runtime-memory gauge, label-switched
triton-inference-server · gaugenv_trt_llm_runtime_memory_metricsprometheus1metric-exposures/w3m-triton-inference-server-nv-trt-llm-runtime-memory-metrics
verifiedhigh · 0.94
vLLM cached prompt-token counter
vllm · countervllm:prompt_tokens_cachedprometheustokensmetric-exposures/w3m-vllm-vllm-prompt-tokens-cached
verifiedhigh · 0.94
vLLM corrupted-request counter
vllm · countervllm:corrupted_requestsprometheusrequestsmetric-exposures/w3m-vllm-vllm-corrupted-requests
verifiedhigh · 0.94
vLLM cumulative generation-token counter (token throughput numerator)
vllm · countervllm:generation_tokensprometheustokensmetric-exposures/w3m-vllm-vllm-generation-tokens
verifiedhigh · 0.94
vLLM cumulative preemption counter
vllm · countervllm:num_preemptionsprometheusrequestsmetric-exposures/w3m-vllm-vllm-num-preemptions
verifiedhigh · 0.94
vLLM cumulative prompt-token counter
vllm · countervllm:prompt_tokensprometheustokensmetric-exposures/w3m-vllm-vllm-prompt-tokens
verifiedhigh · 0.94
vLLM end-to-end request latency histogram
vllm · histogramvllm:e2e_request_latency_secondsprometheussecondsphase-scopedmetric-exposures/w3m-vllm-vllm-e2e-request-latency-seconds
verifiedhigh · 0.94
vLLM engine sleep-state gauge
vllm · gaugevllm:engine_sleep_stateprometheus1metric-exposures/w3m-vllm-vllm-engine-sleep-state
verifiedhigh · 0.94
vLLM external prefix-cache hit counter
vllm · countervllm:external_prefix_cache_hitsprometheusrequestsmetric-exposures/w3m-vllm-vllm-external-prefix-cache-hits
verifiedhigh · 0.94
vLLM external prefix-cache query counter
vllm · countervllm:external_prefix_cache_queriesprometheusrequestsmetric-exposures/w3m-vllm-vllm-external-prefix-cache-queries
verifiedhigh · 0.94
vLLM KV block idle-before-evict histogram
vllm · histogramvllm:kv_block_idle_before_evict_secondsprometheussecondsopt-inmetric-exposures/w3m-vllm-vllm-kv-block-idle-before-evict-seconds
verifiedhigh · 0.94
vLLM KV block lifetime histogram
vllm · histogramvllm:kv_block_lifetime_secondsprometheussecondsopt-inmetric-exposures/w3m-vllm-vllm-kv-block-lifetime-seconds
verifiedhigh · 0.94
vLLM KV block reuse-gap histogram
vllm · histogramvllm:kv_block_reuse_gap_secondsprometheussecondsopt-inmetric-exposures/w3m-vllm-vllm-kv-block-reuse-gap-seconds
verifiedhigh · 0.94
vLLM KV-cache usage gauge (1.0 = 100%)
vllm · gaugevllm:kv_cache_usage_percprometheus1metric-exposures/w3m-vllm-vllm-kv-cache-usage-perc
verifiedhigh · 0.94
vLLM KV-consumer fetch gauge by stage
vllm · gaugevllm:num_requests_kv_fetch_by_stageprometheusrequestsmetric-exposures/w3m-vllm-vllm-num-requests-kv-fetch-by-stage
verifiedhigh · 0.94
vLLM multimodal-cache hit counter
vllm · countervllm:mm_cache_hitsprometheusrequestsmetric-exposures/w3m-vllm-vllm-mm-cache-hits
verifiedhigh · 0.94
vLLM multimodal-cache query counter
vllm · countervllm:mm_cache_queriesprometheusrequestsmetric-exposures/w3m-vllm-vllm-mm-cache-queries
verifiedhigh · 0.94
vLLM per-position accepted-token counter
vllm · countervllm:spec_decode_num_accepted_tokens_per_posprometheustokensmetric-exposures/w3m-vllm-vllm-spec-decode-num-accepted-tokens-per-pos
verifiedhigh · 0.94
vLLM per-request generation-token histogram
vllm · histogramvllm:request_generation_tokensprometheustokensmetric-exposures/w3m-vllm-vllm-request-generation-tokens
verifiedhigh · 0.95
vLLM per-request max-generation-tokens histogram
vllm · histogramvllm:request_max_num_generation_tokensprometheustokensmetric-exposures/w3m-vllm-vllm-request-max-num-generation-tokens
verifiedhigh · 0.95
vLLM per-request preemption-count histogram
vllm · histogramvllm:request_num_preemptionsprometheusrequestsmetric-exposures/w3m-vllm-vllm-request-num-preemptions
verifiedhigh · 0.94
vLLM per-request prefill KV-computed-token histogram
vllm · histogramvllm:request_prefill_kv_computed_tokensprometheustokensmetric-exposures/w3m-vllm-vllm-request-prefill-kv-computed-tokens
verifiedhigh · 0.95
vLLM per-request prompt-token histogram
vllm · histogramvllm:request_prompt_tokensprometheustokensmetric-exposures/w3m-vllm-vllm-request-prompt-tokens
verifiedhigh · 0.95
vLLM per-request time-per-output-token (TPOT) histogram
vllm · histogramvllm:request_time_per_output_token_secondsprometheussecondsphase-scopedmetric-exposures/w3m-vllm-vllm-request-time-per-output-token-seconds
verifiedhigh · 0.94
vLLM per-step inter-token latency histogram
vllm · histogramvllm:inter_token_latency_secondsprometheussecondsphase-scopedmetric-exposures/w3m-vllm-vllm-inter-token-latency-seconds
verifiedhigh · 0.94
vLLM prefill time-to-first-token histogram
vllm · histogramvllm:time_to_first_token_secondsprometheussecondsphase-scopedmetric-exposures/w3m-vllm-vllm-time-to-first-token-seconds
verifiedhigh · 0.94
vLLM prefix-cache hit counter
vllm · countervllm:prefix_cache_hitsprometheusrequestsmetric-exposures/w3m-vllm-vllm-prefix-cache-hits
verifiedhigh · 0.94
vLLM prefix-cache query counter
vllm · countervllm:prefix_cache_queriesprometheusrequestsmetric-exposures/w3m-vllm-vllm-prefix-cache-queries
verifiedhigh · 0.94
vLLM prompt-token counter by source
vllm · countervllm:prompt_tokens_by_sourceprometheustokensmetric-exposures/w3m-vllm-vllm-prompt-tokens-by-source
verifiedhigh · 0.94
vLLM queue-time histogram (time in WAITING phase)
vllm · histogramvllm:request_queue_time_secondsprometheussecondsphase-scopedmetric-exposures/w3m-vllm-vllm-request-queue-time-seconds
verifiedhigh · 0.94
vLLM request decode time histogram
vllm · histogramvllm:request_decode_time_secondsprometheussecondsphase-scopedmetric-exposures/w3m-vllm-vllm-request-decode-time-seconds
verifiedhigh · 0.94
vLLM request inference time histogram
vllm · histogramvllm:request_inference_time_secondsprometheussecondsphase-scopedmetric-exposures/w3m-vllm-vllm-request-inference-time-seconds
verifiedhigh · 0.94
vLLM request prefill (pre-first-token) time histogram
vllm · histogramvllm:request_prefill_time_secondsprometheussecondsphase-scopedmetric-exposures/w3m-vllm-vllm-request-prefill-time-seconds
verifiedhigh · 0.94
vLLM request-completion counter by finish reason
vllm · countervllm:request_successprometheusrequestsmetric-exposures/w3m-vllm-vllm-request-success
verifiedhigh · 0.94
vLLM running-request gauge
vllm · gaugevllm:num_requests_runningprometheusrequestsmetric-exposures/w3m-vllm-vllm-num-requests-running
verifiedhigh · 0.94
vLLM sampling-parameter max_tokens histogram
vllm · histogramvllm:request_params_max_tokensprometheustokensmetric-exposures/w3m-vllm-vllm-request-params-max-tokens
verifiedhigh · 0.95
vLLM sampling-parameter n histogram
vllm · histogramvllm:request_params_nprometheusrequestsmetric-exposures/w3m-vllm-vllm-request-params-n
verifiedhigh · 0.94
vLLM spec-decode accepted-token counter
vllm · countervllm:spec_decode_num_accepted_tokensprometheustokensmetric-exposures/w3m-vllm-vllm-spec-decode-num-accepted-tokens
verifiedhigh · 0.94
vLLM spec-decode draft counter
vllm · countervllm:spec_decode_num_draftsprometheusdraftsmetric-exposures/w3m-vllm-vllm-spec-decode-num-drafts
verifiedhigh · 0.94
vLLM spec-decode draft-token counter
vllm · countervllm:spec_decode_num_draft_tokensprometheustokensmetric-exposures/w3m-vllm-vllm-spec-decode-num-draft-tokens
verifiedhigh · 0.94
vLLM tokens-per-engine-step histogram
vllm · histogramvllm:iteration_tokens_totalprometheustokensmetric-exposures/w3m-vllm-vllm-iteration-tokens-total
verifiedhigh · 0.94
vLLM waiting-request gauge
vllm · gaugevllm:num_requests_waitingprometheusrequestsmetric-exposures/w3m-vllm-vllm-num-requests-waiting
verifiedhigh · 0.94
vLLM waiting-request gauge split by cause
vllm · gaugevllm:num_requests_waiting_by_reasonprometheusrequestsmetric-exposures/w3m-vllm-vllm-num-requests-waiting-by-reason
verifiedhigh · 0.94
Xinference configured concurrency-limit gauge
xinference · gaugexinference:model_request_limitprometheusrequestsopt-inmetric-exposures/w5e-xinference-xinference-model-request-limit
verifiedhigh · 0.90
Xinference failed-request counter
xinference · counterxinference:model_request_errors_totalprometheusrequestsopt-inmetric-exposures/w5e-xinference-xinference-model-request-errors-total
verifiedhigh · 0.90
Xinference generated-token counter
xinference · counterxinference:generate_tokens_totalprometheustokensopt-inmetric-exposures/w5e-xinference-xinference-generate-tokens-total
verifiedhigh · 0.90
Xinference in-flight request gauge
xinference · gaugexinference:model_serve_countprometheusrequestsopt-inmetric-exposures/w5e-xinference-xinference-model-serve-count
verifiedhigh · 0.90
Xinference last model load duration gauge
xinference · gaugexinference:model_last_load_duration_secondsprometheussecondsopt-inmetric-exposures/w5e-xinference-xinference-model-last-load-duration-seconds
verifiedhigh · 0.90
Xinference model request counter
xinference · counterxinference:model_request_totalprometheusrequestsopt-inmetric-exposures/w5e-xinference-xinference-model-request-total
verifiedhigh · 0.90
Xinference model request duration histogram
xinference · histogramxinference:model_request_duration_secondsprometheussecondsopt-inmetric-exposures/w5e-xinference-xinference-model-request-duration-seconds
verifiedhigh · 0.90
Xinference time-to-first-token histogram
xinference · histogramxinference:time_to_first_token_secondsprometheussecondsphase-scopedopt-inmetric-exposures/w5e-xinference-xinference-time-to-first-token-seconds
verifiedhigh · 0.90
Xinference worker CPU utilization
xinference · gaugexinference:worker_cpu_utilizationprometheus1 (ratio)opt-inmetric-exposures/w5e-xinference-xinference-worker-cpu-utilization
verifiedhigh · 0.90
Xinference worker GPU memory used
xinference · gaugexinference:worker_gpu_memory_used_bytesprometheusbytesopt-inmetric-exposures/w5e-xinference-xinference-worker-gpu-memory-used-bytes
verifiedhigh · 0.90
Xinference: an ITL histogram under the vLLM/SGLang name - VERIFIED ABSENT
xinference · unknownxinference:inter_token_latency_secondsunknownmetric-exposures/w5e-xinference-no-xinference-inter-token-latency-seconds
verifiedhigh · 0.88