inference/research
verifiedhigh · 0.86updated 2026-10-05

TEI: a queue-depth gauge under that conventional name - VERIFIED ABSENT

niche-text-embeddings-inference · unknown

Fields

id
metric-exposures/w5e-tei-no-te-queue-size-gauge
Metric name
te_queue_size
Metric type
unknown
Exposure
unknown
Verified negatively
yes
notes
VERIFIED NEGATIVE. TEI does not declare a metric named te_queue_size. The router does have queueing - Infer::try_acquire_permit() is a semaphore acquire that rejects rather than blocks, and the wait is measured as the te_request_queue_duration HISTOGRAM - but there is no instantaneous queue-depth gauge, so an operator can see how long requests waited and not how many are waiting. Recorded under the name a reader would guess so the gap is queryable. What was searched: every counter!/histogram!/gauge! call site in router/src/lib.rs, router/src/http/server.rs and router/src/grpc/server.rs, plus all four Matcher declarations in router/src/prometheus.rs. Do not read this record's metric_name as something the engine exports; it is the name that is missing.

Notes

VERIFIED NEGATIVE. TEI does not declare a metric named te_queue_size. The router does have queueing - Infer::try_acquire_permit() is a semaphore acquire that rejects rather than blocks, and the wait is measured as the te_request_queue_duration HISTOGRAM - but there is no instantaneous queue-depth gauge, so an operator can see how long requests waited and not how many are waiting. Recorded under the name a reader would guess so the gap is queryable. What was searched: every counter!/histogram!/gauge! call site in router/src/lib.rs, router/src/http/server.rs and router/src/grpc/server.rs, plus all four Matcher declarations in router/src/prometheus.rs. Do not read this record's metric_name as something the engine exports; it is the name that is missing.

Cross-references