inference/research
verifiedhigh · 0.90updated 2026-10-05

TEI input-length histogram

niche-text-embeddings-inference · histogram

Fields

id
metric-exposures/w5e-tei-te-request-input-length
Metric name
te_request_input_length
Metric type
histogram
Unit
tokens
Exposure
prometheus
Endpoint
:9000/metrics (--prometheus-port, default 9000)
Histogram buckets
1248163264128256512
Enabled by default
yes
notes
Matcher::Full("te_request_input_length") with buckets (0..20).map(2^i).filter(<= max_input_length). The bucket list is therefore DEPLOYMENT-DEPENDENT: it is filtered against the router's --max-input-length at startup, so the ladder on one deployment is not the ladder on another. This record's list is the ladder for max_input_length = 512, which yields bounds 1,2,4,...,512 (10 bounds). A reader quantiling this histogram must know the serving --max-input-length or the interpolation is wrong - the same failure mode as vLLM's request_tokens family.

Notes

Matcher::Full("te_request_input_length") with buckets (0..20).map(2^i).filter(<= max_input_length). The bucket list is therefore DEPLOYMENT-DEPENDENT: it is filtered against the router's --max-input-length at startup, so the ladder on one deployment is not the ladder on another. This record's list is the ladder for max_input_length = 512, which yields bounds 1,2,4,...,512 (10 bounds). A reader quantiling this histogram must know the serving --max-input-length or the interpolation is wrong - the same failure mode as vLLM's request_tokens family.

Cross-references