inference/research
verifiedhigh · 0.90updated 2026-10-05

TEI request duration histogram

niche-text-embeddings-inference · histogram

Fields

id
metric-exposures/w5e-tei-te-request-duration
Metric name
te_request_duration
Metric type
histogram
Unit
seconds
Exposure
prometheus
Endpoint
:9000/metrics (--prometheus-port, default 9000)
Histogram buckets
0.00002250.000033750.0000506250000000000040.00007593750.000113906250000000010.0001708593750.00025628906250.000384433593750.00057665039062500010.00086497558593750.00129746337890625010.0019461950683593750.00291929260253906250.0043789389038085940.0065684083557128910.0098526125335693360.0147789188003540040.0221683782005310070.033252567300796510.0498788509511947640.074818276426792150.112227414640188220.168341121960282340.25251168294042350.378767524410635270.56815128661595280.85222692992392931.2783403948858941.9175105923288412.87626588849326134.3143988327398926.4715982491098389.70739737366475614.56109606049713521.841644090745703
Enabled by default
yes
Phase scoped
no
notes
TEI main, router/src/lib.rs line ~661: metrics::histogram!("te_request_duration") .record(self.start_time.elapsed().as_secs_f64()). Buckets come from router/src/prometheus.rs: Matcher::Suffix("duration") catches this name and builds a 35-bound geometric ladder starting at 1e-5 and multiplying by 1.5 each step. The arithmetic detail that matters: the loop multiplies FIRST and then pushes, so the first emitted bound is 1.5e-5 and 1e-5 itself is NOT a bound. Effective range is roughly 1.5e-5 s to 1.5e-5 * 1.5^34 ~= 2.6e4 s. This is whole-request latency, not a phase breakdown. NOTE the Suffix matcher is a substring match, so te_request_duration, te_request_queue_duration, te_request_tokenization_duration and te_request_inference_duration ALL share this one ladder - they are not separately configured.

Notes

TEI main, router/src/lib.rs line ~661: metrics::histogram!("te_request_duration") .record(self.start_time.elapsed().as_secs_f64()). Buckets come from router/src/prometheus.rs: Matcher::Suffix("duration") catches this name and builds a 35-bound geometric ladder starting at 1e-5 and multiplying by 1.5 each step. The arithmetic detail that matters: the loop multiplies FIRST and then pushes, so the first emitted bound is 1.5e-5 and 1e-5 itself is NOT a bound. Effective range is roughly 1.5e-5 s to 1.5e-5 * 1.5^34 ~= 2.6e4 s. This is whole-request latency, not a phase breakdown. NOTE the Suffix matcher is a substring match, so te_request_duration, te_request_queue_duration, te_request_tokenization_duration and te_request_inference_duration ALL share this one ladder - they are not separately configured.

Cross-references