Dynamo lib/llm/src/http/service/metrics.rs - frontend TTFT/ITL/request histograms and their env-configurable buckets
Fields
sources/w5e-dynamo-http-service-metricsNotes
Read at main 2026-10-05. 5234 lines. Buckets are NOT literals: generate_log_buckets(min,max,count) produces log-spaced values rounded to 2 significant figures, always starting with a 0.0 bound, then dedups. Defaults: request duration (1.0, 512.0, 10); input sequence (50.0, 128000.0, 12); output sequence (50.0, 32000.0, 10); TTFT (0.001, 480.0, 18); ITL (0.001, 80.0, 20); embedding latency (0.001, 10.0, 14). Each is overridable per-axis via DYN_METRICS_{REQUEST_DURATION,INPUT_SEQUENCE,OUTPUT_SEQUENCE,TTFT,ITL,EMBEDDING_LATENCY}_{MIN,MAX,COUNT}, with a deprecated DYN_HISTOGRAM_-prefixed fallback still honoured and a warning. The shipped ITL ladder is asserted as an exact contract in the test itl_default_buckets_reach_80_seconds, whose comment records that the ceiling was once 2.0 s so p99 pinned at exactly 2.0.
Cross-references
This record references no other records.