Envoy AI Gateway time-per-output-token histogram (OTel gen_ai semconv)
Fields
metric-exposures/w5e-envoy-ai-gateway-gen-ai-server-time-per-output-tokenNotes
mustRegisterHistogram with WithDescription('Time per output token generated after the first token for successful responses.') - the 'after the first token' and 'for successful responses' qualifiers are load-bearing. The first token is excluded, so this is not simply request_duration/n_tokens, and FAILED requests contribute nothing, so a regression that manifests as errors rather than slowness does not move this series at all. 13 explicit bounds, tight around interactive decode rates. Envoy AI Gateway main (envoyproxy/ai-gateway), read 2026-10-05. Metrics go through an OpenTelemetry MeterProvider (internal/metrics/metrics.go NewMeterFromEnv), not a bare prometheus_client registry: a Prometheus reader is ALWAYS attached, and a console or OTLP exporter is added additionally when OTEL_METRICS_EXPORTER is 'console', or when it is neither 'none' nor 'prometheus' and an OTLP endpoint env var is set. OTEL_SDK_DISABLED=true turns off the extra exporters but not the Prometheus reader. That means the metric names are dotted OTel names (gen_ai.server.*), which the Prometheus exporter renders with dots replaced by underscores and a unit suffix appended - so the scraped series is NOT the literal constant in genai.go. Names follow the OpenTelemetry GenAI semantic conventions, not any engine's private scheme.