inference/research
verifiedhigh · 0.93updated 2026-10-05

Envoy AI Gateway internal/metrics/genai.go - OTel GenAI semconv metrics with explicit bucket boundaries

repo

Fields

id
sources/w5e-envoy-ai-gateway-metrics-genai
Publisher
envoyproxy
Kind
repo
Accessed
2026-10-05
notes
Read at main 2026-10-05. Four metric names declared as constants following the OpenTelemetry GenAI semantic conventions: gen_ai.client.token.usage, gen_ai.server.request.duration, gen_ai.server.time_to_first_token, gen_ai.server.time_per_output_token. All four are Float64Histogram with EXPLICIT bucket boundaries via metric.WithExplicitBucketBoundaries - this repo is unusual in not relying on SDK default buckets, so the ladders are exactly these literals. Units are declared: 'token' for the usage histogram, 's' for the three latency histograms. Operations enumerated: chat, completion, embeddings, messages, image_generation, responses, speech, transcription, translation, rerank, systemone, tokenize, responses_input_tokens, count_tokens. Providers: openai, azure.openai, aws.bedrock, aws.anthropic, gcp.vertex_ai, gcp.anthropic, anthropic, cohere, typesafe.

Notes

Read at main 2026-10-05. Four metric names declared as constants following the OpenTelemetry GenAI semantic conventions: gen_ai.client.token.usage, gen_ai.server.request.duration, gen_ai.server.time_to_first_token, gen_ai.server.time_per_output_token. All four are Float64Histogram with EXPLICIT bucket boundaries via metric.WithExplicitBucketBoundaries - this repo is unusual in not relying on SDK default buckets, so the ladders are exactly these literals. Units are declared: 'token' for the usage histogram, 's' for the three latency histograms. Operations enumerated: chat, completion, embeddings, messages, image_generation, responses, speech, transcription, translation, rerank, systemone, tokenize, responses_input_tokens, count_tokens. Providers: openai, azure.openai, aws.bedrock, aws.anthropic, gcp.vertex_ai, gcp.anthropic, anthropic, cohere, typesafe.

Cross-references

This record references no other records.