inference/research
verifiedhigh · 0.90updated 2026-10-04

Envoy AI Gateway (Agent Router) - AIGatewayRoute + InferencePool guide

whitepaper

Fields

id
sources/dep-envoy-ai-gateway-inferencepool-guide
Publisher
Envoy AI Gateway project (now Agentic AI Foundation)
Kind
whitepaper
Accessed
2026-10-04
notes
Fetched 2026-10-04, version 0.5 docs. Header note: 'Formerly Envoy AI Gateway - now an Agent Router project. Same code, same maintainers. CRD, API, and CLI names are unchanged.' Requires a Kubernetes cluster with Gateway API support, then installs the Gateway API Inference Extension CRDs at v1.0.1, enables InferencePool support in Envoy Gateway and restarts the deployment. 'Advanced Features' enumerated: Model-Based Routing, Token Rate Limiting, Enhanced Observability - i.e. the gateway layer's added value is model-aware routing, per-token (not per-request) quota, and observability. Backend set includes 'AIServiceBackend for Mixed Routing' so one route can span self-hosted and vendor endpoints. Confirms the install is multi-component (Gateway CRDs + inference-extension CRDs + envoy-gateway + ai-gateway-controller + GatewayClass) - the control plane itself is a distributed system with its own failure modes.

Notes

Fetched 2026-10-04, version 0.5 docs. Header note: 'Formerly Envoy AI Gateway - now an Agent Router project. Same code, same maintainers. CRD, API, and CLI names are unchanged.' Requires a Kubernetes cluster with Gateway API support, then installs the Gateway API Inference Extension CRDs at v1.0.1, enables InferencePool support in Envoy Gateway and restarts the deployment. 'Advanced Features' enumerated: Model-Based Routing, Token Rate Limiting, Enhanced Observability - i.e. the gateway layer's added value is model-aware routing, per-token (not per-request) quota, and observability. Backend set includes 'AIServiceBackend for Mixed Routing' so one route can span self-hosted and vendor endpoints. Confirms the install is multi-component (Gateway CRDs + inference-extension CRDs + envoy-gateway + ai-gateway-controller + GatewayClass) - the control plane itself is a distributed system with its own failure modes.

Cross-references

This record references no other records.