vLLM Production Stack - vllm_router README (request router)

Fetched 2026-10-04, main branch, 186 lines. Router 'key features' verbatim: 'Support routing to endpoints that run different models'; 'Exporting observability metrics for each serving engine instance, including QPS, time-to-first-token (TTFT), number of pending/running/finished requests, and uptime'; 'Support automatic service discovery and fault tolerance by Kubernetes API'; 'Model aliases'; 'Multiple different routing algorithms: Round-robin routing / Session-ID based routing / (WIP) prefix-aware routing'. Flags quoted verbatim: --service-discovery {static|k8s}, --static-backends, --static-models, --static-aliases, --static-backend-health-checks ('Enable this flag to make vllm-router check periodically if the models work by sending dummy requests to their endpoints'), --k8s-port (default 8000), --k8s-namespace, --k8s-label-selector, --routing-logic {roundrobin|session} (required), --session-key, --engine-stats-interval (default 30 s), --request-stats-window (default 60 s), --log-stats-interval (default 10), --dynamic-config-yaml/json, Sentry sampling flags. NOTE the README still lists prefix-aware routing as '(WIP)' while the docs site ships a prefix-aware tutorial and the router tree has a PrefixAwareRouter class - the router README is behind the code.