vLLM Production Stack - Prefix Aware Routing tutorial (docs)
Fields
sources/dep-vllm-prodstack-prefix-aware-tutorialNotes
Fetched 2026-10-04. States the design intent verbatim: 'Prefix aware routing ensures that subsequent requests with the same prompt prefix are routed to the same instance, maximizing KV cache utilization and improving performance.' Deploys two vLLM instances with helm -f tutorials/assets/values-18-prefix-aware.yaml, then port-forwards svc/vllm-router-service 30080:80. The demo sends two requests with a shared prefix and shows LMCache log lines proving the second one hits the first's cache: 'Scheduled to load 5 tokens for request cmpl-... (lmcache...vllm_v1_adapter)' and 'Retrieved 6 out of 6 out of total 6 tokens'. Page requires 'A Kubernetes environment with GPU support'. This is the canonical statement that the router's decision input is the prompt PREFIX, not the model name alone.
Cross-references
This record references no other records.