vLLM Production Stack issue #687 - router crashes from OOM, prefix cache HashTrie memory usage is unbounded

Opened 2025-09-08, still open as of 2026-10-04. Reporter: 'We found an issue during the stability test that router Pod keeps restarting after running for a certain amount of time. It seems the router is restarted by k8s because of OOM. We configured the router Pod limit with 2G memory initially. After raising to 16G, we observed that the memory usage of router is unbounded and keeps ramping up.' Repro: multiple vLLM model server pods behind the router, 2 GB limit, continuous long-document QA benchmark for over 2 hours, router periodically OOM-restarted. Requested fix: cap the HashTrie and GC stale nodes by TTL or LRU. Important as a topology fact: the router is a stateful, memory-consuming process, not a stateless L7 proxy - the control plane can be the thing that falls over while every data-plane engine is healthy.