Towards Load-Aware Prefill Deflection for Disaggregated LLM Serving (Kairos, arXiv:2607.02043)

Kairos: proactive prefill-deflecting scheduler for disaggregated LLM serving on vLLM 0.18.1 with DeepSeek-V2-Lite on 2P2D (2 prefill + 2 decode nodes, each A100 80GB). Key finding: prefill execution accounts for only 2-23% of P95 TTFT; remaining 77-98% is prefill queue wait and inter-node KV-cache transfer. Five workloads tested (LPLD, LPHD, HPLD, HPHD, Bursty) with production-derived traces normalized to 32K context. Deflecting prefill to decode nodes (building KV cache in place) eliminates KV transfer and reduces P95 TTFT by up to 81%.