llm-d Autoscaling overview (WVA deprecation and capability gaps against KEDA + EPP)
Fields
sources/ops-llmd-autoscaling-wva-deprecatedNotes
The thesis, verbatim: 'llm-d scales on signals emitted by the Endpoint Picker (EPP) - queued requests, active concurrency, token backlog, or estimated latency against an SLO - because accelerator utilization is an unreliable proxy for LLM load: GPU utilization can sit near 100% during active batching whether the server is lightly loaded or saturated.' Flags 'WVA is deprecated. Use KEDA + EPP for new deployments.' The deprecation is NOT feature-parity, stated as a table: multiple variants of one model - 'WVA: Optimally placed across models and topologies to minimize cost' versus 'KEDA + EPP: Each Deployment scales independently; no cost-aware preference between variants'; limited accelerators - WVA fair-share versus Kueue below the autoscaler; pending pods awareness - WVA incorporated pending pods versus KEDA 'a trigger can read replica counts from kube-state-metrics'; scale to zero - both claim support. Closing line: 'If a deployment depends on cost-aware placement across variants or on fair sharing of a scarce accelerator pool, stay on WVA for now.' This matters for an honest ops record: the officially recommended path has a documented capability hole relative to the deprecated one.
Cross-references
This record references no other records.