vLLM issue #24885 [RFC] Clarifying vLLM Shutdown Semantics
Fields
sources/ops-vllm-shutdown-rfc-24885Notes
Maintainer RFC (labels feature request, stale, closed as not planned) documenting DESIRED semantics rather than shipped ones, so it is evidence of intent and of acknowledged gaps. Notable admissions, verbatim: 'vLLM does not support any partial restart scenario, for example a "reload model" or "restart workers" capability.' Expected online-serving semantics are 'Stop accepting new HTTP requests immediately. Allow in-flight requests to complete, bounded by a configurable max wait time.' It states 'Each request is relatively expensive (100ms+)' and that deployments 'typically involve multiple replicas behind a load balancer (e.g. Kubernetes Service)'. Under Observability it asks whether 'I am shutting down' status 'should be observable, e.g. via the /health API' - an open question, i.e. no drain-state health signal exists. The KV Transfer section asks that KVConnector.shutdown() 'allow the connector to defer worker process (and parents) shutdown until all pending KV transfers complete or time out'. The RFC notes terminationGracePeriodSeconds defaults to 30s and mentions a preStop sleep to allow straggling requests.
Cross-references
This record references no other records.