inference/research
verifiedmedium · 0.80updated 2026-10-04

vLLM issue #24885 [RFC] Clarifying vLLM Shutdown Semantics

forum

Fields

id
sources/ops-vllm-shutdown-rfc-24885
Publisher
vLLM project
Kind
forum
Published
2025-09-15
Accessed
2026-10-04
notes
Maintainer RFC (labels feature request, stale, closed as not planned) documenting DESIRED semantics rather than shipped ones, so it is evidence of intent and of acknowledged gaps. Notable admissions, verbatim: 'vLLM does not support any partial restart scenario, for example a "reload model" or "restart workers" capability.' Expected online-serving semantics are 'Stop accepting new HTTP requests immediately. Allow in-flight requests to complete, bounded by a configurable max wait time.' It states 'Each request is relatively expensive (100ms+)' and that deployments 'typically involve multiple replicas behind a load balancer (e.g. Kubernetes Service)'. Under Observability it asks whether 'I am shutting down' status 'should be observable, e.g. via the /health API' - an open question, i.e. no drain-state health signal exists. The KV Transfer section asks that KVConnector.shutdown() 'allow the connector to defer worker process (and parents) shutdown until all pending KV transfers complete or time out'. The RFC notes terminationGracePeriodSeconds defaults to 30s and mentions a preStop sleep to allow straggling requests.

Notes

Maintainer RFC (labels feature request, stale, closed as not planned) documenting DESIRED semantics rather than shipped ones, so it is evidence of intent and of acknowledged gaps. Notable admissions, verbatim: 'vLLM does not support any partial restart scenario, for example a "reload model" or "restart workers" capability.' Expected online-serving semantics are 'Stop accepting new HTTP requests immediately. Allow in-flight requests to complete, bounded by a configurable max wait time.' It states 'Each request is relatively expensive (100ms+)' and that deployments 'typically involve multiple replicas behind a load balancer (e.g. Kubernetes Service)'. Under Observability it asks whether 'I am shutting down' status 'should be observable, e.g. via the /health API' - an open question, i.e. no drain-state health signal exists. The KV Transfer section asks that KVConnector.shutdown() 'allow the connector to defer worker process (and parents) shutdown until all pending KV transfers complete or time out'. The RFC notes terminationGracePeriodSeconds defaults to 30s and mentions a preStop sleep to allow straggling requests.

Cross-references

This record references no other records.