llm-d Graceful Shutdown and Request Draining guide

Verbatim on the default: 'Default - vLLM immediately aborts in-flight requests and exits. Those requests fail with an error status code.' And 'With --shutdown-timeout N - vLLM catches SIGTERM and continues serving the currently running requests for up to N seconds. After the timeout, any requests still in flight are aborted and returned an error.' The operator trap, flagged IMPORTANT: 'terminationGracePeriodSeconds must be greater than --shutdown-timeout (plus a small buffer for the preStop hook and process cleanup). Otherwise Kubernetes sends SIGKILL before the drain completes, and the graceful window is never fully used.' Recommended config is terminationGracePeriodSeconds 120 with --shutdown-timeout 90 plus a readiness probe on /v1/models, and the sizing rule 'Set --shutdown-timeout to roughly the p99 request duration you want to allow to complete on drain.' The pod termination sequence is enumerated as: Terminating, InferencePool endpoint removal (so the router stops sending new traffic), preStop hook, SIGTERM, then terminationGracePeriodSeconds (default 30s) before SIGKILL. Also documents request cancellation on client disconnect, where 'vLLM triggers its abort codepath, releasing the KV cache and compute resources held by that request', and that during EPP graceful shutdown the flow controller 'evicts queued requests with a retryable 503 Service Unavailable (outcome rejected-shutting-down), signaling transient unavailability so that clients and upstream gateways retry rather than treating the drain as a hard fault', with the caveat that 'These retryable semantics only apply while the EPP is still reachable over its ext_proc stream.'