inference/research
verifiedhigh · 0.85updated 2026-10-03

vLLM issue #56828: KV Offload P2P peer-down on one rank does not fail loads or block new loads to sibling ranks

forum

Fields

id
sources/pd-gh-vllm-56828-p2p-peer-down-no-fail
Publisher
vllm-project/vllm
Kind
forum
Published
2026-09-14
Accessed
2026-10-03
notes
Disaggregated P/D deployment (llm-d), multi-rank prefill pods with 8 P2P control ports. When a source pod died mid-transfer, the consumer marked only that rank's peer down and then submitted a NEW load to a sibling rank of the same dead pod, which ran the full 30s load timeout; requests bound to that source fell back to local prefill only after 36-43s. Confirmed by maintainer he-yufeng against main: peer_id is per control port so an 8-rank pod registers as 8 independent peers; _reap_dead_sessions only reaps the session whose own ZMQ connection died; there is no progress field anywhere in the load path, so 'the 30s walk is the only terminator'. Also: a rank that was still answering control messages while its pod failed health checks meant the ZMQ heartbeat never fired - the data path stalled on a control-alive peer.

Notes

Disaggregated P/D deployment (llm-d), multi-rank prefill pods with 8 P2P control ports. When a source pod died mid-transfer, the consumer marked only that rank's peer down and then submitted a NEW load to a sibling rank of the same dead pod, which ran the full 30s load timeout; requests bound to that source fell back to local prefill only after 36-43s. Confirmed by maintainer he-yufeng against main: peer_id is per control port so an 8-rank pod registers as 8 independent peers; _reap_dead_sessions only reaps the session whose own ZMQ connection died; there is no progress field anywhere in the load path, so 'the 30s walk is the only terminator'. Also: a rank that was still answering control messages while its pod failed health checks meant the ZMQ heartbeat never fired - the data path stalled on a control-alive peer.

Cross-references

This record references no other records.