SGLang docs: PD Disaggregation
Fields
sources/pd-sglang-pd-disagg-docsNotes
States the two failure modes disaggregation removes: 'Prefill Interruption: Incoming prefill batches frequently interrupt ongoing decode batches' and 'DP Attention Imbalance: In data-parallel (DP) attention, one DP worker may process a prefill batch while another handles a decode batch simultaneously, leading to increased decode latency'. Transfer path: 'Currently, we support Mooncake and NIXL as the transfer engine', with per-instance flags such as --disaggregation-ib-device selecting specific RDMA HCAs, and an 'ascend' transfer backend for NPUs. Scheduling detail worth recording: --disaggregation-decode-allocation-policy prefill_complete defers decode-side KV slot reservation until prefill signals readiness, because 'By default, decode reserves each request's destination KV slots before prefill finishes. Long prefills can hold these slots while decode waits for the KV data.'
Cross-references
This record references no other records.