inference/research
verifiedhigh · 0.85updated 2026-10-03

SGLang issue #36140: DFLASH speculative decoding unsupported under PD disaggregation (spec_info None crash, then watchdog self-kill)

forum

Fields

id
sources/pd-gh-sglang-36140-dflash-spec-under-pd
Publisher
sgl-project/sglang
Kind
forum
Published
2026-08-24
Accessed
2026-10-03
notes
Open as of 2026-10-03. Speculative decoding under disaggregation is per-algorithm, not automatic: 'DSPARK and EAGLE both work under PD, but DFLASH has no disagg wiring.' Crash 1: SpeculativeAlgorithm.build_disagg_draft_input() implements EAGLE and DSPARK but returns None for DFLASH, so DisaggDecodeScheduleBatchMixin never sets batch.spec_info for relayed requests and the first update_running_batch on the decode side dies with AttributeError: 'NoneType' object has no attribute 'prepare_for_decode' at sglang/srt/speculative/spec_utils.py:1010. A contributing factor named by the reporter: the PD-decode event loop also runs update_running_batch on IDLE batches, which never go through build_disagg_draft_input at all. Crash 2 follows once crash 1 is locally patched. Env: lmsysorg/sglang:kimi-k3 (0.5.16-based), Kimi-K3 + Kimi-K3-DFlash draft, 8x RTX 6000D sm_120 across 4 prefill + 4 decode nodes, KV transfer Mooncake RDMA 0.3.12.post1. Relevant because it shows the decode-side relay path is a distinct code path from the normal decode path, which is where disaggregation-specific state handling gaps live.

Notes

Open as of 2026-10-03. Speculative decoding under disaggregation is per-algorithm, not automatic: 'DSPARK and EAGLE both work under PD, but DFLASH has no disagg wiring.' Crash 1: SpeculativeAlgorithm.build_disagg_draft_input() implements EAGLE and DSPARK but returns None for DFLASH, so DisaggDecodeScheduleBatchMixin never sets batch.spec_info for relayed requests and the first update_running_batch on the decode side dies with AttributeError: 'NoneType' object has no attribute 'prepare_for_decode' at sglang/srt/speculative/spec_utils.py:1010. A contributing factor named by the reporter: the PD-decode event loop also runs update_running_batch on IDLE batches, which never go through build_disagg_draft_input at all. Crash 2 follows once crash 1 is locally patched. Env: lmsysorg/sglang:kimi-k3 (0.5.16-based), Kimi-K3 + Kimi-K3-DFlash draft, 8x RTX 6000D sm_120 across 4 prefill + 4 decode nodes, KV transfer Mooncake RDMA 0.3.12.post1. Relevant because it shows the decode-side relay path is a distinct code path from the normal decode path, which is where disaggregation-specific state handling gaps live.

Cross-references

This record references no other records.