Design goal
throughput-firstfine-grained compute isolationshared-GPU-memory disaggregation
Scheduling
continuous-batchingdisaggregated-prefill-decodenone (radix cache disabled in the documented setup)
Notable features
Phase-wise disaggregated computation with UNIFIED storage - prefill and decode share GPU memory rather than transferring KV across a link, which removes the handoff bandwidth cost entirelyFine-grained compute isolation via CUDA MPS rather than separate devices or processes: CUDA_MPS_ENABLE_PER_CTX_DEVICE_MULTIPROCESSOR_PARTITIONING=1 splits SMs between the prefill and decode contexts on one GPUEnabled by a single flag (--enable-semi-pd) on a forked SGLang codebaseDual NVIDIA/AMD build pathsPaper: arXiv:2504.19867
Hardware caveats
Research prototype, self-described as 'a research prototype' with 'no complete feature parity with open-source SGLang'; created 2025-04-07, last push 2025-12-25, ~136 starsRequires CUDA MPS to be enabled manually on NVIDIA; on AMD MPS is on by defaultDoes not share activations between prefill and decode, so 'may result in slightly higher memory usage compared to the original SGLang' - the documented remedy is lowering --mem-fraction-staticThe documented launch command includes --disable-radix-cache, so the prefix-reuse path is not exercised in the reference setupMPS partitioning is a coarse isolation mechanism; it does not give the hard isolation a separate node or process would
Best for
Single-node disaggregation where the interconnect is too slow to make a KV handoff pay - the shared-memory design makes the transfer cost zero by constructionStudying the alternative to the copy: if you can avoid moving KV rather than move it faster, the cost model changes shape entirelyNVIDIA and AMD comparison work, since both build paths are documented
Avoid for
Production - it is a fork with reduced feature parity, and a fork of a fast-moving engine is a maintenance liabilityMulti-node deployments, which is the regime every other system here targetsWorkloads relying on prefix/radix caching, which the reference setup disables
notes
Research prototype (arXiv:2504.19867), explicitly self-described as such. KV transfer path: NONE, by design - this is the record's distinguishing claim. Semi-PD's premise is that the disaggregation tax is the KV copy, so if prefill and decode can address the same GPU memory, the tax is eliminated and the only cost is the compute partitioning. That is a genuinely different cost model from DistServe/Splitwise (which pay bytes x bandwidth) and from Mooncake (which pays bytes x fabric bandwidth), and it is why this record sits in the repo next to them rather than as a variant. The cost it pays instead: no hard isolation (MPS partitions SMs, not failures), higher memory use from not sharing activations, and reduced SGLang feature parity. ||| MAINTENANCE (dated fact, GitHub API, 2026-10-04): last commit 2025-12-25 (9.3 months before this record), NO GitHub release or tag found, 4 contributors (distinct committers including anonymous), open_issues_count 8 (issues PLUS pull requests - see [[ev-github-open-issues-count-includes-pull-requests]]), not archived, 136 stars. See [[ev-maintenance-status-is-four-dated-facts]] for how to read this fact set. ||| DEPLOYMENT: NO EVIDENCE OF PRODUCTION DEPLOYMENT FOUND as of 2026-10-04 - repositories, READMEs, docs and issue trackers were searched. See [[ev-production-evidence-is-scarce-and-nearly-always-single-site]] for the grading scheme; note that for an artifact-evaluation or paper-code release, non-deployment is documented intent rather than a negative finding.