Semi-PD GitHub repository (infinigence/Semi-PD)

Apache-2.0, 136 stars, created 2025-04-07, last push 2025-12-25 - research prototype. Self-described: 'A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation', paper arXiv:2504.19867. Two architectural choices that distinguish it from DistServe/Splitwise: it keeps prefill and decode KV in SHARED GPU memory rather than moving KV over a link, and it achieves compute isolation with CUDA MPS (CUDA_MPS_ENABLE_PER_CTX_DEVICE_MULTIPROCESSOR_PARTITIONING=1) rather than separate processes/devices. README states it is 'a fork of the SGLang project', that it 'does not have complete feature parity with open-source SGLang', and that it does 'not share activations between the prefill and decode phases, which may result in slightly higher memory usage'. Must be launched with --disable-radix-cache in the documented setup.