verifiedhigh · 0.92updated 2026-10-04

DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving (OSDI 2024)

paper

Fields

sources/sched-distserve-osdi24

URL

https://arxiv.org/abs/2401.09670

Publisher

arXiv

Kind

paper

Published

2024-01

Accessed

2026-10-04

Archived URL

https://arxiv.org/html/2401.09670v3

notes

Disaggregates prefill and decode onto separate GPUs. Key results: 7.4x more requests or 12.6x tighter SLO vs state-of-the-art. The arXiv HTML was fetched and verified.

Notes

Disaggregates prefill and decode onto separate GPUs. Key results: 7.4x more requests or 12.6x tighter SLO vs state-of-the-art. The arXiv HTML was fetched and verified.

Cross-references

This record references no other records.