DistServe (arXiv 2401.09670) · inference-research

Fields

id

sources/pap-sys-distserve-arxiv

URL

https://arxiv.org/abs/2401.09670

Publisher

arXiv (Peking University / UC San Diego / StepFun)

Kind

paper

Published

2024-01-18

Accessed

2026-10-03

notes

arXiv comment field states 'OSDI 2024'. Venue independently confirmed against the USENIX OSDI '24 technical-sessions page, which lists 'DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving' at usenix.org/conference/osdi24/presentation/zhong-yinmin. Abstract page gives the headline claim: 7.4x more requests or 12.6x tighter SLO vs state-of-the-art, >90% of requests within latency constraints.

Notes

arXiv comment field states 'OSDI 2024'. Venue independently confirmed against the USENIX OSDI '24 technical-sessions page, which lists 'DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving' at usenix.org/conference/osdi24/presentation/zhong-yinmin. Abstract page gives the headline claim: 7.4x more requests or 12.6x tighter SLO vs state-of-the-art, >90% of requests within latency constraints.

Cross-references

This record references no other records.