Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloads (arXiv 2401.11181)

Fields

id

sources/pap-sys-tetriinfer-arxiv

URL

https://arxiv.org/abs/2401.11181

Publisher

arXiv (SUSTech / HKUST)

Kind

paper

Published

2024-01-20

Accessed

2026-10-03

notes

arXiv cats cs.DC, comment field empty (no venue claim). The system is named TetriInfer in the body; the arXiv title is 'Inference without Interference'. Abstract: schedules and groups requests by characteristics so that prefill and decode do not interfere, for mixed downstream workloads.

Notes

arXiv cats cs.DC, comment field empty (no venue claim). The system is named TetriInfer in the body; the arXiv title is 'Inference without Interference'. Abstract: schedules and groups requests by characteristics so that prefill and decode do not interfere, for mixed downstream workloads.

Cross-references

This record references no other records.