Token Latency Fairness: Performance Isolation for Multi-Tenant LLM Serving (arXiv 2609.18112)

Fields

id

sources/pap-sys-fairinference-arxiv

URL

https://arxiv.org/abs/2609.18112

Publisher

arXiv (University of California, Berkeley)

Kind

paper

Published

2026-09-16

Accessed

2026-10-03

notes

arXiv cats cs.DC/cs.LG, no venue claim in comment. Introduces a delta-token fairness guarantee: if a well-behaved client's token takes d time units in isolation, it is produced within d + delta under multi-tenant execution. Authors include Dev Bali, Soujanya Ponnapalli, Yichuan Wang, Natacha Crooks, Scott Shenker, Matei Zaharia.

Notes

arXiv cats cs.DC/cs.LG, no venue claim in comment. Introduces a delta-token fairness guarantee: if a well-behaved client's token takes d time units in isolation, it is produced within d + delta under multi-tenant execution. Authors include Dev Bali, Soujanya Ponnapalli, Yichuan Wang, Natacha Crooks, Scott Shenker, Matei Zaharia.

Cross-references

This record references no other records.