FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving (MLSys 2025)

Fields

id

sources/sched-flashinfer-mlsys25

URL

https://arxiv.org/abs/2501.01005

Publisher

arXiv

Kind

paper

Published

2025-01

Accessed

2026-10-04

Archived URL

https://arxiv.org/html/2501.01005v2

notes

FlashInfer paper. Key results: 29-69% inter-token-latency reduction vs compiler backends, 28-30% latency reduction for long-context inference, 13-17% speedup for LLM serving with parallel generation. Introduces load-balanced scheduling compatible with CUDA Graphs. The arXiv abstract was fetched and verified.

Notes

FlashInfer paper. Key results: 29-69% inter-token-latency reduction vs compiler backends, 28-30% latency reduction for long-context inference, 13-17% speedup for LLM serving with parallel generation. Introduces load-balanced scheduling compatible with CUDA Graphs. The arXiv abstract was fetched and verified.

Cross-references

This record references no other records.