FlashInfer: Kernel Library for LLM Serving

Fields

id

sources/src5-flashinfer-github

URL

https://github.com/flashinfer-ai/flashinfer

Publisher

flashinfer-ai

Kind

repo

Published

2024-01-01

Accessed

2026-10-04

notes

FlashInfer is a kernel library for LLM serving. Inspired by FlashAttention, vLLM, stream-K, CUTLASS, AITemplate. Provides attention kernels, MoE support, JIT compilation, profiling tools. 6.5k stars. Used by vLLM, TensorRT-LLM, SGLang, TGI, MLC-LLM, LightLLM, lorax, ScaleLLM. Paper: arXiv:2501.01005.

Notes

FlashInfer is a kernel library for LLM serving. Inspired by FlashAttention, vLLM, stream-K, CUTLASS, AITemplate. Provides attention kernels, MoE support, JIT compilation, profiling tools. 6.5k stars. Used by vLLM, TensorRT-LLM, SGLang, TGI, MLC-LLM, LightLLM, lorax, ScaleLLM. Paper: arXiv:2501.01005.

Cross-references

This record references no other records.