SGLang: Efficient Execution of Structured Language Model Programs

Fields

id

sources/flop-sglang

URL

https://arxiv.org/abs/2312.07104

Publisher

arXiv (SGLang team)

Kind

paper

Published

2023-12-12

Accessed

2026-10-03

notes

RadixAttention: a radix tree over the KV cache enabling prefix-aware reuse across requests, plus compressed finite state machines for constrained decoding. Up to 6.4x throughput vs prior systems on agent, RAG and JSON-decoding workloads.

Notes

RadixAttention: a radix tree over the KV cache enabling prefix-aware reuse across requests, plus compressed finite state machines for constrained decoding. Up to 6.4x throughput vs prior systems on agent, RAG and JSON-decoding workloads.

Cross-references

This record references no other records.