XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models

States the per-token grammar cost problem directly in its abstract: 'executing context-free grammar requires going through several stack states over all tokens in vocabulary during runtime, bringing non-negligible overhead for structured generation.' Its four fixes are the workarounds recorded on the constrained-decoding flop record: split the vocabulary into context-independent tokens that can be prechecked and context-dependent tokens interpreted at runtime, expand the grammar context and reduce the count of context-independent tokens, use a persistent stack for the dependent checks, and co-design with the inference engine to overlap grammar computation with GPU execution. Reports up to 100x speedup over existing solutions and 'near-zero overhead structure generation' when co-designed. | SOURCE AUDIT 2026-10-03: the 'publisher' field held a paragraph of prose (authors + affiliations, 56 chars) instead of a publisher name; trimmed to 'arXiv' with the authorship detail moved here.