XGrammar
cuda, cpu, metal, opencl, vulkan, tpu, xnnpack
Fields
id
engines/niche-xgrammarRepo
mlc-ai/xgrammar
Languages
C++PythonTypeScriptCUDASwiftC
License
Apache-2.0
First release
2024
Design goal
low-latencyproductionportablestructured-output
Scheduling
none
Backends
cudacpumetalopenclvulkantpuxnnpack
Formats
tokenizer-vocab
Notable features
Not an inference engine - a structured-generation library that sits on the per-token critical path inside someone else's engine. The distinction matters for how you read the recordClaims near-zero overhead in JSON generation, which is the entire design target: constraint checking that does not eat decode throughputIntegrated into vLLM (2024/12) and SGLang (2024/11) as the default structured-generation backend, per the README. That default status is why it displaced lm-format-enforcer (see [[niche-lm-format-enforcer]]) and part of what outlines competes withWritten in C++ with Python, JavaScript and Swift APIs, plus a `pip install xgrammar[metal]` variant for Apple MPSv0.2.x line, actively released (v0.2.8 on 2026-09-24)System paper (arXiv:2411.15100) is the primary source for the overhead argument
Hardware caveats
Backend-agnostic by design - it is a token-level automaton over a tokenizer vocabulary, so it has no accelerator kernels of its own and inherits whatever the host engine runs on. supported_backends here reflects host reach, not XGrammar's own code paths. Do not read this list as 'XGrammar has a CUDA path'The overhead claim is a claim. 'Near-zero overhead in JSON generation' comes from the project itself with no benchmark table in the READMEConstraint checking is still per-token work on the critical path. With free-form generation and no active constraint, the library should cost approximately nothing; with a tight schema it costs real time and the win comes from not needing to parse and retryHardware support is bounded by the host engine. On [[niche-vllm-ascend]] or [[niche-tt-metal]], structured output works only if that engine's logit processor can call out to XGrammarREPO MOVE WARNING: the canonical repo is mlc-ai/xgrammar. The handle github.com/xgrammar-ai/xgrammar returns HTTP 404 as of 2026-10-03, so any blog post or config referencing it is stale. Worth knowing because the project had that org name firstThe GitHub issue-search endpoint did not return an open-issue count for this repo (1944 stars), so the issue backlog is not visible from this probe - do not read the missing number as an absence of issues. Last push 2026-09-30 and v0.2.8 released 2026-09-24 indicate active maintenance
Best for
Any production server that must return schema-valid JSON and currently retries on parse failure - this is the cheapest fix, because it removes retries rather than making them fasterReading how a token-level grammar engine keeps overhead near zero, and why the C++ core with a thin Python binding is the right shape for per-token codeConstrained generation on non-CUDA hosts, since it is pure logic over vocabularies rather than device code
Avoid for
Free-form generation with no format requirement - there is nothing to enforceCases where you need semantic guarantees rather than syntactic ones. Valid JSON is not correct JSON for your schema's intent, and no grammar library fixes thatAnything expecting a standalone server; there is none, and there should not be
notes
One of three structured-generation records in this slice (with outlines and lm-format-enforcer) because all three are still maintained and all three are imported by production serving stacks rather than deployed standalone. Ranking: XGrammar is the production default inside vLLM and SGLang. All metadata from the GitHub API on 2026-10-03; arXiv title and date verified via the arXiv API. ||| MAINTENANCE (dated fact, GitHub API, 2026-10-04): last commit 2026-09-30 (0.1 months before this record), latest release v0.2.8 published 2026-09-24, 74 contributors (distinct committers including anonymous), open_issues_count 103 (issues PLUS pull requests - see [[ev-github-open-issues-count-includes-pull-requests]]), not archived, 1944 stars. See [[ev-maintenance-status-is-four-dated-facts]] for how to read this fact set.
Notes
One of three structured-generation records in this slice (with outlines and lm-format-enforcer) because all three are still maintained and all three are imported by production serving stacks rather than deployed standalone. Ranking: XGrammar is the production default inside vLLM and SGLang. All metadata from the GitHub API on 2026-10-03; arXiv title and date verified via the arXiv API. ||| MAINTENANCE (dated fact, GitHub API, 2026-10-04): last commit 2026-09-30 (0.1 months before this record), latest release v0.2.8 published 2026-09-24, 74 contributors (distinct committers including anonymous), open_issues_count 103 (issues PLUS pull requests - see [[ev-github-open-issues-count-includes-pull-requests]]), not archived, 1944 stars. See [[ev-maintenance-status-is-four-dated-facts]] for how to read this fact set.
Cross-references
This record references no other records — it is a leaf. It is cited by 0.