Text Generation Inference

Status deprecated and confidence deliberately moderate: the repo is ARCHIVED and read-only (archived flag true, last push 2026-03-21, final release v3.3.7 on 2025-12-19), so most capability claims are historical rather than current. CORRECTION TO A COMMON MISCONCEPTION, verified against the README on 2026-10-04: TGI was NOT merged into vLLM. It was retired. The README's own words are 'text-generation-inference is now in maintenance mode. Going forward, we will accept pull requests for minor bug fixes, documentation improvements and lightweight maintenance tasks', followed by an explanation that 'TGI has initiated the movement for optimized inference engines to rely on a transformers model architectures. This approach is now adopted by downstream inference engines, which we contribute to and recommend using going forward: vLLM, SGLang, as well as local engines with inter-compatibility such as llama.cpp or MLX.' The architectural idea - optimised transformers model architectures shared with downstream engines - was donated rather than the codebase. So there is no TGI code inside vLLM; there is a design pattern and a recommended migration path. A reader who believes TGI was absorbed should correct that here. Recorded because TGI was a significant GPU serving engine and its retirement is itself a fact worth having in the knowledge base; recommended migration targets are vLLM or SGLang. Confirmed still in use at Hugging Face to power Hugging Chat and the Inference API/Endpoints, which is the useful nuance behind the deprecation: retired upstream by maintainers, still deployed in production at Hugging Face. ||| MAINTENANCE (dated fact, GitHub API, 2026-10-04): last commit 2026-03-21 (6.5 months ago), latest release v3.3.7 published 2025-12-19, 148 contributors, open_issues_count 324 (issues PLUS pull requests), ARCHIVED flag TRUE, 10882 stars. This confirms and sharpens the existing record rather than contradicting it: the repo is read-only, the last release is ten months before this record's date, and the last commit is six and a half months before it. DEPLOYMENT: this engine remains the one clear NAMED DEPLOYMENT with no numbers - confirmed still in production at Hugging Face powering Hugging Chat and the Inference API/Endpoints, which is graded YES-without-numbers, and HF publishes no throughput, request volume or fleet size for it. That combination is the whole point of the record and the grading scheme in [[ev-production-evidence-is-scarce-and-nearly-always-single-site]] keeps it distinct from 'no evidence': archived upstream, deployed downstream, no figures either way. CHUNKED PREFILL and STRUCTURED OUTPUT: unstated for this engine and not verified. tracing_support set to basic on 2026-10-04, carried from the same-repo record obs-tgi-observability: OTel is present but emits only generic gRPC SERVER spans named after the RPC method with rpc.* attributes, opt-in via --otlp-endpoint. No gen_ai attributes and no prefill/decode split. The TTFT and KV-occupancy metrics it lacks were established by zero-hit searches documented there.