LightLLM GitHub repository · inference-research

Fields

id

sources/niche-lightllm-repo

URL

https://github.com/ModelTC/LightLLM

Publisher

ModelTC

Kind

repo

Accessed

2026-10-03

notes

LightLLM README as of v1.2.0. Pure-Python framework with Triton kernels; acknowledges FasterTransformer, TGI, vLLM, SGLang, flashinfer and FlashAttention 1&2 as inputs. News items: v1.0.0 (2025/02) claimed fastest DeepSeek-R1 serving on a single H200 (vendor claim), v1.1.0 (2025/09), prefix-KV transfer between DP ranks (2025/11). Crucially the README lists vLLM, SGLang, Aphrodite, Microsoft ParrotServe, LoongServe, S-LoRA and Ant Group OmniKV as projects that use LightLLM's kernels. API metadata 2026-10-03: created 2023-07-22, last push 2026-09-30, v1.2.0 released 2026-08-10, Apache-2.0, 250 open issues, 4306 stars.

Notes

LightLLM README as of v1.2.0. Pure-Python framework with Triton kernels; acknowledges FasterTransformer, TGI, vLLM, SGLang, flashinfer and FlashAttention 1&2 as inputs. News items: v1.0.0 (2025/02) claimed fastest DeepSeek-R1 serving on a single H200 (vendor claim), v1.1.0 (2025/09), prefix-KV transfer between DP ranks (2025/11). Crucially the README lists vLLM, SGLang, Aphrodite, Microsoft ParrotServe, LoongServe, S-LoRA and Ant Group OmniKV as projects that use LightLLM's kernels. API metadata 2026-10-03: created 2023-07-22, last push 2026-09-30, v1.2.0 released 2026-08-10, Apache-2.0, 250 open issues, 4306 stars.

Cross-references

This record references no other records.