TensorRT-LLM: LLM inference on NVIDIA GPUs

Fields

id

sources/src5-tensorrt-llm-github

URL

https://github.com/NVIDIA/TensorRT-LLM

Publisher

NVIDIA

Kind

repo

Published

2023-01-01

Accessed

2026-10-04

notes

TensorRT-LLM provides Python API for defining LLMs with state-of-the-art optimizations for NVIDIA GPUs. Contains Python and C++ runtimes. Supports FP8, FP4 quantization. 14.8k stars. Works with NVIDIA Dynamo for datacenter-scale serving. Includes Triton backend and Triton kernels.

Notes

TensorRT-LLM provides Python API for defining LLMs with state-of-the-art optimizations for NVIDIA GPUs. Contains Python and C++ runtimes. Supports FP8, FP4 quantization. 14.8k stars. Works with NVIDIA Dynamo for datacenter-scale serving. Includes Triton backend and Triton kernels.

Cross-references

This record references no other records.