NVIDIA TensorRT-LLM performance overview — trtllm-bench throughput tables (H100/H200/GH200/B200/GB200, Llama FP8/FP4)
benchmark
Fields
id
sources/nvidia-tensorrt-llm-perf-overviewPublisher
NVIDIA
Kind
benchmark
Accessed
2026-10-15
notes
Vendor-measured trtllm-bench 'throughput' (PyTorch backend, max-load offline). Metric = Total Output Throughput tok/s, aggregate. TP size, ISL/OSL, ModelOpt FP8/FP4 quants per column. GH200 96GB HBM3 + 480GB LPDDR5X.
Notes
Vendor-measured trtllm-bench 'throughput' (PyTorch backend, max-load offline). Metric = Total Output Throughput tok/s, aggregate. TP size, ISL/OSL, ModelOpt FP8/FP4 quants per column. GH200 96GB HBM3 + 480GB LPDDR5X.
Cross-references
This record references no other records.