MLPerf v5.0 Quanta_Cloud_Technology 4x NVIDIA H100-PCIe-80GB: llama2-70b-99.9 Offline aggregate tok/s
7013.17 output tokens/s (aggregate over 4 accelerators, Offline)
Fields
id
benchmarks/mlperf-v5-0-qct-4xh100pcie-llama2-70b-llama2-70b-99-9-offlineMetric
tps_aggregate
Value
7,013.17
Unit
output tokens/s (aggregate over 4 accelerators, Offline)
Model
llama2-70b-99.9
Methodology
VENDOR-SUBMITTED MLPerf Inference v5.0 closed division, Datacenter suite, Offline scenario. Quanta_Cloud_Technology 4xH100-PCIe-80GB: 4x NVIDIA H100-PCIe-80GB. Framework TensorRT-LLM. 'Tokens/s' = AGGREGATE output-token throughput across the 4 accelerators (NOT per-request). Value verbatim from summary_results.json. min_query_count/min_duration per MLPerf rules; binding latency constraint applies to Server/Interactive.
Measured by
vendor
Reproducible
yes