NVIDIA H100 PCIe 80GB x2 (Dell R760), MLPerf v4.1 Offline: GPT-J 6B aggregate tok/s
3317.08 output tok/s AGGREGATE across 2x H100-PCIe-80GB (Offline; ~1658 tok/s per card)
Fields
id
benchmarks/h100pcie-2x-mlperf-v4-1-gptj-offline-tokensMetric
tps_aggregate
Value
3,317.08
Unit
output tok/s AGGREGATE across 2x H100-PCIe-80GB (Offline; ~1658 tok/s per card)
Model
GPT-J 6B (FP8)
Methodology
VENDOR-MEASURED (MLPerf v4.1 closed submission, Dell R760). 2x H100-PCIe-80GB, TensorRT 10.2.0 / CUDA 12.4, GPT-J 6B FP8, Offline scenario. 'Tokens per second (inferred)' = 3317.08 verbatim from mlperf_log_summary.txt. AGGREGATE across 2 cards and a deep batch — NOT a single-user decode figure (~1658/card).
Measured by
vendor
Reproducible
yes
notes
H100-PCIe is the 350W 2.0 TB/s HBM2e SKU (vs 700W 3.35 TB/s SXM). GPT-J is a 6B model — small; figure is prefill+decode aggregate.
Notes
H100-PCIe is the 350W 2.0 TB/s HBM2e SKU (vs 700W 3.35 TB/s SXM). GPT-J is a 6B model — small; figure is prefill+decode aggregate.