NVIDIA H100 PCIe 80GB x2 (Cisco C245M8), MLPerf v4.1 Offline: GPT-J 6B aggregate tok/s
3268.94 output tok/s AGGREGATE across 2x H100-PCIe-80GB (Offline; ~1634 tok/s per card)
Fields
id
benchmarks/h100pcie-2x-mlperf-v4-1-gptj-offline-tokens-ciscoMetric
tps_aggregate
Value
3,268.94
Unit
output tok/s AGGREGATE across 2x H100-PCIe-80GB (Offline; ~1634 tok/s per card)
Model
GPT-J 6B (FP8)
Methodology
VENDOR-MEASURED (MLPerf v4.1 closed submission, Cisco C245M8). 2x H100-PCIe-80GB, TensorRT 10.2.0 / CUDA 12.4, GPT-J 6B FP8, Offline scenario. 'Tokens per second (inferred)' = 3268.94 verbatim from mlperf_log_summary.txt. AGGREGATE across 2 cards and a deep batch — NOT a single-user decode figure (~1634/card).
Measured by
vendor
Reproducible
yes
notes
H100-PCIe is the 350W 2.0 TB/s HBM2e SKU (vs 700W 3.35 TB/s SXM). GPT-J is a 6B model — small; figure is prefill+decode aggregate.
Notes
H100-PCIe is the 350W 2.0 TB/s HBM2e SKU (vs 700W 3.35 TB/s SXM). GPT-J is a 6B model — small; figure is prefill+decode aggregate.