inference/research
verifiedmedium · 0.80updated 2026-10-14

NVIDIA H100 NVL 94GB x8 (Lenovo SR675 V3), MLPerf Offline (v4.1): Llama-2-70B aggregate tok/s

15875.8 output tok/s AGGREGATE across 8x H100-NVL-94GB (Offline (v4.1); ~1984 tok/s per card)

Fields

id
benchmarks/h100nvl-8x-mlperf-v4-1-llama2-70b-offline-tokens
Metric
tps_aggregate
Value
15,875.8
Unit
output tok/s AGGREGATE across 8x H100-NVL-94GB (Offline (v4.1); ~1984 tok/s per card)
Model
Llama-2-70B (FP8)
Methodology
VENDOR-MEASURED (MLPerf closed submission, Lenovo SR675 V3). 8x H100-NVL-94GB (PCIe, 3-card NVLink bridge pair), TensorRT, Llama-2-70B FP8, Offline (v4.1). 'Tokens per second' = 15875.8 verbatim from mlperf_log_summary.txt. AGGREGATE across 8 cards — NOT a single-user decode figure (~1984/card).
Measured by
vendor
Reproducible
yes

Cross-references

Accelerators (1)

Engine (1)

Format (1)

Interconnect (2)