NVIDIA H100 NVL 94GB x2 (Cisco C245M8), MLPerf Offline (v5.0): Llama-2-70B aggregate tok/s
3741.32 output tok/s AGGREGATE across 2x H100-NVL-94GB (Offline (v5.0); ~1871 tok/s per card)
Fields
id
benchmarks/h100nvl-2x-mlperf-v5-0-llama2-70b-offline-tokensMetric
tps_aggregate
Value
3,741.32
Unit
output tok/s AGGREGATE across 2x H100-NVL-94GB (Offline (v5.0); ~1871 tok/s per card)
Model
Llama-2-70B (FP8)
Methodology
VENDOR-MEASURED (MLPerf closed submission, Cisco C245M8). 2x H100-NVL-94GB (PCIe, 3-card NVLink bridge pair), TensorRT, Llama-2-70B FP8, Offline (v5.0). 'Tokens per second' = 3741.32 verbatim from mlperf_log_summary.txt. AGGREGATE across 2 cards — NOT a single-user decode figure (~1871/card).
Measured by
vendor
Reproducible
yes