NVIDIA H100 NVL 94GB x2 (Cisco C245M8), MLPerf Server (v5.0): Llama-2-70B aggregate tok/s
3337.55 output tok/s AGGREGATE across 2x H100-NVL-94GB (Server (v5.0); ~1669 tok/s per card)
Fields
id
benchmarks/h100nvl-2x-mlperf-v5-0-llama2-70b-server-tokensMetric
tps_aggregate
Value
3,337.55
Unit
output tok/s AGGREGATE across 2x H100-NVL-94GB (Server (v5.0); ~1669 tok/s per card)
Model
Llama-2-70B (FP8)
Methodology
VENDOR-MEASURED (MLPerf closed submission, Cisco C245M8). 2x H100-NVL-94GB (PCIe, 3-card NVLink bridge pair), TensorRT, Llama-2-70B FP8, Server (v5.0). 'Tokens per second' = 3337.55 verbatim from mlperf_log_summary.txt. AGGREGATE across 2 cards — NOT a single-user decode figure (~1669/card).
Measured by
vendor
Reproducible
yes