NVIDIA H100 NVL 94GB x4 (HPE DL380a), MLPerf Offline (v4.1, open): Mixtral-8x7B aggregate tok/s
18839 output tok/s AGGREGATE across 4x H100-NVL-94GB (Offline (v4.1, open); ~4710 tok/s per card)
Fields
id
benchmarks/h100nvl-4x-mlperf-v4-1-mixtral-offline-tokensMetric
tps_aggregate
Value
18,839
Unit
output tok/s AGGREGATE across 4x H100-NVL-94GB (Offline (v4.1, open); ~4710 tok/s per card)
Model
Mixtral-8x7B (FP8)
Methodology
VENDOR-MEASURED (MLPerf closed submission, HPE DL380a). 4x H100-NVL-94GB (PCIe, 3-card NVLink bridge pair), TensorRT, Mixtral-8x7B FP8, Offline (v4.1, open). 'Tokens per second' = 18839 verbatim from mlperf_log_summary.txt. AGGREGATE across 4 cards — NOT a single-user decode figure (~4710/card).
Measured by
vendor
Reproducible
yes
notes
Open division. MoE — high tokens/s is token-count not dense-equivalent.
Notes
Open division. MoE — high tokens/s is token-count not dense-equivalent.