MLPerf v6.1 Red Hat 2x Xeon 6972P: llama3.1-8b INT8 Offline aggregate tok/s
1507.21 tok/s AGGREGATE over 2x Xeon 6972P (96c, 12-channel)
Fields
id
benchmarks/mlperf-v6-1-redhat-2x6972p-llama31-8b-int8-offlineMetric
tps_aggregate
Value
1,507.21
Unit
tok/s AGGREGATE over 2x Xeon 6972P (96c, 12-channel)
Model
llama3.1-8b
Methodology
MLPerf Inference v6.1 closed division, CPU-only (summary.csv accelerator_model_name is N/A). Values copied from summary.csv, not from a per-system mlperf_log_summary.txt. AGGREGATE throughput for the whole node, NOT per-request decode and NOT per-socket. Prompt length, output length, and batch size are not in the summary row. weight_data_types INT8. Framework column is PyTorch. Scenario: Offline. System: RedHat_Intel 1-node-2S-GNR_96C. 96-core Granite Rapids-AP, two sockets. 1507.21 tok/s Offline vs 1916.74 on the 128-core 6980P — about 79% of the 128-core box at the same 12-channel width. INT8, PyTorch, notes field N/A. measured_by vendor: these are submitter results inside the MLCommons closed division (Intel, Dell, Supermicro, Quanta, Red Hat, Cisco), not an independent re-run.
Measured by
vendor
Reproducible
yes
notes
96-core Granite Rapids-AP, two sockets. 1507.21 tok/s Offline vs 1916.74 on the 128-core 6980P — about 79% of the 128-core box at the same 12-channel width. INT8, PyTorch, notes field N/A.
Notes
96-core Granite Rapids-AP, two sockets. 1507.21 tok/s Offline vs 1916.74 on the 128-core 6980P — about 79% of the 128-core box at the same 12-channel width. INT8, PyTorch, notes field N/A.