MLPerf v6.1 Cisco 4x Xeon 6788P: llama3.1-8b INT8 Server aggregate tok/s
1030.45 tok/s AGGREGATE over 4x Xeon 6788P (Server scenario)
Fields
id
benchmarks/mlperf-v6-1-cisco-4x6788p-llama31-8b-int8-serverMetric
tps_aggregate
Value
1,030.45
Unit
tok/s AGGREGATE over 4x Xeon 6788P (Server scenario)
Model
llama3.1-8b
Methodology
MLPerf Inference v6.1 closed division, CPU-only (summary.csv accelerator_model_name is N/A). Values copied from summary.csv, not from a per-system mlperf_log_summary.txt. AGGREGATE throughput for the whole node, NOT per-request decode and NOT per-socket. Prompt length, output length, and batch size are not in the summary row. weight_data_types INT8. Framework column is PyTorch. Scenario: Server. System: Cisco X410M8, 1-node-4S-GNR_86C. Server scenario, four sockets. 1030.45 tok/s aggregate. About 2.0x the 2S Quanta Server number, unlike Offline which did not double. measured_by vendor: these are submitter results inside the MLCommons closed division (Intel, Dell, Supermicro, Quanta, Red Hat, Cisco), not an independent re-run.
Measured by
vendor
Reproducible
yes
notes
Server scenario, four sockets. 1030.45 tok/s aggregate. About 2.0x the 2S Quanta Server number, unlike Offline which did not double.
Notes
Server scenario, four sockets. 1030.45 tok/s aggregate. About 2.0x the 2S Quanta Server number, unlike Offline which did not double.