8x Intel Gaudi 2 HL-225B, MLPerf v4.0 offline: Llama-2-70B FP8 aggregate output tok/s
8034.85 output tok/s AGGREGATE across 8x Gaudi 2 (batch-1024 class; ~1004 tok/s per module)
Fields
id
benchmarks/comm-gaudi2-8x-llama2-70b-fp8-mlperfMetric
tps_aggregate
Value
8,034.85
Unit
output tok/s AGGREGATE across 8x Gaudi 2 (batch-1024 class; ~1004 tok/s per module)
Model
Llama-2-70B FP8
Methodology
VENDOR-MEASURED (MLPerf v4.0 offline submission, Intel). 8x Gaudi 2 HL-225B: 8,034.85 output tok/s AGGREGATE on Llama-2-70B FP8. Per-module ~1,004 tok/s at batch-1024 - THIS IS NOT A SINGLE-USER DECODE NUMBER. Respectable per-module throughput, but it is batch-1024 serving measured on a chassis that a hobbyist cannot practically buy or power.
Measured by
vendor
Reproducible
yes
notes
TRAP for solo use. The MLPerf number is real but the HARDWARE PATH is the trap: HL-225 is OAM-only, has no PCIe SKU, and needs the Gaudi UBB chassis (Supermicro 820GH-TNR2 ~$11,995 for the 8-module baseboard) plus a 24x 100GbE fabric to even light up - you cannot put one in a workstation. Bare module ~$1,500-2,700; the rumoured $700-900 market is NOT showing on eBay Oct-2026 - treat that figure as stale. SynapseAI/vLLM-fork works but the ecosystem is thin and Intel is winding the line down.
Notes
TRAP for solo use. The MLPerf number is real but the HARDWARE PATH is the trap: HL-225 is OAM-only, has no PCIe SKU, and needs the Gaudi UBB chassis (Supermicro 820GH-TNR2 ~$11,995 for the 8-module baseboard) plus a 24x 100GbE fabric to even light up - you cannot put one in a workstation. Bare module ~$1,500-2,700; the rumoured $700-900 market is NOT showing on eBay Oct-2026 - treat that figure as stale. SynapseAI/vLLM-fork works but the ecosystem is thin and Intel is winding the line down.