MLPerf v6.0 MangoBoost 8x AMD Instinct MI325X 256GB HBM3e: llama2-70b-99.9 Server aggregate tok/s
31888.9 output tokens/s (aggregate over 8 accelerators, Server)
Fields
id
benchmarks/mlperf-v6-0-mangoboost-8xmi325x-llama2-70b-server-llama2-70b-99-9-serverMetric
tps_aggregate
Value
31,888.9
Unit
output tokens/s (aggregate over 8 accelerators, Server)
Model
llama2-70b-99.9
Methodology
VENDOR-SUBMITTED MLPerf Inference v6.0 closed division, Datacenter suite, Server scenario. MangoBoost 8xMI325X_2xEPYC_9655: 8x AMD Instinct MI325X 256GB HBM3e. Framework ROCm 6.10.5 (vLLM-class). 'Tokens/s' = AGGREGATE output-token throughput across the 8 accelerators (NOT per-request). Value verbatim from summary_results.json. min_query_count/min_duration per MLPerf rules; binding latency constraint applies to Server/Interactive.
Measured by
vendor
Reproducible
yes