inference/research
verifiedhigh · 0.95updated 2026-10-04

Dell XE9785L 8xMI355X, MLPerf Inference v6.1 Offline: Llama 3.1 8B, third-party OEM reproduction of AMD silicon

158458 output tokens/s (aggregate, harness 'Tokens per second')

Fields

id
benchmarks/bench-mlperf-v6-1-dell-mi355x-llama3-1-8b-offline
Metric
tps_aggregate
Value
158,458
Unit
output tokens/s (aggregate, harness 'Tokens per second')
Model
Llama 3.1 8B (MLPerf v6.1 submission, FP8 weights)
Methodology
AGGREGATE. MLPerf Inference v6.1 Closed division, Datacenter suite, Offline scenario. LOADS: batch size not a fixed published number (closed-loop Offline queue, submitter-chosen scheduler batching); llama3.1-8b/user.conf enforces min_duration 600000 ms and Server min_duration 120000 ms / target_qps 0.5; min_query_count is NOT set for llama3.1-8b (only llama2-70b has it, at 2000). Early stopping satisfied: Yes. Model shape: the accuracy block reports GEN_LEN 8127480 for this run; the reference SUT sets max_tokens 128, so llama3.1-8b's tokens-per-sample is a much smaller constant than llama2-70b's ~267 and this figure is therefore NOT comparable with any Llama 2 70B tok/s number. Hardware: 1 node, 8x AMD Instinct MI355X 288GB HBM3e, Dell PowerEdge XE9785L. Framework per summary.csv: 'vLLM 0.22.0, PyTorch 2.10.0, ROCm 7.2.2'. Samples per second 1237.97. POWER: has_power=False, design_power_watts=null, no declared cap, no cooling configuration. OMITTED BY SOURCE: TP size, KV dtype, concurrency, clock state.
Measured by
vendor
Reproducible
yes
notes
Dell's MI355X node matches AMD's own for Llama 2 70B within the OEM band, and this record exists mainly as an independent OEM data point rather than for its own value. Its real value is as the fastest Llama-family result on any AMD node in this round: 158458 tok/s. HPE's MI355X node gives 157694 and Supermicro's 159060 for the same scenario - a 0.9% band across three OEMs on three different chassis, which is unusually tight and is good evidence that on this hardware the implementation, not the box, is the variable worth chasing. INTERCONNECT IDS FILLED 2026-10-04: interconnect_ids was empty; it now carries the interconnect list of the accelerator record(s) this run names (amd-instinct-mi355x). BASIS AND ITS LIMITS, stated because this field is easy to over-read: it records the FABRIC PRESENT IN THE HARDWARE CONFIGURATION, which is how every previously-filled record in this repo reads it, not traffic measured over that fabric. The clearest evidence for that convention is that single-GPU records already carry a scale-up fabric here - trtllm-llama31-8b-fp8-h100-1k1k-tps is 1x H100 at TP=1 and carries ['nvlink-4','nvswitch-4'] - and that host-link-only records carry a PCIe value (arc-pro-b70-mlperf-v6-1-llama2-70b-offline-tokens carries ['pcie-gen5'] on 4x Arc Pro B70, which has no GPU-to-GPU fabric at all). So a value here does NOT mean the interconnect was exercised by this particular workload, and must not be cited as evidence that it was. Whether the run actually crossed that fabric is a question about TP/EP/PP size, which this repo records in methodology, not here. NOT INVENTED: nothing was filled where the accelerator record itself carries no interconnect, so this field stays empty for any part whose own record has none - which is the honest state for a single-socket Apple sOC and for parts whose vendor publishes no host interface.

Notes

Dell's MI355X node matches AMD's own for Llama 2 70B within the OEM band, and this record exists mainly as an independent OEM data point rather than for its own value. Its real value is as the fastest Llama-family result on any AMD node in this round: 158458 tok/s. HPE's MI355X node gives 157694 and Supermicro's 159060 for the same scenario - a 0.9% band across three OEMs on three different chassis, which is unusually tight and is good evidence that on this hardware the implementation, not the box, is the variable worth chasing. INTERCONNECT IDS FILLED 2026-10-04: interconnect_ids was empty; it now carries the interconnect list of the accelerator record(s) this run names (amd-instinct-mi355x). BASIS AND ITS LIMITS, stated because this field is easy to over-read: it records the FABRIC PRESENT IN THE HARDWARE CONFIGURATION, which is how every previously-filled record in this repo reads it, not traffic measured over that fabric. The clearest evidence for that convention is that single-GPU records already carry a scale-up fabric here - trtllm-llama31-8b-fp8-h100-1k1k-tps is 1x H100 at TP=1 and carries ['nvlink-4','nvswitch-4'] - and that host-link-only records carry a PCIe value (arc-pro-b70-mlperf-v6-1-llama2-70b-offline-tokens carries ['pcie-gen5'] on 4x Arc Pro B70, which has no GPU-to-GPU fabric at all). So a value here does NOT mean the interconnect was exercised by this particular workload, and must not be cited as evidence that it was. Whether the run actually crossed that fabric is a question about TP/EP/PP size, which this repo records in methodology, not here. NOT INVENTED: nothing was filled where the accelerator record itself carries no interconnect, so this field stays empty for any part whose own record has none - which is the honest state for a single-socket Apple sOC and for parts whose vendor publishes no host interface.

Cross-references

Accelerators (1)

Engine (1)

Format (1)

Interconnect (3)