verifiedmedium · 0.75updated 2026-10-15

Argonne MI300X llama.cpp Mixtral-8x7B b1

33.0 tok/s aggregate

Fields

benchmarks/argonne-mi300x-llamacpp-mixtral-8x7b-fp16-b1-128io-tps

Metric

tps_aggregate

Value

Unit

tok/s aggregate

Model

mistralai/Mixtral-8x7B-v0.1

Methodology

THIRD-PARTY MEASURED (Argonne LLM-Inference-Bench arXiv:2411.00136). llama.cpp, FP16/BF16, 128in/128out. throughput=batch*(in+out)/e2e latency = AGGREGATE over batch. 1x bs1 llama.cpp HIP. CSV 'AMD MI300X GPU' = ALCF 8x MI300X node (xGMI). N = tiles used.

Measured by

third_party

Reproducible

yes

notes

CORRECTED 2026-10-15: value re-read from All_results.csv

Notes

CORRECTED 2026-10-15: value re-read from All_results.csv

Cross-references

Accelerators (1)

AcceleratorsAMD Instinct MI300X

Engine (1)

Inference enginesllama.cpp