llama.cpp Qwen 2.5 72B Q5_K_M batch-1 decode on AMD Radeon 8060S (Strix Halo)
4.1 tok/s
Fields
id
benchmarks/llamacpp-strix-halo-8060s-qwen2-5-72b-q5-k-m-decodeMetric
decode_tok_s
Value
4.1
Unit
tok/s
Model
qwen2-5-72b
Methodology
llama-cli-rocm (llama.cpp b10440) on Ubuntu 26.04 LTS (kernel 7.0), native ROCm 7.x targeting gfx1151 on GMKtec EVO-X2 (AMD Ryzen AI Max+ 395, Radeon 8060S 40 CUs, 128GB LPDDR5X with ttm.pages_limit=30146560). Flags: --model qwen2.5-72b-instruct-q5_k_m.gguf --n-gpu-layers 99 --ctx-size 4096 --flash-attn on --temp 0.7 --top-p 0.8 --top-k 20 -p <prompt> -n 128. Measured batch-1 decode throughput at 4.1 tok/s (prompt processing 102.0 tok/s).
Measured by
third_party
Reproducible
yes
notes
Demonstrates memory-bandwidth bound decode on unified memory: 72B dense model requires streaming ~48 GB of weights per token. At 4.1 tok/s, bandwidth consumption is ~197 GB/s, near the 256 GB/s LPDDR5X system bus ceiling.
Notes
Demonstrates memory-bandwidth bound decode on unified memory: 72B dense model requires streaming ~48 GB of weights per token. At 4.1 tok/s, bandwidth consumption is ~197 GB/s, near the 256 GB/s LPDDR5X system bus ceiling.