AMD Strix Halo (Ryzen AI Max+ 395, Radeon 8060S) 128GB, llama.cpp Vulkan: Llama-3.3-70B Q4_K_M decode tok/s
5.0 decode tok/s (single-stream, ~41 GB weights)
Fields
id
benchmarks/comm-strix-halo-llama33-70b-q4-decodeMetric
decode_tok_s
Value
5
Unit
decode tok/s (single-stream, ~41 GB weights)
Model
Llama-3.3-70B (dense) Q4_K_M (GGUF)
Methodology
THIRD-PARTY MEASURED (lhl/strix-halo-testing llm-bench, Vulkan AMDVLK, Framework Desktop-class hardware). Llama-3.3-70B dense Q4_K_M: ~5.0 tok/s decode, pp512 26.4. The ~256 GB/s unified-memory wall is the wall - Strix Halo is FANTASTIC for sparse MoE (gpt-oss-120b MXFP4 33.7 t/s, GLM-4.5-Air Q4_K_XL 23.4, Qwen3-235B-A22B Q3_K_XL 15.1, Llama-4-Scout 19.4) and MEDIOCRE for dense 70B. Mirror image of the Mac: less bandwidth, more capacity per dollar, x86/Linux.
Measured by
third_party
Reproducible
yes
notes
The best value in this slice for MoE workloads and the only sub-$2,500 ticket to 128GB. Framework Desktop 128GB ~$1,999-2,300, GMKtec EVO-X2 128GB $2,199 (went UP from ~$1,999 on DRAM cost). Measured power (Geerling sbc-reviews#80): 12.5 W idle at wall, ~136-155 W load, essentially silent. DO NOT buy for dense 70B - ~5 tok/s is 2-3x slower than a used M1 Ultra at similar-or-lower price. Buy it for gpt-oss-120b / GLM-4.5-Air / Qwen3-30B/235B-A3B-class MoE.
Notes
The best value in this slice for MoE workloads and the only sub-$2,500 ticket to 128GB. Framework Desktop 128GB ~$1,999-2,300, GMKtec EVO-X2 128GB $2,199 (went UP from ~$1,999 on DRAM cost). Measured power (Geerling sbc-reviews#80): 12.5 W idle at wall, ~136-155 W load, essentially silent. DO NOT buy for dense 70B - ~5 tok/s is 2-3x slower than a used M1 Ultra at similar-or-lower price. Buy it for gpt-oss-120b / GLM-4.5-Air / Qwen3-30B/235B-A3B-class MoE.