inference/research
verifiedlow · 0.60updated 2026-10-03

AMD Ryzen AI Max+ Radeon 8060S iGPU (RDNA 3.5) under Ollama: Qwen3.5 35B-A3B MoE generation throughput, 100% GPU offload

42.04 tok/s generation throughput (per-request, 100% GPU offload)

Fields

id
benchmarks/ryzen-ai-max-8060s-qwen35-35b-a3b-decode-tok-s
Metric
decode_tok_s
Value
42.04
Unit
tok/s generation throughput (per-request, 100% GPU offload)
Model
Qwen3.5 35B-A3B (MoE, 3B active)
Methodology
PER-REQUEST generation throughput from 'ollama run qwen3.5:35b --verbose'. Prompt processing 154.52 tok/s in the same run. 100% of model layers offloaded to the integrated GPU (AMD states ollama reports '100% GPU'). 20.5 GB model footprint. Hardware: AMD Ryzen AI Max+ with Radeon 8060S integrated graphics (RDNA 3.5) and up to 128GB unified memory shared between CPU and GPU. APU-class: CPU and GPU share one memory pool and one thermal envelope, so this is not a discrete-GPU number. OMITTED BY SOURCE: the specific Ryzen AI Max+ SKU (3950X / 3970X / 'Max' branding are not distinguished), the GPU clock, the number of GPU compute units or WDDM, ROCm version, Ollama version, prompt length, output length, thread count, batch size, and the quantization used (implied by Ollama's default Q4_K_M for this model but never stated). Reproducible is false: too many unstated parameters, and the SKU is ambiguous.
Measured by
vendor
Reproducible
no
notes
The most decision-relevant non-CUDA consumer datapoint here: ~42 tok/s for a 35B MoE running entirely on an APU's integrated RDNA 3.5 iGPU, using the CPU's unified memory as VRAM. Sibling figures in the same source: Qwen3.5 9B dense at 29.84 tok/s (6.2 GB), and Qwen3.5 122B-A10B MoE at 8.59 tok/s (76 GB, split 61% GPU / 39% CPU - i.e. partially offloaded to CPU because it does not fit cleanly). Note the non-monotonicity worth flagging: the 35B MoE (3B active) is FASTER than the 9B dense (29.84 tok/s) despite being 3.5x larger, because only 3B parameters are active per token and MoE decode is bandwidth-bound on the active experts, not the full weight set. That is the same effect that makes the Apple 30B-A3B row fast, seen on AMD silicon. ACCELERATOR ID CAVEAT: no record exists in data/accelerators/ for the Radeon 8060S / Ryzen AI Max APU; this is a planned id and the validator will warn. The nearest existing AMD RDNA-family records are amd-radeon-pro-vii (gfx906, 2019) and the Instinct MI-series, neither of which is this part - the Radeon Pro VII specifically is a 2019 CDNA1 part and does NOT support modern ROCm LLM paths, so no RDNA/Vulkan decode measurement for gfx906 was obtainable from a citable source and none is recorded. Confidence is held at 0.6 because this is a vendor blog with no engine version, no prompt/output length and no SKU - reproducible in spirit, not in practice. INTERCONNECT IDS FILLED 2026-10-04: interconnect_ids was empty; it now carries the interconnect list of the accelerator record(s) this run names (amd-radeon-8060s-ryzen-ai-max). BASIS AND ITS LIMITS, stated because this field is easy to over-read: it records the FABRIC PRESENT IN THE HARDWARE CONFIGURATION, which is how every previously-filled record in this repo reads it, not traffic measured over that fabric. The clearest evidence for that convention is that single-GPU records already carry a scale-up fabric here - trtllm-llama31-8b-fp8-h100-1k1k-tps is 1x H100 at TP=1 and carries ['nvlink-4','nvswitch-4'] - and that host-link-only records carry a PCIe value (arc-pro-b70-mlperf-v6-1-llama2-70b-offline-tokens carries ['pcie-gen5'] on 4x Arc Pro B70, which has no GPU-to-GPU fabric at all). So a value here does NOT mean the interconnect was exercised by this particular workload, and must not be cited as evidence that it was. Whether the run actually crossed that fabric is a question about TP/EP/PP size, which this repo records in methodology, not here. NOT INVENTED: nothing was filled where the accelerator record itself carries no interconnect, so this field stays empty for any part whose own record has none - which is the honest state for a single-socket Apple sOC and for parts whose vendor publishes no host interface.

Notes

The most decision-relevant non-CUDA consumer datapoint here: ~42 tok/s for a 35B MoE running entirely on an APU's integrated RDNA 3.5 iGPU, using the CPU's unified memory as VRAM. Sibling figures in the same source: Qwen3.5 9B dense at 29.84 tok/s (6.2 GB), and Qwen3.5 122B-A10B MoE at 8.59 tok/s (76 GB, split 61% GPU / 39% CPU - i.e. partially offloaded to CPU because it does not fit cleanly). Note the non-monotonicity worth flagging: the 35B MoE (3B active) is FASTER than the 9B dense (29.84 tok/s) despite being 3.5x larger, because only 3B parameters are active per token and MoE decode is bandwidth-bound on the active experts, not the full weight set. That is the same effect that makes the Apple 30B-A3B row fast, seen on AMD silicon. ACCELERATOR ID CAVEAT: no record exists in data/accelerators/ for the Radeon 8060S / Ryzen AI Max APU; this is a planned id and the validator will warn. The nearest existing AMD RDNA-family records are amd-radeon-pro-vii (gfx906, 2019) and the Instinct MI-series, neither of which is this part - the Radeon Pro VII specifically is a 2019 CDNA1 part and does NOT support modern ROCm LLM paths, so no RDNA/Vulkan decode measurement for gfx906 was obtainable from a citable source and none is recorded. Confidence is held at 0.6 because this is a vendor blog with no engine version, no prompt/output length and no SKU - reproducible in spirit, not in practice. INTERCONNECT IDS FILLED 2026-10-04: interconnect_ids was empty; it now carries the interconnect list of the accelerator record(s) this run names (amd-radeon-8060s-ryzen-ai-max). BASIS AND ITS LIMITS, stated because this field is easy to over-read: it records the FABRIC PRESENT IN THE HARDWARE CONFIGURATION, which is how every previously-filled record in this repo reads it, not traffic measured over that fabric. The clearest evidence for that convention is that single-GPU records already carry a scale-up fabric here - trtllm-llama31-8b-fp8-h100-1k1k-tps is 1x H100 at TP=1 and carries ['nvlink-4','nvswitch-4'] - and that host-link-only records carry a PCIe value (arc-pro-b70-mlperf-v6-1-llama2-70b-offline-tokens carries ['pcie-gen5'] on 4x Arc Pro B70, which has no GPU-to-GPU fabric at all). So a value here does NOT mean the interconnect was exercised by this particular workload, and must not be cited as evidence that it was. Whether the run actually crossed that fabric is a question about TP/EP/PP size, which this repo records in methodology, not here. NOT INVENTED: nothing was filled where the accelerator record itself carries no interconnect, so this field stays empty for any part whose own record has none - which is the honest state for a single-socket Apple sOC and for parts whose vendor publishes no host interface.

Cross-references

Accelerators (1)

Engine (1)

Interconnect (1)