W7900 llama.cpp ROCm: Qwen3.5-35B-A3B decode at 2.9K prompt (u/neuromacmd)
78.9 tok/s (single-request decode, batch 1, ~2.9K prompt)
Fields
id
benchmarks/nm-1s4bggo-w7900-35b-rocm-q4km-2k9-tgMetric
decode_tok_s
Value
78.9
Unit
tok/s (single-request decode, batch 1, ~2.9K prompt)
Model
Qwen3.5-35B-A3B (unsloth UD-Q4_K_M, 21 GB)
Methodology
r/LocalLLaMA post 1s4bggo, EDIT 3. llama.cpp, the same unsloth GGUF files copied onto every machine, batch size 1, one request. Prompt processing column is labelled 'PP tok/s (2.9K)' so the prefill figure is for a ~2.9K-token prompt; generation is the decode rate of that single request, not an aggregate across concurrent requests. Fedora 43 host for the AMD cards (Ryzen 9 9900X, 192 GB DDR5-5600). Build for the AMD rows of this edit is the post's original llama.cpp (b8500 is named only for the Mac Metal run). Backend: ROCm.
Measured by
third_party
Reproducible
yes
notes
EDIT 3 of the post, ROCm backend, single GPU. The post's original table and this corrected same-GGUF table agree on these AMD cells. Per-request, batch 1.
Notes
EDIT 3 of the post, ROCm backend, single GPU. The post's original table and this corrected same-GGUF table agree on these AMD cells. Per-request, batch 1.