M4 Max 128GB llama.cpp b8353 Qwen3.5-35B-A3B Q4_K_M tg128 depth 8192
82.31 tok/s
Fields
id
benchmarks/llamacpp-m4max128-qwen35-35b-a3b-q4km-d8192-tg128Metric
decode_tok_s
Value
82.31
Unit
tok/s
Model
Qwen3.5-35B-A3B Q4_K_M (unsloth)
Methodology
llama-bench on one M4 Max MacBook Pro, 128 GB unified memory (u/pmttyji). llama.cpp b8353 (commit 28955214d, 2026-03-04), Metal, macOS. Command: llama-bench -m <gguf> -fa 1 -d 0,8192 -p 512 -n 128 -ngl 99 -t 12. ONE request: pp512 is prefill of a 512-token prompt and tg128 is decode of 128 generated tokens, both batch 1. NOT aggregate. -d 0 and -d 8192 are two separate depth points (KV already filled to that depth before the timed prompt). Flash attention on. This row is the depth 8192 cell.
Measured by
third_party
Reproducible
yes
notes
Per-request. Same machine and build as the other seven cells in this post; only model and KV depth change. Q4_K_M is the filename quant; mapped to gguf-q4-k-m.
Notes
Per-request. Same machine and build as the other seven cells in this post; only model and KV depth change. Q4_K_M is the filename quant; mapped to gguf-q4-k-m.