llama.cpp Metal on Apple M4 (Mac mini, 10 CPU cores, 24 GB): Qwen2.5-Coder 7B Q4_K_M tg128
22.6 tok/s (tg128, per-request, batch 1)
Fields
id
benchmarks/llamacpp-metal-m4-base-qwen25-coder-7b-q4km-tg128-2e97c5fMetric
decode_tok_s
Value
22.6
Unit
tok/s (tg128, per-request, batch 1)
Model
Qwen2.5-Coder 7B Q4_K_M (4.36 GiB, 7.62B)
Methodology
COMMUNITY llama-bench, llama.cpp discussion #4167 comment 16886673. Hardware stated by @SpyderMYK in discussion comment 16886673 (2026-05-12): Apple M4, 10 CPU cores, 24 GB unified memory, macOS 26.4. No GPU-core count is printed, so the record attaches to the base M4 and not an M4 Pro or M4 Max. Backend BLAS,MTL, 4 threads, -ngl 99, -r 3. Build 2e97c5f96 (b9100). Reported 22.60 +/- 0.01. Per-request batch-1, NOT aggregate.
Measured by
third_party
Reproducible
yes
notes
Decode counterpart. 22.60 tok/s on a 7.6B Q4_K_M is consistent with a ~120 GB/s base-M4 bandwidth ceiling.
Notes
Decode counterpart. 22.60 tok/s on a 7.6B Q4_K_M is consistent with a ~120 GB/s base-M4 bandwidth ceiling.