llama.cpp Metal on Apple M4 (Mac mini, 10 CPU cores, 24 GB): Qwen2.5-Coder 7B Q4_K_M pp512
239.73 tok/s (pp512, per-request, batch 1)
Fields
id
benchmarks/llamacpp-metal-m4-base-qwen25-coder-7b-q4km-pp512-2e97c5fMetric
prefill_tok_s
Value
239.73
Unit
tok/s (pp512, per-request, batch 1)
Model
Qwen2.5-Coder 7B Q4_K_M (4.36 GiB, 7.62B)
Methodology
COMMUNITY llama-bench, llama.cpp discussion #4167 comment 16886673. Hardware stated by @SpyderMYK in discussion comment 16886673 (2026-05-12): Apple M4, 10 CPU cores, 24 GB unified memory, macOS 26.4. No GPU-core count is printed, so the record attaches to the base M4 and not an M4 Pro or M4 Max. Backend BLAS,MTL, 4 threads, -ngl 99, -r 3. Build 2e97c5f96 (b9100). GGUF sha256 60e05f2100071479f596b964f89f510f057ce397ea22f2833a0cfe029bfc2463. Reported 239.73 +/- 2.07. Per-request batch-1, NOT aggregate.
Measured by
third_party
Reproducible
yes
notes
Same machine, second model. Prefill drops from 530 to 240 tok/s going from a 3.2B to a 7.6B Q4_K_M.
Notes
Same machine, second model. Prefill drops from 530 to 240 tok/s going from a 3.2B to a 7.6B Q4_K_M.