llama.cpp Metal on Apple M4 (Mac mini, 10 CPU cores, 24 GB): llama 3.2 3B Q4_K_M tg128
45.95 tok/s (tg128, per-request, batch 1)
Fields
id
benchmarks/llamacpp-metal-m4-base-llama32-3b-q4km-tg128-2e97c5fMetric
decode_tok_s
Value
45.95
Unit
tok/s (tg128, per-request, batch 1)
Model
llama 3.2 3B Q4_K_M (1.87 GiB, 3.21B)
Methodology
COMMUNITY llama-bench, llama.cpp discussion #4167 comment 16886673. Hardware stated by @SpyderMYK in discussion comment 16886673 (2026-05-12): Apple M4, 10 CPU cores, 24 GB unified memory, macOS 26.4. No GPU-core count is printed, so the record attaches to the base M4 and not an M4 Pro or M4 Max. Backend BLAS,MTL, 4 threads, -ngl 99, -r 3. Build 2e97c5f96 (b9100). Reported 45.95 +/- 0.02. Per-request batch-1, NOT aggregate.
Measured by
third_party
Reproducible
yes
notes
Decode counterpart of the pp512 row in the same comment.
Notes
Decode counterpart of the pp512 row in the same comment.