llama.cpp Metal on Apple M5 Pro (MBP 14, 16-core GPU, 24 GB): llama 7B F16 tg128 (discussion #4167, c3a1128)
21.11 tok/s (tg128, per-request, batch 1)
Fields
id
benchmarks/llamacpp-metal-m5-pro-16core-24gb-f16-tg128-c3a1128Metric
decode_tok_s
Value
21.11
Unit
tok/s (tg128, per-request, batch 1)
Model
llama 7B F16 (6.74B)
Methodology
COMMUNITY llama-bench posted to llama.cpp discussion #4167. llama-bench -p 512 -n 128 -ngl 99, mean of the reporter's repeats, single stream, batch size 1. The number is per-request, NOT an aggregate across concurrent requests. Model is the canonical llama-2-7b GGUF (6.74B) the thread standardises on. Posted by @bnggge, comment 16556242 (2026-04-14): M5 Pro, MacBook Pro 14-inch, 10+5 CPU, 16-core GPU, 24 GB. Backend MTL,BLAS, 5 threads, ngl 99. The pinned scoreboard rows for this chip are builds 60081bb and c1d0e7a; none of these three builds is either of those. Build c3a1128ce (b8509), the tensor-API build.
Measured by
third_party
Reproducible
yes
notes
Same machine and harness as the other rows in this comment. Recorded mean only; the reporter's +/- is in the methodology. format_id is null for F16 because data/quantization has no F16 GGUF record.
Notes
Same machine and harness as the other rows in this comment. Recorded mean only; the reporter's +/- is in the methodology. format_id is null for F16 because data/quantization has no F16 GGUF record.