llama.cpp Metal on Apple M5 Max (MBP 16, 40-core GPU, 64 GB): llama 7B F16 tg128 (discussion #4167, 8e672ef)
37.58 tok/s (tg128, per-request, batch 1)
Fields
id
benchmarks/llamacpp-metal-m5-max-40core-64gb-f16-tg128-8e672efMetric
decode_tok_s
Value
37.58
Unit
tok/s (tg128, per-request, batch 1)
Model
llama 7B F16 (6.74B)
Methodology
COMMUNITY llama-bench posted to llama.cpp discussion #4167. llama-bench -p 512 -n 128 -ngl 99, mean of the reporter's repeats, single stream, batch size 1. The number is per-request, NOT an aggregate across concurrent requests. Model is the canonical llama-2-7b GGUF (6.74B) the thread standardises on. Posted by @Hassan-A-K, comment 16353087 (2026-03-28): M5 Max, 6+12 CPU, 40 GPU, 64 GB. Posted before the 2026-08-25 harness change, on build 8e672ef (b1550), the same build the M1-M4 scoreboard uses. The pinned c1d0e7a M5 Max row already on main (pp512 3158) is a different binary and is not this number. Backend Metal, ngl 99.
Measured by
third_party
Reproducible
yes
notes
Same machine and harness as the other rows in this comment. Recorded mean only; the reporter's +/- is in the methodology. format_id is null for F16 because data/quantization has no F16 GGUF record.
Notes
Same machine and harness as the other rows in this comment. Recorded mean only; the reporter's +/- is in the methodology. format_id is null for F16 because data/quantization has no F16 GGUF record.