llama.cpp Metal on Apple M3 Max (MBP 16, 40-core GPU): llama 7B Q4_0 pp512 (discussion #4167, c1d0e7a)
889.62 tok/s (pp512, per-request, batch 1)
Fields
id
benchmarks/llamacpp-metal-m3-max-40core-q4-0-pp512-c1d0e7aMetric
prefill_tok_s
Value
889.62
Unit
tok/s (pp512, per-request, batch 1)
Model
llama 7B Q4_0 (6.74B)
Methodology
COMMUNITY llama-bench posted to llama.cpp discussion #4167. llama-bench -p 512 -n 128 -ngl 99, mean of the reporter's repeats, single stream, batch size 1. The number is per-request, NOT an aggregate across concurrent requests. Model is the canonical llama-2-7b GGUF (6.74B) the thread standardises on. Reply by @TeeAaTeeUu (comment 18201257, 2026-08-29) to the original @ymcui M3 Max post (comment 7656623): MacBook Pro 16-inch, M3 Max, 12+4 CPU, 40-core GPU. Backend MTL,BLAS, 12 threads. Build c1d0e7a00. The reporter calls it 'per updated instructions', so this is the post-2026-08-25 harness, not the 2023 8e672ef numbers already filed for this chip. Normal power mode.
Measured by
third_party
Reproducible
yes
notes
Same machine and harness as the other rows in this comment. Recorded mean only; the reporter's +/- is in the methodology. format_id is null for F16 because data/quantization has no F16 GGUF record.
Notes
Same machine and harness as the other rows in this comment. Recorded mean only; the reporter's +/- is in the methodology. format_id is null for F16 because data/quantization has no F16 GGUF record.