Apple M2 Max (30-core GPU), llama.cpp Metal: Llama-2-7B Q4_0 pp512
537.6 tok/s (pp512, Metal, -ngl 99)
Fields
id
benchmarks/llamacpp-metal-m2-max-30core-llama27b-q40-pp512-8e672efMetric
prefill_tok_s
Value
537.6
Unit
tok/s (pp512, Metal, -ngl 99)
Model
Llama-2-7B (GGUF, TheBloke llama-2-7b.Q4_0.gguf)
Methodology
llama.cpp discussion #4167 canonical M-series table, maintainer-run llama-bench on commit 8e672ef (Nov 2023, pre-FA-improvement baseline): Llama-2-7B GGUF, Metal backend, -ngl 99, pp512 = prompt-processing batch, tg128 = single-stream decode. Batch-1 per-request decode. GPU-core variant: 30-core.
Measured by
third_party
Reproducible
yes
notes
Canonical #4167 table row. 8e672ef predates the Metal FA improvements; a current build would be higher. Decode on a 7B is memory-bandwidth-bound.
Notes
Canonical #4167 table row. 8e672ef predates the Metal FA improvements; a current build would be higher. Decode on a 7B is memory-bandwidth-bound.