Apple M2 Max (38-core GPU), llama.cpp Metal: Llama-2-7B F16 tg128
24.65 tok/s (tg128, Metal, -ngl 99)
Fields
id
benchmarks/llamacpp-metal-m2-max-38core-llama27b-f16-tg128-8e672efMetric
decode_tok_s
Value
24.65
Unit
tok/s (tg128, Metal, -ngl 99)
Model
Llama-2-7B (GGUF, TheBloke llama-2-7b.F16.gguf)
Methodology
llama.cpp discussion #4167 canonical M-series table, maintainer-run llama-bench on commit 8e672ef (Nov 2023, pre-FA-improvement baseline): Llama-2-7B GGUF, Metal backend, -ngl 99, pp512 = prompt-processing batch, tg128 = single-stream decode. Batch-1 per-request decode. GPU-core variant: 38-core.
Measured by
third_party
Reproducible
yes
notes
Canonical #4167 table row. 8e672ef predates the Metal FA improvements; a current build would be higher. Decode on a 7B is memory-bandwidth-bound.
Notes
Canonical #4167 table row. 8e672ef predates the Metal FA improvements; a current build would be higher. Decode on a 7B is memory-bandwidth-bound.