Apple M3 Max (40-core GPU), llama.cpp Metal: Llama-2-7B Q8_0 tg128
42.75 tok/s (tg128, Metal, -ngl 99)
Fields
id
benchmarks/llamacpp-metal-m3-max-40core-llama27b-q80-tg128-8e672efMetric
decode_tok_s
Value
42.75
Unit
tok/s (tg128, Metal, -ngl 99)
Model
Llama-2-7B (GGUF, TheBloke llama-2-7b.Q8_0.gguf)
Methodology
llama.cpp discussion #4167 canonical M-series table, maintainer-run llama-bench on commit 8e672ef (Nov 2023, pre-FA-improvement baseline): Llama-2-7B GGUF, Metal backend, -ngl 99, pp512 = prompt-processing batch, tg128 = single-stream decode. Batch-1 per-request decode. GPU-core variant: 40-core.
Measured by
third_party
Reproducible
yes
notes
Canonical #4167 table row, commit 8e672ef baseline build.
Notes
Canonical #4167 table row, commit 8e672ef baseline build.