Apple M3 Max (30-core GPU), llama.cpp Metal: Llama-2-7B Q8_0 tg128
34.3 tok/s (tg128, Metal, -ngl 99)
Fields
id
benchmarks/llamacpp-metal-m3-max-30core-llama27b-q80-tg128-8e672efMetric
decode_tok_s
Value
34.3
Unit
tok/s (tg128, Metal, -ngl 99)
Model
Llama-2-7B (GGUF, TheBloke llama-2-7b.Q8_0.gguf)
Methodology
llama.cpp discussion #4167 canonical M-series table, maintainer-run llama-bench on commit 8e672ef (Nov 2023, pre-FA-improvement baseline): Llama-2-7B GGUF, Metal backend, -ngl 99, pp512 = prompt-processing batch, tg128 = single-stream decode. Batch-1 per-request decode. GPU-core variant: 30-core.
Measured by
third_party
Reproducible
yes
notes
Canonical #4167 table row, commit 8e672ef baseline build.
Notes
Canonical #4167 table row, commit 8e672ef baseline build.