llama.cpp Metal on Apple M2 (base, 8- or 10-core GPU) (8 GPU): Llama-2-7B Q8_0 pp512 tok/s (scoreboard #4167, 8e672ef)
147.27 tok/s (pp512, batch 512, per-request)
Fields
id
benchmarks/llamacpp-metal-m2-8core-llama27b-q8-0-pp512-8e672efMetric
prefill_tok_s
Value
147.27
Unit
tok/s (pp512, batch 512, per-request)
Model
Llama-2-7B gguf-q8-0.gguf
Methodology
COMMUNITY-SCOREBOARD (llama.cpp discussion #4167, ggerganov-maintained). llama-bench -hf ggml-org/Llama-2-7B-GGUF -p 512 -n 128 -ngl 99 --delay 30, Metal backend, full GPU offload. Build commit 8e672ef. pp512 = prefill of one prompt at batch 512; tg128 = batch-1 decode of 128 tokens. mean over the default -r 5. Per-request, NOT aggregate. Chip: Apple M2 (base, 8- or 10-core GPU), 8 GPU cores, 100 GB/s unified-memory bandwidth as printed in the scoreboard.
Measured by
third_party
Reproducible
yes
notes
Maintainer-curated Apple Silicon llama-bench table (discussion #4167), the same Llama-2-7B weights and pp512/tg128 protocol as the CUDA/ROCm scoreboards. This row is the 8-core GPU bin; the other bin of the same die is a separate record. F16 cells the scoreboard leaves blank are not filed.
Notes
Maintainer-curated Apple Silicon llama-bench table (discussion #4167), the same Llama-2-7B weights and pp512/tg128 protocol as the CUDA/ROCm scoreboards. This row is the 8-core GPU bin; the other bin of the same die is a separate record. F16 cells the scoreboard leaves blank are not filed.