llama.cpp Metal on Apple M1 Pro (14- or 16-core GPU) (14 GPU): Llama-2-7B Q4_0 pp512 tok/s (scoreboard #4167, 8e672ef)
232.55 tok/s (pp512, batch 512, per-request)
Fields
id
benchmarks/llamacpp-metal-m1-pro-14core-llama27b-q4-0-pp512-8e672efMetric
prefill_tok_s
Value
232.55
Unit
tok/s (pp512, batch 512, per-request)
Model
Llama-2-7B gguf-q4-0.gguf
Methodology
COMMUNITY-SCOREBOARD (llama.cpp discussion #4167, ggerganov-maintained). llama-bench -hf ggml-org/Llama-2-7B-GGUF -p 512 -n 128 -ngl 99 --delay 30, Metal backend, full GPU offload. Build commit 8e672ef. pp512 = prefill of one prompt at batch 512; tg128 = batch-1 decode of 128 tokens. mean over the default -r 5. Per-request, NOT aggregate. Chip: Apple M1 Pro (14- or 16-core GPU), 14 GPU cores, 200 GB/s unified-memory bandwidth as printed in the scoreboard.
Measured by
third_party
Reproducible
yes
notes
Maintainer-curated Apple Silicon llama-bench table (discussion #4167), the same Llama-2-7B weights and pp512/tg128 protocol as the CUDA/ROCm scoreboards. This row is the 14-core GPU bin; the other bin of the same die is a separate record. F16 cells the scoreboard leaves blank are not filed.
Notes
Maintainer-curated Apple Silicon llama-bench table (discussion #4167), the same Llama-2-7B weights and pp512/tg128 protocol as the CUDA/ROCm scoreboards. This row is the 14-core GPU bin; the other bin of the same die is a separate record. F16 cells the scoreboard leaves blank are not filed.