llama.cpp Metal on M4 Pro (20 GPU): Llama-2-7B gguf-q4-0 tg128 tok/s (scoreboard #4167, 8e672ef)
50.74 tok/s (tg128, batch 1, per-request)
Fields
id
benchmarks/llamacpp-metal-m4-pro-llama27b-q4-0-tg128-8e672efMetric
decode_tok_s
Value
50.74
Unit
tok/s (tg128, batch 1, per-request)
Model
Llama-2-7B gguf-q4-0.gguf
Methodology
COMMUNITY-SCOREBOARD (llama.cpp discussion #4167, ggerganov-maintained). llama-bench -hf ggml-org/Llama-2-7B-GGUF -p 512 -n 128 -ngl 99 --delay 30, Metal backend, full GPU offload. Build commit 8e672ef. pp512 = prefill at batch 512; tg128 = batch-1 decode. mean +/- std over default -r 5. Per-request, NOT aggregate. Chip: 20 GPU cores, 273 GB/s memory BW.
Measured by
third_party
Reproducible
yes
notes
Maintainer-curated Apple Silicon llama-bench table (discussion #4167). Same Llama-2-7B weights, same pp512/tg128 protocol as the CUDA/ROCm scoreboards, so cross-vendor comparisons against #15013/#15021 rows are apples-to-apples.
Notes
Maintainer-curated Apple Silicon llama-bench table (discussion #4167). Same Llama-2-7B weights, same pp512/tg128 protocol as the CUDA/ROCm scoreboards, so cross-vendor comparisons against #15013/#15021 rows are apples-to-apples.