M3 Max Llama-2-7B Q4_0 pp512 (llama.cpp #4167)
759.7 tok/s (pp512, batch 512)
Fields
id
benchmarks/llamacpp-metal-m3-max-llama27b-q4-0-pp512-8e672efMetric
prefill_tok_s
Value
759.7
Unit
tok/s (pp512, batch 512)
Model
Llama-2-7B
Methodology
llama.cpp discussion #4167 (ggerganov table, commit 8e672ef / ymcui 55978ce). M3 Max 40-core GPU, Metal backend, ngl=99, llama-bench. Llama-2-7B. tg128 = batch-1 decode tok/s; pp512 = batch-512 prefill tok/s.
Measured by
third_party
Reproducible
yes
Thermal state
unknown
notes
Apple Silicon M3 Max 40 GPU cores, 400 GB/s unified memory.
Notes
Apple Silicon M3 Max 40 GPU cores, 400 GB/s unified memory.