M3 Max Llama-2-7B Q8_0 pp512 (llama.cpp #4167)
757.64 tok/s (pp512, batch 512)
Fields
id
benchmarks/llamacpp-metal-m3-max-llama27b-q8-0-pp512-8e672efMetric
prefill_tok_s
Value
757.64
Unit
tok/s (pp512, batch 512)
Model
Llama-2-7B
Methodology
llama.cpp discussion #4167 (ggerganov table, commit 8e672ef / ymcui 55978ce). M3 Max 40-core GPU, Metal backend, ngl=99, llama-bench. Llama-2-7B. tg128 = batch-1 decode tok/s; pp512 = batch-512 prefill tok/s.
Measured by
third_party
Reproducible
yes
Thermal state
unknown
notes
Apple Silicon M3 Max 40 GPU cores, 400 GB/s unified memory.
Notes
Apple Silicon M3 Max 40 GPU cores, 400 GB/s unified memory.