MacBook Pro M4 Max Llama-3.3-70B 8-bit decode (Silicon Score)
6.5 tok/s
Fields
id
benchmarks/comm-apple-m4-max-llama33-70b-8bit-decodeMetric
decode_tok_s
Value
6.5
Unit
tok/s
Model
Llama-3.3-70B (dense) 8-bit
Methodology
THIRD-PARTY MEASURED (Silicon Score Llama 3.3 70B rows). 8-bit quant on ~546 GB/s unified memory; the 4-bit row on the same chip is 11.8 t/s (comm-apple-m4-max-llama33-70b-4bit-decode).
Measured by
third_party
Reproducible
yes
notes
128GB MacBook Pro config. 8-bit 70B is marginal for interactive chat; this is a quality-vs-speed datapoint, not a reason to run 8-bit.
Notes
128GB MacBook Pro config. 8-bit 70B is marginal for interactive chat; this is a quality-vs-speed datapoint, not a reason to run 8-bit.