Geerling ai-benchmarks: Mac Studio M3 Ultra 512GB, Ollama, Llama-3.1-70b (Q4), 14.08 tok/s eval rate, 243W peak
14.08 tok/s eval rate (Ollama-reported decode)
Fields
id
benchmarks/geerling-m3ultra-llama31-70b-q4-ollama-decodeMetric
decode_tok_s
Value
14.08
Unit
tok/s eval rate (Ollama-reported decode)
Model
llama3.1:70b (Ollama Q4_K_M tag)
Methodology
THIRD-PARTY MEASURED (Jeff Geerling, geerlingguy/ai-benchmarks issue #42, obench.sh on Ollama). Mac Studio M3 Ultra 512GB unified memory. Model: llama3.1:70b Ollama default quant (Q4_K_M). Eval rate 14.08 tok/s; peak system power 243W. Per-request decode, batch 1, Ollama default context.
Measured by
third_party
Reproducible
yes
Energy basis
wall power at the plug measured alongside the run; 243W peak reported in the table
Mean power (W)
243
Power scope
node
notes
Same machine does Llama-3.1-405b (Q4) at 1.96 tok/s at 256.4W peak — the 405B spill into ~300GB of weights keeps it inside the 512GB unified pool but decode falls to memory-bandwidth-bound single digits. 14 tok/s on 70B-Q4 is the reference Apple-side answer to 'can a Mac Studio serve 70B'.
Notes
Same machine does Llama-3.1-405b (Q4) at 1.96 tok/s at 256.4W peak — the 405B spill into ~300GB of weights keeps it inside the 512GB unified pool but decode falls to memory-bandwidth-bound single digits. 14 tok/s on 70B-Q4 is the reference Apple-side answer to 'can a Mac Studio serve 70B'.