Geerling ai-benchmarks: M1 Ultra (64-core GPU, 128GB), Ollama, Llama-3.1-70b (Q4), 9.84 tok/s eval rate
9.84 tok/s eval rate (Ollama-reported decode)
Fields
id
benchmarks/geerling-m1ultra-64gpu-llama31-70b-q4-ollama-decodeMetric
decode_tok_s
Value
9.84
Unit
tok/s eval rate (Ollama-reported decode)
Model
llama3.1:70b (Ollama Q4_K_M tag)
Methodology
THIRD-PARTY MEASURED (Jeff Geerling, geerlingguy/ai-benchmarks issue #48). M1 Ultra Mac Studio, 64-core GPU config, 128GB unified memory. llama3.1:70b Ollama default quant. Eval rate 9.84 tok/s. Per-request decode, batch 1. Power not published for this system ('N/A' in the table).
Measured by
third_party
Reproducible
yes
Power scope
node
notes
M1 Ultra 128GB is the cheapest Apple-Silicon route to a full-70B Q4 resident entirely in unified memory; 9.84 tok/s decode is slow-but-usable for a single interactive session. The 48-core-GPU 64GB sibling does NOT fit 70B-Q4 alongside KV cache, which is why its row only appears in the DeepSeek-14b table.
Notes
M1 Ultra 128GB is the cheapest Apple-Silicon route to a full-70B Q4 resident entirely in unified memory; 9.84 tok/s decode is slow-but-usable for a single interactive session. The 48-core-GPU 64GB sibling does NOT fit 70B-Q4 alongside KV cache, which is why its row only appears in the DeepSeek-14b table.