Apple M3 Ultra (512GB), llama.cpp/MLX: Llama-3.3-70B 8-bit decode tok/s (Silicon Score measured)
8.5 decode tok/s (single-stream, 8-bit)
Fields
id
benchmarks/comm-apple-m3-ultra-llama33-70b-8bit-decodeMetric
decode_tok_s
Value
8.5
Unit
decode tok/s (single-stream, 8-bit)
Model
Llama-3.3-70B (dense) 8-bit
Methodology
THIRD-PARTY MEASURED (Silicon Score Llama 3.3 70B model page + chip rows, cited by research-big-memory.md). M3 Ultra 512GB, ~819 GB/s unified memory: 8.5 tok/s on 70B 8-bit (vs 15.5 tok/s on the same model at 4-bit - the 8-bit weight set reads ~2x the bytes per token, so the halving is the bandwidth-bound signature, not a slowdown).
Measured by
third_party
Reproducible
yes
notes
Gap-fill for the Oct-14 Phase-A ingest: research-big-memory.md gave both the 4-bit (15.5) and 8-bit (8.5) rows for the M3 Ultra but only the 4-bit had a record. The pair is the cleanest demonstration on this hardware class that decode tok/s tracks bytes-per-token at fixed bandwidth.
Notes
Gap-fill for the Oct-14 Phase-A ingest: research-big-memory.md gave both the 4-bit (15.5) and 8-bit (8.5) rows for the M3 Ultra but only the 4-bit had a record. The pair is the cleanest demonstration on this hardware class that decode tok/s tracks bytes-per-token at fixed bandwidth.