MLX-LM on 64GB Apple M4 Max: Qwen3-30B-A3B-Instruct-2507 8-bit generation throughput (128 tokens)
83.16 tok/s (generation of 128 tokens, batch 1)
Fields
id
benchmarks/mlx-m4-max-qwen3-30b-a3b-q8-decode-tok-sMetric
decode_tok_s
Value
83.16
Unit
tok/s (generation of 128 tokens, batch 1)
Model
Qwen3-30B-A3B-Instruct-2507 (MLX 8-bit, mlx-community repo)
Methodology
mlx_lm.benchmark --model mlx-community/Qwen3-30B-A3B-Instruct-2507-8bit -p 2048 -g 128. Batch size 1, 2048 prompt tokens, 128 generated tokens. Hardware: Apple M4 Max 64GB unified memory. Software: mlx 0.29.2.dev20251008+85a8824a8, mlx-lm 0.28.2, macOS 26.1. Peak memory: 33.46 GB. MMLU Pro: 72.46.
Measured by
vendor
Reproducible
yes
notes
Official benchmark published by Apple MLX maintainers in mlx_lm/BENCHMARKS.md. Demonstrates that 8-bit quantization on M4 Max achieves 83.16 tok/s with 33.46 GB memory footprint and MMLU Pro of 72.46 (vs 113.33 tok/s and 70.71 MMLU Pro for 4-bit).
Notes
Official benchmark published by Apple MLX maintainers in mlx_lm/BENCHMARKS.md. Demonstrates that 8-bit quantization on M4 Max achieves 83.16 tok/s with 33.46 GB memory footprint and MMLU Pro of 72.46 (vs 113.33 tok/s and 70.71 MMLU Pro for 4-bit).