MLX-LM on 64GB Apple M4 Max: Qwen3-4B-Instruct-2507 4-bit generation throughput (128 tokens)
134.52 tok/s (generation of 128 tokens, batch 1)
Fields
id
benchmarks/mlx-m4-max-qwen3-4b-q4-decode-tok-sMetric
decode_tok_s
Value
134.52
Unit
tok/s (generation of 128 tokens, batch 1)
Model
Qwen3-4B-Instruct-2507 (MLX 4-bit, mlx-community repo)
Methodology
mlx_lm.benchmark --model mlx-community/Qwen3-4B-Instruct-2507-4bit -p 2048 -g 128. Batch size 1, 2048 prompt tokens, 128 generated tokens. Hardware: Apple M4 Max 64GB unified memory. Software: mlx 0.29.2.dev20251008+85a8824a8, mlx-lm 0.28.2, macOS 26.1. Peak memory: 3.35 GB. MMLU Pro: 60.72.
Measured by
vendor
Reproducible
yes
notes
Official benchmark published by Apple MLX maintainers in mlx_lm/BENCHMARKS.md. Demonstrates memory bandwidth scaling on unified memory: 4-bit quantization reduces memory footprint from 9.02 GB to 3.35 GB and increases generation throughput from 52.47 tok/s to 134.52 tok/s (a 2.56x speedup) while prefill stays roughly flat (1780 -> 1622 tok/s).
Notes
Official benchmark published by Apple MLX maintainers in mlx_lm/BENCHMARKS.md. Demonstrates memory bandwidth scaling on unified memory: 4-bit quantization reduces memory footprint from 9.02 GB to 3.35 GB and increases generation throughput from 52.47 tok/s to 134.52 tok/s (a 2.56x speedup) while prefill stays roughly flat (1780 -> 1622 tok/s).