M5 Max 128GB mlx_lm: Qwen3.5-27B-Claude-4.6-Opus-Distilled-MLX-6bit decode at 4k (128 generated tokens)
23.648 tok/s (single-request decode, 128 generated tokens, batch 1)
Fields
id
benchmarks/m5max128-mlx-qwen35-27b-6bit-4k-tgMetric
decode_tok_s
Value
23.648
Unit
tok/s (single-request decode, 128 generated tokens, batch 1)
Model
Qwen3.5-27B-Claude-4.6-Opus-Distilled-MLX-6bit
Methodology
mlx_lm.generate on one 14-inch MacBook Pro, M5 Max, 128 GB unified memory, 40-core GPU (u/cryingneko). Command: mlx_lm.generate --model <local path> --prompt "$(cat /tmp/prompt_<N>.txt)" --max-tokens 128. ONE request, batch size 1: the prompt-tokens-per-sec line is prefill of that single prompt and the generation-tokens-per-sec line is decode of that single completion. NOT aggregate. mlx-lm version not printed. Author states these replaced an earlier BatchGenerator run. Model path /Volumes/SSD/Models/Qwen3.5-27B-Claude-4.6-Opus-Distilled-MLX-6bit. Prompt: 4107 tokens. Generation: 128 tokens. Peak memory: 25.319 GB. DECODE of one request, batch size 1, not aggregate.
Measured by
third_party
Reproducible
yes
notes
Generation tokens-per-sec line. No 4k row for this model exists; siliconscore only kept the 122B and Coder-Next 4k rows.
Notes
Generation tokens-per-sec line. No 4k row for this model exists; siliconscore only kept the 122B and Coder-Next 4k rows.