M3 Ultra 256GB rapid-mlx: Mistral-Small-3.2 4bit long decode
47.2 tok/s (long-context decode, per-request; token count not printed)
Fields
id
benchmarks/rapidmlx-m3ultra256-mistral-small-32-long-decodeMetric
decode_tok_s
Value
47.2
Unit
tok/s (long-context decode, per-request; token count not printed)
Model
Mistral-Small-3.2 (4bit, MLX)
Methodology
rapid-mlx (MLX) eval scorecard, Apple M3 Ultra 256GB, auto-generated 2026-03-05. enable_thinking false, cache cleared between suites. The scorecard prints two decode rates per model, short and long, without stating the token counts behind them, so only the long figure is recorded and the unit says so. TTFT is the cold figure from the per-model block, converted from the printed seconds where the table used seconds. Single machine, one model resident at a time; per-request, not an aggregate across concurrent users. The exact prompt length for the cold TTFT is not printed. This record is the LONG decode column.
Measured by
third_party
Reproducible
yes
notes
Long decode from the scorecard's Decode (l) column. The short figure is in the same source and is not filed, because the two lengths are unnamed. The r/LocalLLaMA post 1rkcvqa quotes this long figure, rounded.
Notes
Long decode from the scorecard's Decode (l) column. The short figure is in the same source and is not filed, because the two lengths are unnamed. The r/LocalLLaMA post 1rkcvqa quotes this long figure, rounded.