MLX-LM on MacBook Pro 14-inch M3 Pro 18GB: Qwen3-4B-Instruct-2507 4-bit generation throughput

PER-REQUEST generation throughput, 128 tokens generated after a 2048-token prompt, batch 1, 5 trials averaged (the PR states '5 trials, averages reported'). Command: mlx_lm.benchmark --model mlx-community/Qwen3-4B-Instruct-2507-4bit -p 2048 -g 128. Hardware: MacBook Pro 14-inch M3 Pro, 18GB unified memory, 18-core Neural Engine. Software: mlx 0.32.0, mlx-lm 0.31.3, macOS Sonoma 14.4. Same run reports prompt processing 615.28 tok/s and peak memory 3.30 GB; MMLU Pro 60.72. OMITTED BY SOURCE: GPU core count, sustained-load thermal behaviour (the significant omission for a laptop part), power limit, and whether the 5 trials were interleaved or sequential. This is a laptop APU in a chassis with a shared thermal envelope, unlike the M4 Max desktop rows.

WEAKEST-SOURCED record in this slice and marked draft/0.45 accordingly: a community PR by one poster on a laptop, closed unmerged. Retained because it is the only non-desktop Apple-silicon decode datapoint and because it usefully bounds the low end - 44.62 tok/s for a 4B 4-bit model on an 18GB M3 Pro, versus ~126-135 tok/s for the same 4B model 4-bit on a 64GB M4 Max from a different (Apple-authored) source. That is a ~3x gap for what is nominally the same model and quantization, and it is the clearest evidence in this slice that unified-memory BANDWIDTH, not TOPS, sets Apple decode speed. The two figures are not strictly comparable (different mlx versions, different hosts, one a laptop under thermal pressure) and are flagged as such. ACCELERATOR ID CAVEAT: the repo has accelerators/apple-m3-max and apple-m3-ultra but NO apple-m3-pro record; apple-m3-max is referenced as the nearest existing id so the record resolves, but the silicon is an M3 Pro, not an M3 Max. A dedicated apple-m3-pro accelerator record should be created by whoever owns that slice and this reference corrected. The PR author also asserts 'accuracy is hardware-independent' re: the identical MMLU Pro score - that is a reasonable claim but was not independently checked here. PROVENANCE SEARCHED AND LEFT UNCHANGED 2026-10-08 (w8-bench-src). This was one of the 21 figures measured at Tier D/E and it is the one case where a Tier-A ARTIFACT WAS FOUND AND IS DELIBERATELY NOT CITED, so the reasoning is written out. The artifact: mlx-lm's own merged BENCHMARKS.md, added by PR #552 (merged 2025-10-15T19:24:12Z per the GitHub API, fetched and read), now recorded as [[w8-src-mlxlmm-552-benchmarks-merged]]. It does NOT contain this number. It holds only two model tables - Qwen3-4B-Instruct-2507 (bf16 1780.63 prompt / 52.47 gen; q8 86.907; q6 104.68; q5 g32 110.29; q5 116.39; q4 g32 126.00; q4 134.52) and Qwen3-30B-A3B-Instruct-2507 (q8 83.16 ... q4 113.33) - both explicitly labelled '64GB M4 Max'. So the vendor's own benchmark table covers the M4 Max and SKIPS the M3 Pro, which means 44.62 tok/s is a single community PR (mlx-lm PR #1569, verified 2026-10-08 via GET /repos/ml-explore/mlx-lm/pulls/1569: state=closed, merged=FALSE, merged_at=null, author shreelaxmi-11) on a laptop, and there is no first-party M3 Pro figure to cite. The comparison sentence above ('~126-135 tok/s for the same 4B model 4-bit on a 64GB M4 Max from a different (Apple-authored) source') is CONFIRMED against that merged table: q4 g32 is 126.00 and q4 is 134.52, matching the stated range exactly. VALUE, STATUS, CONFIDENCE AND measured_by ALL UNCHANGED. The verified negative is recorded in [[gotchas/w8-no-vendor-publishes-tok-s-for-these-six-shapes]].