mlx-lm's own merged benchmark table (mlx_lm/BENCHMARKS.md, added by PR #552 'LLM Benchmarks' by awni, merged 2025-10-15)

THE MERGED ARTIFACT, which is what raises the provenance of the two records that cited only the closed PR. VERIFIED 2026-10-08, TWO WAYS. (a) GitHub API: GET /repos/ml-explore/mlx-lm/pulls/552 returns state=closed, merged=TRUE, merged_at=2025-10-15T19:24:12Z, merge_commit_sha a4c64703906f709a493341508ac31af4085d2638, author awni (ml-explore member), title 'LLM Benchmarks'; GET /repos/ml-explore/mlx-lm/commits?path=mlx_lm/BENCHMARKS.md returns exactly one commit, 'benchmarks (#552)' dated 2025-10-15T19:24:12Z, so the file at main IS that merge commit and has not been touched since. (b) The file at this URL was fetched and read on 2026-10-08. It states its own commands (mlx_lm.benchmark --model model/repo -p 2048 -g 128; mlx_lm.evaluate --model model/repo --task mmlu_pro) and carries exactly two tables. Qwen/Qwen3-30B-A3B-Instruct-2507, 'Performance benchmarks on 64GB M4 Max', mlx 0.29.2.dev20251008+85a8824a8, mlx-lm 0.28.2, macOS 26.1: q8 prompt 1719.47 / gen 83.16 / 33.46 GB / MMLU Pro 72.46; q6 1667.45 / 94.14 / 25.82 / 72.41; q5 1664.24 / 101.00 / 22.01 / 71.97; q4 1753.90 / 113.33 / 18.20 / 70.71; the bf16 row is :skull:. Qwen/Qwen3-4B-Instruct-2507: bf16 prompt 1780.63 / gen 52.47 / 9.02 GB / MMLU Pro 64.05; q8 1606.573 / 86.907 / 5.254 / 63.85; q6 1576.73 / 104.68 / 4.25 / 63.53; q5 g32 1570.80 / 110.29 / 4.00 / 63.16; q5 1584.33 / 116.39 / 3.86 / 62.38; q4 g32 1610.03 / 126.00 / 3.603 / 61.46; q4 1622.27 / 134.52 / 3.35 / 60.72. THE CORRECTION THIS RECORD CARRIES: the previous record notes said the table 'comes from mlx-lm PR #552, which was closed WITHOUT merging in Aug 2026 by maintainer zcbenz citing review capacity' - that is WRONG, and the API shows it. It merged on 2025-10-15. The PR is closed because it merged, not because it was abandoned, and reading a closed PR as an unmerged one is exactly how a live first-party artifact gets filed as a Tier-E community post. WHAT THIS DOES NOT ESTABLISH, said plainly so it is not over-read: this is the project's own published table by the author who wrote the harness, in the project's repository - first-party for MLX-LM as software. It is not a hardware vendor's sustained-throughput spec, and it states no trial count, no GPU-core count for the M4 Max SKU, no thermal state and no sustained-load behaviour. So it upgrades the citation from 'a PR that may never have landed' to 'a document a reader can fetch today, by the author, in the vendor org's repo' - which is what Tier A means here, and nothing more.