27 — Affordable 35B-class fit matrix

Which recorded accelerators can hold a ~35B-class model, and what does the cheapest one cost per unit of decode throughput.

Refresh note (docs/fit-matrix-oct14, Oct-18, this pass). Re-ran python tools/fit_matrix_35b.py on local main 547e8ad (236 accelerator records, 218 supply records, 4,667 benchmarks — #287 through #294 on top of the #286 matrix, which was itself stamped on 81b8382). The selection rule still yields 159 qualifying / 43 priced / 116 unpriced (69 of the unpriced fit 35B-A3B, 47 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match the table already below, so the ranked table does not move and no rank is renumbered. None of the eight commits adds an accelerator or a purchase price. What they add is two more offload-shaped MoE checkpoints, a 74B hybrid whose 4-bit packing sits between the 24 GB fit bar and the 48 GB stretch bar, a diffusion LM whose cache field is zero, and one more graph-capture offload paper measured on the same 35B-A3B class this ceiling is built for.

• SeqMoE says a 24 GB card holds 40% of Qwen3.6-35B-A3B-BF16 and still returns three quarters of full-load decode. SeqMoE: Toward Full-Load Performance via Predictive and Graph-Compatible MoE Offloading measures batch-1 decode with the cache at 45% of experts (40% for Qwen3.6): 75.69% of full-load throughput on Qwen3.6-35B-A3B-BF16, against 84.82% on Qwen3-30B-A3B-FP8, 79.20% on GPT-OSS-120B-MXFP4 and 76.65% on DeepSeek-V4-Flash, averaged to 80.22% in the abstract. Full-load means every weight resident, with pipeline parallelism when one GPU cannot hold them. The same section puts an eager offload at about 2% of full-load, so the gap this paper claims is the orchestration, not the bandwidth. Hardware in its Section 9.1 is three cards already in this table: NVIDIA GeForce RTX 4090 (rank 22, 24 GB) with 120 GB host RAM over PCIe 4.0, NVIDIA GeForce RTX 5090 (rank 12, 32 GB) with 120 GB over PCIe 5.0, and NVIDIA RTX PRO 6000 Blackwell Workstation Edition (rank 34, 96 GB) with 256 GB over PCIe 5.0. On the 4090 over PCIe 4.0 one layer of Qwen3-30B-FP8 compute overlaps 1.39 expert transfers, which is why a one-layer lookahead cannot hide the copy. The predictor itself is 120 MB, 228 MB, 112 MB and 223 MB on those four models, and the authors say the remaining miss (3.25% average, abstract hit rate 96.97%) is still the largest single piece of decode time. These are the authors' measurements; no independent tok/s is in the record, and Figure 6's absolute tok/s were not transcribed. Nothing here changes a fit flag: a Y is still "the 20 GB Q4 file fits," and 75.69% of full-load is a different question this table does not answer.

• Intern-S2-397B is the same 17B-active body as Qwen3.5-397B-A17B, and the file is what does not fit. Intern-S2-397B is 403.423B total / 17B active, 10 of 512 routed experts plus one shared expert, continued from Qwen3.5-397B-A17B. 17/20 = 0.85× the 20 GB the 35B ceiling divides by (derived) — closer to a dense read than Qwen3.6-35B-A3B's 3/20 = 0.15, and the ceiling still divides by the resident file. A 4-bit packing of the safetensors count is 403.423e9 / 2 = 201,711,500,000 bytes = 201,711,500,000 / 1,073,741,824 = 187.86 GiB (derived). That is past 24 GB, past the 48 GB the 70B column tests, and past the 96 GB of rank 34; it needs the 192 GB row (Apple M2 Ultra, rank 33) or larger, or the offload the record describes. The notes derive KV on the 15 full-attention layers as 15 × 2 × 256 × 2 × 2 = 30,720 B/token, the same figure the base record stores. 30,720/20,480 = 1.50× the Qwen3.6-35B-A3B geometry (derived). 8,192 tokens is 30,720 × 8,192 = 240 MiB (derived), and the 4 GB the fit flag reserves holds 4 × 1,024³ / 30,720 = 139,810 tokens (derived). The record's kv_cache_bytes_per_token field is 15,360, half the notes' arithmetic and not what this paragraph uses. No benchmark of this checkpoint lands on a card in §4b.

• DeepSeek-V4-Pro-0813 is a post-train of the 1.6T preview, not a smaller file. DeepSeek-V4-Pro-0813 keeps the preview nameplate: 1,600B total / 49B active, 6 of 384 routed experts plus one shared expert. 49/20 = 2.45× the 20 GB denominator (derived), so the dense-equivalent ceiling overstates a 49B-active decode relative to the 3B-active model it was built for, and the file is the part that does not fit anywhere in the priced table as a single resident checkpoint. A 4-bit packing of the nameplate is 1,600e9 / 2 = 800,000,000,000 bytes = 800,000,000,000 / 1,073,741,824 = 745.06 GiB (derived) — past the 512 GB M3 Ultra (rank 41). The record's own weights are the FP4/FP8 mixed release (safetensors metadata 1,650,497,936,906 parameters, dominated by I8), so the on-disk checkpoint is below that 4-bit-of-BF16 figure, and kv_cache_bytes_per_token stays null because the compressor layers do not collapse to one byte count. Six experts per token is narrower than the 8-expert routing the SeqMoE paper's overlap number was measured against. The card's Terminal-Bench 2.1 is 87.9, a vendor claim, and it is not a tok/s.

• Nemotron 3.5 Bolt is the 24 GB-class cut of the 120B Super VL, and the BF16 parent does not fit 24 GB. NVIDIA Nemotron 3.5 Bolt 74B-A9B is 74B total / 9B active, 8 attention blocks out of 88. 9/20 = 0.45× the 20 GB denominator (derived). A 4-bit packing of the card's round 74B is 74e9 / 2 = 37,000,000,000 bytes = 37,000,000,000 / 1,073,741,824 = 34.46 GiB (derived): past the 24 GB the 35B fit flag tests (so a 24 GB card is still - for the file) and inside the 48 GB the 70B column tests. The card's recommended artifact is a separate 2-bit checkpoint it says fits in 24 GB; this record is the BF16 parent, and that quant is not a record here. KV on the 8 attention blocks is 8 × 2 × 128 × 2 × 2 = 8,192 B/token, 8,192/20,480 = 0.40× the Qwen3.6 figure (derived) — the same ratio the previous note computed for the 120B parent. 8,192 tokens is 64 MiB (derived), and 4 GB holds 4 × 1,024³ / 8,192 = 524,288 tokens (derived). The Mamba state is extra and is not in that KV number.

• K2-Horizon-MoVA's 4-bit file fits inside the 20 GB the ceiling divides by; its KV does not fit inside the 4 GB headroom. K2-Horizon-MoVA-36B-A4B is 37.445B total / 4B active, 8 of 100 routed experts plus one shared expert. 4/20 = 0.20× the denominator (derived). A 4-bit packing is 37.445e9 / 2 = 18,722,500,000 bytes = 18,722,500,000 / 1,073,741,824 = 17.44 GiB (derived), under the 20 GB weights budget, so a card the 35B column marks Y holds this file with room to spare. The cache does not: kv_cache_bytes_per_token is 196,608, 196,608/20,480 = 9.60× the Qwen3.6 geometry (derived), because the Mixture-of-Values heads ride on top of ordinary GQA. 8,192 tokens is 196,608 × 8,192 = 1.50 GiB (derived), and the 4 GB headroom holds 4 × 1,024³ / 196,608 = 21,845 tokens (derived) before the cache is the thing that spills. Terminal-Bench 2.1 at 58.6 is a vendor claim and not a tok/s.

• LLaDA2.2-mini is the one new checkpoint a 12 GB card can hold, and it is not an autoregressive decode. LLaDA2.2-mini is 16.256B total / 1.4B active, 8 of 256 experts. 1.4/20 = 0.07× the denominator (derived). A 4-bit packing is 16.256e9 / 2 = 8,128,000,000 bytes = 8,128,000,000 / 1,073,741,824 = 7.57 GiB (derived), inside the 12 GB of Intel Arc B580 (rank 2), which the 35B column marks -. kv_cache_bytes_per_token is 0: the record is iterative block diffusion with use_cache false, so the 4 GB headroom is not a KV budget. A full bidirectional K+V buffer, which the notes say is not what the field records, would be 20 × 4 × 128 × 2 × 2 = 40,960 B/token (the notes' arithmetic). The card's own serving example is still SGLang at tensor-parallel size 8, because the diffusion block and the 128K context are the constraint, not the parameter count. No absolute tok/s is in the record.

• #294 measures the stretch model on the unpriced 48 GB card, and the prefill half of the same page. Hardware Corner Jun 2025: Quadro RTX 8000, ExLlamaV2 0.2.9, Llama 3.3 70B 5.0 bpw, decode 10.43 t/s at 8k is 10.43 decode tok/s on NVIDIA Quadro RTX 8000 (Turing, 48 GB) (48 GB, 672 GB/s, Y/Y/Y, 35B ceiling 672/20 = 33.6 tok/s (derived), no purchase price, so it stays in §4) for Llama 3.3 70B EXL2 5.0 bpw at 8k, and Hardware Corner Jun 2025: Quadro RTX 8000, llama.cpp b5508, Llama 3.3 70B Q5_K_M, decode 7.59 t/s at 8k is 7.59 tok/s for the same card on Llama 3.3 70B Q5_K_M. The 70B ceiling is 672/40 = 16.8 tok/s (derived); 10.43/16.8 = 0.62× and 7.59/16.8 = 0.45× that floor (derived). Both sit under it, which is what a 5-bit file does against a ceiling built for a 4-bit 40 GB file, and neither is the 35B-A3B workload the ranking sorts on. The rest of the commit is batch-1 llama.cpp prompt processing, not decode. Qwen3 30B-A3B Q4_K_XL on the rank-11 3090 is 1,958.99 tok/s at 16k and 1,336.79 at 32k (Hardware Corner Dec 2025 ranking: RTX 3090, llama.cpp CUDA, Qwen3 30B-A3B Q4_K_XL GGUF, prefill 1958.99 t/s @16K ctx, Hardware Corner Dec 2025 ranking: RTX 3090, llama.cpp CUDA, Qwen3 30B-A3B Q4_K_XL GGUF, prefill 1336.79 t/s @32K ctx). The rank-34 RTX PRO 6000 Blackwell runs the same model at 5,084.45 / 3,863.04 / 2,864.83 / 1,270.31 tok/s for 16k / 32k / 65k / 131k (Hardware Corner Dec 2025 ranking: RTX Pro 6000, llama.cpp CUDA, Qwen3 30B-A3B Q4_K_XL GGUF, prefill 5084.45 t/s @16K ctx, Hardware Corner Dec 2025 ranking: RTX Pro 6000, llama.cpp CUDA, Qwen3 30B-A3B Q4_K_XL GGUF, prefill 3863.04 t/s @32K ctx, Hardware Corner Dec 2025 ranking: RTX Pro 6000, llama.cpp CUDA, Qwen3 30B-A3B Q4_K_XL GGUF, prefill 2864.83 t/s @65K ctx, Hardware Corner Dec 2025 ranking: RTX Pro 6000, llama.cpp CUDA, Qwen3 30B-A3B Q4_K_XL GGUF, prefill 1270.31 t/s @131K ctx). Prefill is not the memory-bandwidth roofline this table divides by, so none of those cells enters the $/tok-s ranking.

• #293's other benchmark cites do not move a byte. The same commit attaches vendor scores to Qwen3.6-35B-A3B (SWE-bench Verified 73.4, GPQA Diamond 86.0, MMLU-Pro 85.2) and to Qwen3.8-27B (SWE-bench Verified 77.2), plus scores on checkpoints whose geometry the earlier notes already sized (VeriLoop E2, Solar Open 2, NVIDIA Nemotron 3.5 Super VL 120B-A12B, Step 5 Preview, Naive-N0.5-Flash, Ornith-1.5-35B-A3B, Qwen3.5-122B-A10B, Qwen3.5-397B-A17B, MiMo-V2.6-Pro-RL, Ling-3.1-flash, Beam 1.0 501B-A23B, AliceAI-Foundation-80B-A3B-Base, Kimi-K2.6, MiMo-V2.6-Flash-RL, NVIDIA Nemotron Labs Dory 1.0 4B, Ornith-1.5-9B, Xing4.0-29B-A4B). None of those edits changes params_b, active_params_b, or a KV byte count, and none is a measured tok/s, so none enters §4b. SpecPrefetch: Parameter-Efficient Expert Prefetching for Sparse MoE Foundation Models was already on main before the #286 stamp; this pass does not re-derive it.

Refresh note (docs/fit-matrix-oct14, Oct-18, prior pass). Re-ran python tools/fit_matrix_35b.py on local main 81b8382 (236 accelerator records, 218 supply records, 4,532 benchmarks — #286 on top of the #285 matrix, which was itself stamped on 84256e2). The selection rule still yields 159 qualifying / 43 priced / 116 unpriced (69 of the unpriced fit 35B-A3B, 47 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match the table already below, so the ranked table does not move and no rank is renumbered. #286 adds no accelerator and no purchase price. What it adds is a 120B vision-language hybrid whose 4-bit packing clears the 48 GB stretch bar and still fits a 96 GB card, with a KV cache smaller than the 35B-A3B the ceiling is built for.

• Nemotron 3.5 Super VL is a 12B-active read of a 120B file, and the file is the part that does not fit. NVIDIA Nemotron 3.5 Super VL 120B-A12B is 120B total / 12B active, hybrid Mamba-2 plus MoE plus attention, 22 of 512 routed experts plus one shared expert. 12/20 = 0.60× the 20 GB the 35B ceiling divides by (derived) — the dense-equivalent ceiling understates a 12B-active decode by less than it understates Qwen3.6-35B-A3B's 3B (3/20 = 0.15), and it still divides by the resident file. The record's params_b is the card's round 120B, not a safetensors count, so a 4-bit packing is 120e9 / 2 = 60,000,000,000 bytes = 60,000,000,000 / 1,073,741,824 = 55.88 GiB (derived, 4-bit = 0.5 B/param). That is past the 24 GB the 35B fit flag tests and past the 48 GB the 70B column tests, and it is inside the 96 GB of NVIDIA RTX PRO 6000 Blackwell Workstation Edition (rank 34) and the 80 GB A100/H100 rows. The card says the other ~490 experts stay in RAM and only the 22 move per token, so the resident-file test and the per-token read are different numbers; this table's Y is the file. KV is the 8 attention blocks only: 8 × 2 kv heads × 128 × 2 × 2 = 8,192 B/token, 8,192/20,480 = 0.40× the Qwen3.6-35B-A3B figure (derived). 8,192 tokens is 8,192 × 8,192 = 64 MiB (derived), and the 4 GB the fit flag reserves holds 4 × 1,024³ / 8,192 = 524,288 tokens (derived) before that headroom is the cache. A 128k window is 8,192 × 131,072 = 1.00 GiB (derived). The record says the Mamba recurrent state is the other memory term and is not in the 8,192. No benchmark of this checkpoint, so it does not enter §4b, and it does not move a fit flag: the 24 GB line is a weight test.

Refresh note (docs/fit-matrix-oct14, Oct-18, prior pass). Re-ran python tools/fit_matrix_35b.py on local main 84256e2 (236 accelerator records, 218 supply records, 4,532 benchmarks — #280, #282 and #283 on top of the #281 matrix, which was itself stamped on db47423). The selection rule still yields 159 qualifying / 43 priced / 116 unpriced (69 of the unpriced fit 35B-A3B, 47 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match the table already below, so the ranked table does not move and no rank is renumbered. None of the three commits adds an accelerator or a purchase price. What they add is a 37.4B MoE whose 4-bit packing fits inside the 20 GB the 35B column divides by, a 250B hybrid whose 4-bit packing does not, a CPU expert kernel whose own numbers are ratios and not tok/s, and the first four measurements on a 72 GB card §4 already lists as unpriced.

• K2-Horizon-MoVA-36B-A4B packs inside the 20 GB column, and its KV does not. K2-Horizon-MoVA-36B-A4B is 37.445B / 4B active, 8 of 100 routed plus one shared expert, and the first three layers are dense. 4/20 = 0.20× the 20 GB the 35B ceiling divides by (derived). A 4-bit packing of the safetensors count is 37.445e9 / 2 = 18,722,500,000 bytes = 17.44 GiB (derived, 4-bit = 0.5 B/param), under the 20 GB weight bar and under the 24 GB the fit flag actually tests. KV is the ordinary GQA formula, and the record says the MoVA value routing is not in it: 48 layers × 8 kv heads × 128 × 2 × 2 = 196,608 B/token, 196,608/20,480 = 9.60× the Qwen3.6-35B-A3B figure (derived). 8,192 tokens is 196,608 × 8,192 = 1,536 MiB (derived), and the 4 GB the fit flag reserves holds 4 × 1,024³ / 196,608 = 21,845 tokens (derived) before the headroom is gone. 17.44 + 1.50 = 18.94 GiB of weights plus that 8k cache (derived), which is why a 24 GB card still holds the short context and why a 128k window does not. The card's Terminal-Bench 2.1 of 58.6 is a vendor claim against Xing4.0-29B-A4B at 57.50, so it does not enter §4b. No benchmark, no rank.
• Solar Open 2 is the other shape: the active slice is small and the checkpoint is not. Solar Open 2 is 250.288B / 15B active, 8 of 320 plus one shared expert, hybrid attention with KV only on 12 of 48 layers. 15/20 = 0.75× that same denominator (derived) — closer to a dense 20 GB read than Qwen3.6-35B-A3B's ~3B, so the ceiling overstates it less and the card still cannot hold the file. A 4-bit packing is 250.288e9 / 2 = 125,144,000,000 bytes = 116.55 GiB (derived), past the 24 GB bar, past the 48 GB the 70B column tests, and past the 96 GB PRO 6000. Growing KV is the 12 softmax layers only: 49,152 B/token, 49,152/20,480 = 2.40× the Qwen3.6-35B-A3B geometry (derived), and 8,192 tokens is 49,152 × 8,192 = 384 MiB (derived). The record's own check is that all 48 layers would be 4× that, 196,608 B/token, which is the quarter the card claims. The 36 linear-attention layers are a fixed recurrent state. LiveCodeBench v6 of 92.4 is a vendor claim. No benchmark, so it does not enter §4b, and it does not move a fit flag: the 24 GB line is a weight test.
• HiNa-MoE changes the CPU expert kernel. It does not change a ceiling. HiNa-MoE: High-Performance, Non-Intrusive MoE Inference on CPUs with Matrix Engines keeps routed-expert weights in plain row-major and fuses the AMX layout transform into the gathered-token matrix, so one buffer serves the CPU path and the GPU path. The authors' own FFN kernel, geometric mean, averages 1.73× over IPEX and 1.68× over KTransformers (peaks 3.37× and 2.16×) and reaches 96% of measured local bandwidth on a dual Xeon Gold 6430 (STREAM local 234 GB/s, cross-socket 95 GB/s, 95/234 = 0.41× (derived)). End to end, batch 1, 128 in and 128 out, dual 6430 plus an NVIDIA RTX A6000: decode averages 1.99× / 1.70× / 1.22× over vLLM CPU-only, IPEX and KTransformers, and the record says the IPEX and KTransformers builds already contain the HiNa-MoE operators. No absolute decode tok/s is printed. Prefill on Qwen2-57B-A14B with GQA on the A6000 and experts on the CPU is 1,526.66 tok/s at 2K against 1,187.43 for SGLang plus KTransformers (1.29×, 1526.66/1187.43 (derived)) and 2,085.47 at 32K against 1,252.54 (1.67× (derived)). A6000 stays the unpriced §4 row at 768 GB/s (38.4 tok/s ceiling (derived, 768/20)) and the §4b cell at 131.2 tok/s. A kernel ratio on a host-RAM expert path does not move that row.
• The 72 GB PRO 5000 now has four numbers, and none of them is the 35B-A3B. NVIDIA RTX PRO 5000 72GB Blackwell Workstation Edition is already the §4 row at 72 GB, 1,344 GB/s, ceiling 1,344/20 = 67.2 tok/s (derived), Y/Y/Y, no bare-card price — tied with the 48 GB sibling on bandwidth, with the extra 24 GB buying the stretch model and no tok/s. HOSTKEY's own Ollama table (quant, prompt length and output length unstated, confidence 0.55) is the first published set: deepseek-r1:14b 67.71 (HOSTKEY: RTX PRO 5000 72GB Blackwell, deepseek-r1:14b, Ollama, avg decode, 67.71/67.2 = 1.01× the 20 GB floor (derived)), deepseek-r1:32b 32.89 (HOSTKEY: RTX PRO 5000 72GB Blackwell, deepseek-r1:32b, Ollama, avg decode, 32.89/84.0 = 0.39× the 27B ceiling of 1,344/16 (derived)), gpt-oss:20b 149.38 (HOSTKEY: RTX PRO 5000 72GB Blackwell, gpt-oss:20b, Ollama, avg decode, 149.38/67.2 = 2.22× (derived) — the wrong byte count for a ~21B MoE, same caveat the 4080/5080 note already uses), and qwen3-coder-next 74.34 (HOSTKEY: RTX PRO 5000 72GB Blackwell, qwen3-coder-next, Ollama, avg decode, 74.34/67.2 = 1.11× (derived)). The page says that last model is about 51 GB, so the dense-equivalent read is 1,344/51 = 26.4 tok/s and 74.34/26.4 = 2.82× (derived), which is a calibration of a ~51 GB file, not a Q4 35B-A3B. The 48 GB sibling's §4b cell stays 188.8 at 4k on a stated Q4_K; these four are a different engine with the quant unpublished, so they are not averaged into that cell and they do not enter §4b. Still no purchase price, so the row stays in §4.

Refresh note (docs/fit-matrix-oct14, Oct-18, prior pass). Re-ran python tools/fit_matrix_35b.py on local main db47423 (236 accelerator records, 218 supply records, 4,528 benchmarks — #275, #276, #278 and #279 on top of the #274 matrix, which was itself stamped on 4f8ec2e). The selection rule still yields 159 qualifying / 43 priced / 116 unpriced (69 of the unpriced fit 35B-A3B, 47 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match the table already below, so the ranked table does not move and no rank is renumbered. None of the four commits adds an accelerator or a purchase price. What they add is a 27.8B dense hybrid that fits the same cards the 27B column already tests, a 2023 sub-1-bit expert kernel whose own end-to-end number is slower than the uncompressed baseline, 113 ranking-page cells that are a different Hardware Corner page from the ones §4b already quotes, and a draft-selection rule that cuts host-to-device expert copies rather than moving a bandwidth ceiling.

• VeriLoop E2 is the same 27.8B dense hybrid the 27B column already prices. VeriLoop E2 is a fine-tune of Qwen3.8-27B: 27.781B parameters, active_params_b equal to params_b (no num_experts key), 64 layers, KV 65,536 B/token. 27.781/16 = 1.74× the 16 GB the 27B-Q4 weight column divides by (derived), so a Q4 packing of it is the workload that column is for, and the whole checkpoint is resident wherever that one is — the record says so, and nothing is offloaded to RAM. Its KV is 65,536/20,480 = 3.20× the Qwen3.6-35B-A3B figure (derived), and 8,192 tokens is 65,536 × 8,192 = 512 MiB (derived). That is context cost, not a weight-fit change: the 20 GB bar is unchanged, and the card's GPQA / AIME / SWE-bench numbers are vendor claims, so they do not enter §4b.
• QMoE's kernel is a GEMV, and the paper's own end-to-end number loses. QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models stores a ternary routed expert in a 16-bit-codeword dictionary at under 1 bit per parameter. The isolated compressed matrix-vector product beats PyTorch bfloat16 cuBLAS by up to 35 percent on the SwitchTransformer expert shapes, but the same weights decoded to 128 tokens are under 5 percent slower than the uncompressed baseline (The QMoE sub-1-bit expert kernel is a GEMV, and it loses as soon as two tokens share an expert). The kernel is one warp per row and one codeword per step, so a second token on the same expert is a second launch while the baseline runs one matmul. Both numbers are the authors' own, on NVIDIA RTX A6000 and NVIDIA GeForce RTX 3090 only, and the end-to-end comparison is against a simulated baseline that points every expert at one copy of the weights. The checkpoints are a custom dictionary format, not GGUF or GPTQ. A6000 stays the unpriced §4 row at 768 GB/s (38.4 tok/s ceiling (derived, 768/20)); the 3090 stays rank 11. Neither number is a decode tok/s, so neither enters §4b.
• #278 is a second Hardware Corner page, and it does not replace the cells §4b ranks on. 113 new per-request decode rows from the December 9, 2025 ranking table (llama.cpp CUDA 12.8, Q4_K_XL, batch 1, 16K/32K/65K/131K). The commit skips any cell whose value was already stored, which is why these do not collide with the March per-card pages. They also do not agree with them. Qwen3-30B-A3B Q4_K_XL at 16k on this page: 113.84 on the 3090 (Hardware Corner Dec 2025 ranking: RTX 3090, llama.cpp CUDA, Qwen3 30B-A3B Q4_K_XL GGUF, decode 113.84 t/s @16K ctx, 113.84/46.8 = 2.43× (derived)) against the March page's 98.3; 121.88 on the 3090 Ti (Hardware Corner Dec 2025 ranking: RTX 3090 Ti, llama.cpp CUDA, Qwen3 30B-A3B Q4_K_XL GGUF, decode 121.88 t/s @16K ctx, 121.88/50.4 = 2.42×) against 121.9 on the March page and 98.25 on the older ranking ingest; 139.71 on the 4090 (Hardware Corner Dec 2025 ranking: RTX 4090, llama.cpp CUDA, Qwen3 30B-A3B Q4_K_XL GGUF, decode 139.71 t/s @16K ctx, 139.71/50.4 = 2.77×) against 139.2 in March and 109.57 on the older page; 120.12 on the 6000 Ada (Hardware Corner Dec 2025 ranking: RTX 6000 Ada, llama.cpp CUDA, Qwen3 30B-A3B Q4_K_XL GGUF, decode 120.12 t/s @16K ctx, 120.12/48.0 = 2.50×) against 120.1 in March and 104.88 on the older page. Same shape at 32k: 3090 87.21 (87.21/46.8 = 1.86×, Hardware Corner Dec 2025 ranking: RTX 3090, llama.cpp CUDA, Qwen3 30B-A3B Q4_K_XL GGUF, decode 87.21 t/s @32K ctx) against 58.8 in March. The two pages are different measurements of the same card and quant, so neither is averaged into the cell §4b already cites, and neither moves a rank. The page does add depths the March series does not have. NVIDIA GeForce RTX 5090 (rank 12, $1,999, 89.6 tok/s ceiling): 141.63 at 16k (1.58×, Hardware Corner Dec 2025 ranking: RTX 5090, llama.cpp CUDA, Qwen3 30B-A3B Q4_K_XL GGUF, decode 141.63 t/s @16K ctx), 110.65 at 32k (1.23×), 76.56 at 65k (0.85×), 52.84 at 131k (0.59×) — all ÷ 89.6 (derived). The floor is crossed between 32k and 65k, the same place the 3090 and the 6000 Ada already cross it. NVIDIA RTX PRO 6000 Blackwell Workstation Edition (rank 34, $8,650, same 89.6 ceiling) holds the 30B-A3B through 131k above the floor at 16k and 32k: 139.76 / 103.33 / 80.15 / 56.14 (1.56× / 1.15× / 0.89× / 0.63×, Hardware Corner Dec 2025 ranking: RTX Pro 6000 WS, llama.cpp CUDA, Qwen3 30B-A3B Q4_K_XL GGUF, decode 139.76 t/s @16K ctx and the 32k/65k/131k siblings) — the 65k cell is the first one under it. The same card's Llama 70B Q4_K_XL row is the first measured 70B-class cell on a priced 70B-fit part: 28.24 at 16k (Hardware Corner Dec 2025 ranking: RTX Pro 6000 WS, llama.cpp CUDA, Llama 70B Q4_K_XL GGUF, decode 28.24 t/s @16K ctx, 28.24/44.8 = 0.63× the 40 GB dense ceiling (derived)), then 25.67 / 21.67 / 16.62 at 32k/65k/131k. $8,650/28.24 = $306.30/measured-tok/s (derived) at 16k. Its dense Qwen3-32B sibling on the same page is 45.72 at 16k (45.72/56.0 = 0.82× a 32 GB-equivalent read, 1,792/32 (derived)), so the 70B file, not the card, is what puts the row under the floor.
• SpecFit changes which draft tokens cause a copy. It does not change the ceiling. SpecFit: Not All Draft Tokens Cost the Same in Offloaded MoE Inference keeps the EAGLE-3 drafter and the target router, and picks the ancestor-closed subtree that maximises expected committed tokens over the per-layer expert union. On Qwen3-30B-A3B with every expert in host RAM, one NVIDIA A100 40GB PCIe (rank 33, 77.8 tok/s ceiling), 48 GPU slots per layer: 8.27 tok/s on CNN/DailyMail against 6.80 for cached EAGLE-3 (1.22×, 8.27/6.80 (derived)) and 4.67 autoregressive (1.77×), and 10.79 on GSM8K against 10.08 (1.07×) and 4.70 (2.30×). The abstract's 1.97× and 1.72× are against EAGLE-3 with the cache cleared, which the record separates from the cached comparison. On the NVIDIA GeForce RTX 4090 (24 GB, the card the $2,000 sticker prices) with 32 slots the GSM8K gain is 6.70 against 6.05 tok/s (1.11× (derived)). The published runs read routes from an atlas built outside the timer; the record says a deployed selector needs a predictor the paper does not train. 8.27 is an offload measurement on a card whose §4b cell is 136 tok/s resident, so it does not enter that column and it does not move rank 33.

Refresh note (docs/fit-matrix-oct14, Oct-18, prior pass). Re-ran python tools/fit_matrix_35b.py on local main 4f8ec2e (236 accelerator records, 218 supply records, 4,415 benchmarks — #268 through #273 on top of the #267 matrix, which was itself stamped on d17b963). The selection rule still yields 159 qualifying / 43 priced / 116 unpriced (69 of the unpriced fit 35B-A3B, 47 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match the table already below, so the ranked table does not move and no rank is renumbered. None of the six commits adds a purchase price. What they add is six Apple bins that miss the bandwidth floor, four more offload-shaped MoE checkpoints (and the active-parameter count the previous note left null), and two more offload papers.

• Six new Apple records, and none of them clears the gate. #272 files the #4167 bins the catalog was missing: Apple M1 (base, 7- or 8-core GPU) (16 GB, 68 GB/s), Apple M1 Pro (14- or 16-core GPU) (32 GB, 200 GB/s), Apple M2 (base, 8- or 10-core GPU) (24 GB, 100 GB/s), Apple M2 Pro (16- or 19-core GPU) (32 GB, 200 GB/s), Apple M3 (base, 10-core GPU) (24 GB, 100 GB/s) and Apple M3 Pro (14-core GPU bin) (36 GB, 150 GB/s). The widest of the six is 200/300 = 0.67 of the bandwidth floor (derived), and none has a purchase-price supply record, so the <= $3,000 arm does not pull them in either. Two of them would fit the 35B-A3B weight bar if the floor were waived: M1 Pro at 32 GB and M3 Pro 14-core at 36 GB are both past the 24 GB line, and M2 Pro at 32 GB is too. Their ceilings, if the floor did not drop them, would be 200/20 = 10.0, 150/20 = 7.5 and 200/20 = 10.0 tok/s (derived) — under the L4's 15.0, which is the slowest row §4 still lists. They do not enter §3 or §4.
• The cells on those bins are Llama-2-7B, so they calibrate the bus, not the 20 GB file. llama.cpp Metal, build 8e672ef, batch 1, tg128, Q4_0: 14.19 on the M1 7-core and 14.15 on the 8-core (llama.cpp Metal on Apple M1 (base, 7- or 8-core GPU) (7 GPU): Llama-2-7B Q4_0 tg128 tok/s (scoreboard #4167, 8e672ef), llama.cpp Metal on Apple M1 (base, 7- or 8-core GPU) (8 GPU): Llama-2-7B Q4_0 tg128 tok/s (scoreboard #4167, 8e672ef)); 35.52 / 36.41 on the M1 Pro 14-core and 16-core (llama.cpp Metal on Apple M1 Pro (14- or 16-core GPU) (14 GPU): Llama-2-7B Q4_0 tg128 tok/s (scoreboard #4167, 8e672ef), llama.cpp Metal on Apple M1 Pro (14- or 16-core GPU) (16 GPU): Llama-2-7B Q4_0 tg128 tok/s (scoreboard #4167, 8e672ef)); 21.7 / 21.91 on the M2 8-core and 10-core (llama.cpp Metal on Apple M2 (base, 8- or 10-core GPU) (8 GPU): Llama-2-7B Q4_0 tg128 tok/s (scoreboard #4167, 8e672ef), llama.cpp Metal on Apple M2 (base, 8- or 10-core GPU) (10 GPU): Llama-2-7B Q4_0 tg128 tok/s (scoreboard #4167, 8e672ef)); 37.87 / 38.86 on the M2 Pro 16-core and 19-core (llama.cpp Metal on Apple M2 Pro (16- or 19-core GPU) (16 GPU): Llama-2-7B Q4_0 tg128 tok/s (scoreboard #4167, 8e672ef), llama.cpp Metal on Apple M2 Pro (16- or 19-core GPU) (19 GPU): Llama-2-7B Q4_0 tg128 tok/s (scoreboard #4167, 8e672ef)); 21.34 on the M3 10-core (llama.cpp Metal on Apple M3 (base, 8- or 10-core GPU) (10 GPU): Llama-2-7B Q4_0 tg128 tok/s (scoreboard #4167, 8e672ef)); 30.65 on the M3 Pro 14-core (llama.cpp Metal on Apple M3 Pro (14-core GPU bin) (14 GPU): Llama-2-7B Q4_0 tg128 tok/s (scoreboard #4167, 8e672ef)). Dividing by the 20 GB denominator — the only ceiling this doc computes, and the wrong byte count for a 7B Q4_0 — gives 14.19/(68/20) = 14.19/3.4 = 4.2× on the M1 and 36.41/(200/20) = 36.41/10.0 = 3.64× on the M1 Pro 16-core (derived). Same caveat the RTX 4090 D note already uses: a small-model ratio, not a claim the bin decodes a 35B-A3B at that rate. Not a re-rank.
• Four more MoE checkpoints, and the one the previous note left blank now has an active count. MiMo-V2.6-Pro-RL was filed with active_params_b null; #270 fills it at 42B from the card's Model Summary (1.02T total, 8 of 384 routed, no shared expert). 42/20 = 2.10× the 20 GB the 35B column divides by (derived) — the dense-equivalent ceiling overstates the rate by more than it does for Step 5's 1.35×, and the KV the previous note computed does not change: 10 global layers, 51,200 B/token, 8,192 tokens = 400 MiB (derived). Qwen3.5-122B-A10B is 121.349B language-model parameters / 10B active, 8 of 256 plus one shared expert. 10/20 = 0.50× that denominator (derived). Its growing KV is the 12 full-attention layers only: 24,576 B/token, 24,576/20,480 = 1.20× the Qwen3.6-35B-A3B geometry (derived), and 8,192 tokens is 24,576 × 8,192 = 192 MiB (derived). The 36 linear-attention layers are a fixed recurrent state, not part of the per-token number. A 4-bit packing of the language-model count is 121.349e9 / 2 = 60,674,500,000 bytes = 56.51 GiB (derived, 4-bit = 0.5 B/param), past the 24 GB bar and past the 48 GB the 70B column tests. Qwen3.5-397B-A17B is 403.398B / 17B active, 10 of 512 plus one shared expert. 17/20 = 0.85× (derived). Growing KV is the 15 full-attention layers: 30,720 B/token, 30,720/20,480 = 1.50× (derived), and 8,192 tokens is 30,720 × 8,192 = 240 MiB (derived). The checkpoint is BF16, 403,397,928,944 parameters, so a 4-bit packing is half of that byte count: 403,397,928,944 / 2 = 201,698,964,472 bytes = 187.85 GiB (derived). Naive-N0.5-Flash is 309B / 15.5B active, 8 of 256 and no shared expert. 15.5/20 = 0.78× (derived). There is no full-attention layer: the growing term is the 9 sparse-attention layers, 23,040 B/token, 23,040/20,480 = 1.12× (derived), and 8,192 tokens is 23,040 × 8,192 = 180 MiB (derived). The 39 sliding layers are a fixed 128-token window, 24.4 MiB (the record's figure). The record's own storage figure is 617,823,283,433 bytes, which is 617,823,283,433 / 1,073,741,824 = 575.2 GiB (derived) and matches the card's BF16 dump. Kimi-K2.6 is 1,026.879B / 32B active, 8 of 384 plus one shared expert, and the routed experts already ship INT4. 32/20 = 1.60× (derived). KV is MLA, all 61 layers: 70,272 B/token, 70,272/70,656 = 0.99× Step 5's geometry (derived), and 8,192 tokens is 70,272 × 8,192 = 549 MiB (derived). None of the five has a benchmark, so none enters §4b, and none moves a fit flag: the 24 GB line is a weight test, and every one of these checkpoints is an offload.
• HOBBIT is a second copy of each expert, and the card it was timed on is already rank 24. HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inference keeps a high-precision and a low-precision copy of every routed expert and fetches the small one when the gate says the expert is not the top one. The authors' own batch-1 numbers on NVIDIA GeForce RTX 4090 (24 GB, PCIe 4.0, 256 GB host RAM) are 3.21× / 3.29× decode against MoE-Offloading and 2.30× / 3.92× against MoE-Infinity, on Mixtral-8x7B and Phi-MoE. The same record says the prefetch by itself is only about 1.05× once the small copy exists — almost all of the gain is the smaller miss, not the overlap. The 9.93× and the 13.0×/18.9× against llama.cpp are Jetson AGX Orin unified-memory numbers where the baseline is paging off a slow SSD, and the record says so. NVIDIA Jetson AGX Orin 64GB is the 64 GB bin at 204.8 GB/s, the same shape as the GB10/Thor exclusion already in the note below (under 300 GB/s, no <= $3,000 price), so it stays outside the 159. A software multiplier on an offload path does not move a ceiling or a rank.
• MoE-CORE is the same idea on an Ascend the corpus does not price. MoE-CORE: Coordinated Expert Offloading and Residency for Memory-Constrained MoE Inference stages a full expert layer during prefill and a nonuniform per-layer cache during decode, on one Ascend 950PR under an 84 GB cap. Mean TPOT against load-on-demand on DeepSeek-V4-Flash-W4A8 is 1.39× to 1.75× (38.0 vs 66.5 ms on GSM8K, 40.3 vs 70.6 ms on LongBench v2). Prefetch depth on that model: 42.6 ms with none, 36.0 ms at one expert (42.6/36.0 = 1.18×, the record's 15.5%), 35.9 ms at two and 36.5 ms at three — the third expert raised TPOT because the copy no longer fit under the remaining compute. There is no Ascend 950PR accelerator record and no purchase price, so it does not enter §3 or §4. The 38–45 ms vs 1,269 ms gap against vLLM prefetch is the paper's own full-layer baseline at a 128-token cap, not a kernel result, and the record says so.

Refresh note (docs/fit-matrix-oct14, Oct-18, prior pass). Re-ran python tools/fit_matrix_35b.py on local main d17b963 (230 accelerator records, 218 supply records, 4,382 benchmarks — #263, #264, #265 and #266 on top of the #262 matrix, which was itself stamped on c104441). The selection rule still yields 159 qualifying / 43 priced / 116 unpriced (69 of the unpriced fit 35B-A3B, 47 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match the table already below, so the ranked table does not move and no rank is renumbered. None of the four commits adds an accelerator or a purchase price. What they add is three more offload-shaped MoE checkpoints and one prefetch paper, plus the mid-context Hardware Corner cells the 3090 and 5090 rows in §4b were missing.

• Step 5 Preview is a 27B per-token read of a 600B file, and the file is what has to be resident. Step 5 Preview is 600B total / 27B active, 8 of 352 experts plus one shared expert of width 1536, and 88 of 92 backbone layers are MoE. CORRECTED 2026-10-18: an earlier version of this note called the 27B active slice's 27e9 × 0.5 = 13.5 GB Q4 figure "inside the 16 GB the 27B column divides by". That compared a per-token read against a capacity and implied a 600B checkpoint fits a 16 GB card. The two numbers are different: 13.5 GB is the weights read per token (derived), not the memory required. The resident file at 4-bit is 600e9 × 0.5 = 300,000,000,000 bytes = 279.4 GiB (derived), or about 1,118 GiB at BF16. It fits 16 GB, 24 GB and 48 GB on no reading, which is the same conclusion the card's own serving configs give (FP8 on 4 B300s or 8 H200s). 27 / 20 = 1.35× (derived) described only how much larger the active set is than a 3B one; it was never a fit test, because the 20 GB denominator is the byte count of a 35B file and this is a 600B one. Its growing KV counts only the 23 full-attention layers: 70,656 B/token, so 8,192 tokens is 70,656 × 8,192 = 578,813,952 bytes = 552 MiB (derived). The 69 sliding layers are a fixed 512-token window, 103.5 MiB (the record's figure), not part of the per-token number. The BF16 checkpoint is about 1,118 GiB (derived, 600e9 × 2 / 1,073,741,824); the card's serving configs start at FP8 on 4 B300s or 8 H200s. No single-GPU offload recipe and no benchmark, so it does not enter §4b and does not move a fit flag. The 24 GB line is unchanged.

• Ling-3.1-flash is the small-active shape again, and MiMo-V2.6-Pro-RL is the one whose KV the 4 GB headroom does not cover. Ling-3.1-flash is 560B total / 25B active, 8 of 512 experts plus one shared expert, FP8 on disk. Its growing KV is the 9 MLA layers only: 10,368 B/token, 10,368 / 20,480 = 0.51× the Qwen3.6-35B-A3B geometry (derived), and 8,192 tokens is 10,368 × 8,192 = 81 MiB (derived). 25 / 20 = 1.25× the 20 GB denominator (derived). MiMo-V2.6-Pro-RL is the page's round 1T with active_params_b left null (the card publishes no active count), routing 8 of 384 experts and no shared expert. Its growing KV is the 10 global layers: 10 × 8 × (192 + 128) × 2 = 51,200 B/token, 51,200 / 20,480 = 2.50× the 35B-A3B geometry and 51,200 / 70,656 = 0.72× Step 5's (derived). 8,192 tokens is 51,200 × 8,192 = 400 MiB (derived), still inside the 4 GB the fit flag reserves; a 128k context is 51,200 × 131,072 = 6,710,886,400 bytes = 6.25 GiB (derived), which is the headroom and not the weights. The 60 sliding layers are a fixed 128-token window, 39.3 MB (the record's figure). Neither checkpoint has a benchmark, so neither enters §4b.

• DuoServe-MoE is a prefetch schedule, and the cards it was measured on are already in the corpus. DuoServe-MoE: Dual-Phase Expert Prefetch and Caching for LLM Inference QoS Assurance splits the expert fetch by phase: prefill pipelines one expert at a time, decode predicts the next layer, and the on-device expert cache is sized to k rather than to the pool. The authors' own band is TTFT 1.78× to 5.34× and end-to-end 1.42× to 7.55× against on-demand fetch and full-layer prefetch, single request, PCIe 4.0, on an RTX A5000 24 GB and an RTX A6000 48 GB. The A3B-shaped cell is Qwen3-30B-A3B in FP8: TTFT over 5× versus full-layer prefetch and about 1.95× to 1.99× versus on-demand fetch, with peak GPU memory 1.54 GB versus 1.24 / 1.80 / 7.67 GB. NVIDIA RTX A5000 is 24 GB at 768 GB/s, a 38.4 tok/s ceiling on the 20 GB column (derived, 768/20) and unpriced, so it stays in §4. NVIDIA RTX A6000 is already the §4b row at 131.2 tok/s (3.4× its 38.4 floor). The paper's ratios are against other offload schedules, not against that roofline, so they do not enter the × column and do not move a rank.

• #264 fills the mid-context cells on the two priced cards §4b already ranks, and the MXFP4 MoE does not fall off the way the Q4_K one does. Sixty new rows, all llama.cpp llama-bench, batch-1, third-party, on NVIDIA GeForce RTX 3090 (rank 11, $950, 46.8 tok/s ceiling) and NVIDIA GeForce RTX 5090 (rank 12, $1,999, 89.6). The cells that are the workload:

  • 3090, Qwen3.5-35B MXFP4. 107.1 at 16k, 101.2 at 32k, 93.1 at 64k (NVIDIA RTX 3090 llama.cpp: Qwen3.5 35B (MXFP4) token generation at 16k context, NVIDIA RTX 3090 llama.cpp: Qwen3.5 35B (MXFP4) token generation at 32k context, NVIDIA RTX 3090 llama.cpp: Qwen3.5 35B (MXFP4) token generation at 64k context). Against the 46.8 floor that is 2.29× / 2.16× / 1.99× (derived). The Q4_K 30B-A3B series already in §4b is 98.3 / 58.8 / 32.8 at the same three depths, so the MXFP4 cell is 107.1/98.3 = 1.09× at 16k, 101.2/58.8 = 1.72× at 32k and 93.1/32.8 = 2.84× at 64k (derived), and it is still above the floor at 64k where the Q4_K series has already crossed it (0.70×). $950 / 107.1 = $8.87/measured-tok/s at 16k and $950 / 93.1 = $10.20 at 64k (derived). Not a re-rank: a different quant and a different model from the Q4_K series the paragraph below ranks on.
  • 3090, the dense 27B column, and it is under its own floor. Qwen3.5-27B Q4_K is 32.3 at 16k and 31.0 at 32k (NVIDIA RTX 3090 llama.cpp: Qwen3.5 27B (Q4_K) token generation at 16k context, NVIDIA RTX 3090 llama.cpp: Qwen3.5 27B (Q4_K) token generation at 32k context), against the 58.5 tok/s 27B ceiling: 32.3/58.5 = 0.55× and 31.0/58.5 = 0.53× (derived). Gemma4-31B Q4_K at 16k is 33.5, 33.5/58.5 = 0.57× (NVIDIA RTX 3090 llama.cpp: Gemma4 31B (Q4_K) token generation at 16k context). The MoE/dense ratio on this card at 16k is 107.1/32.3 = 3.32× (derived). $950 / 32.3 = $29.41/measured-tok/s (derived).
  • 5090, the same MXFP4 file, and the bandwidth ratio does not show up. 148.3 / 143.2 / 133.5 at 16k / 32k / 64k (NVIDIA RTX 5090 llama.cpp: Qwen3.5 35B (MXFP4) token generation at 16k context, NVIDIA RTX 5090 llama.cpp: Qwen3.5 35B (MXFP4) token generation at 32k context, NVIDIA RTX 5090 llama.cpp: Qwen3.5 35B (MXFP4) token generation at 64k context), which is 148.3/89.6 = 1.66×, 143.2/89.6 = 1.60×, 133.5/89.6 = 1.49× (derived) — still above the floor at 64k, and a smaller margin than the 3090's because the 5090's floor is higher. 148.3/107.1 = 1.38×, 143.2/101.2 = 1.42×, 133.5/93.1 = 1.43× (derived), against a bandwidth ratio of 1,792/936 = 1.91×. $1,999 / 148.3 = $13.48/measured-tok/s at 16k (derived), which does not beat the 3090's $8.87 on the same file.
  • 5090, dense, also under its own floor. Qwen3.5-27B Q4_K is 55.8 / 53.8 / 50.1 at 16k / 32k / 64k (NVIDIA RTX 5090 llama.cpp: Qwen3.5 27B (Q4_K) token generation at 16k context, NVIDIA RTX 5090 llama.cpp: Qwen3.5 27B (Q4_K) token generation at 32k context, NVIDIA RTX 5090 llama.cpp: Qwen3.5 27B (Q4_K) token generation at 64k context): 55.8/112.0 = 0.50×, 53.8/112.0 = 0.48×, 50.1/112.0 = 0.45× (derived). Gemma4-31B Q4_K is 59.2 / 55.4 / 51.6 (0.53× / 0.49× / 0.46×, NVIDIA RTX 5090 llama.cpp: Gemma4 31B (Q4_K) token generation at 16k context and the 32k and 64k siblings) and Qwen3-32B Q4_K at 16k is 50.9 (0.45×, NVIDIA RTX 5090 llama.cpp: Qwen3 32B (Q4_K) token generation at 16k context). The MoE/dense ratio at 16k is 148.3/55.8 = 2.66× (derived). $1,999 / 55.8 = $35.82/measured-tok/s (derived).
  • The smaller files stay above the 20 GB floor for longer, and they are not the workload. Gemma4-26B Q4_K on the 3090 is 115.0 / 107.5 / 98.9 at 16k / 32k / 64k (2.46× / 2.30× / 2.11× the 46.8 floor, NVIDIA RTX 3090 llama.cpp: Gemma4 26B (Q4_K) token generation at 16k context and siblings); on the 5090 it is 167.2 / 159.4 / 149.4 (1.87× / 1.78× / 1.67×, NVIDIA RTX 5090 llama.cpp: Gemma4 26B (Q4_K) token generation at 16k context and siblings). gpt-oss-20B Q4_K on the 5090 is 249.2 / 215.1 / 169.0 (2.78× / 2.40× / 1.89×, NVIDIA RTX 5090 llama.cpp: gpt-oss 20B (Q4_K) token generation at 16k context and siblings). Qwen3-14B Q4_K on the 5090 is the one that does cross: 123.8 at 4k (1.38×), 102.7 at 16k (1.15×), 82.4 at 32k (0.92×), 57.6 at 64k (0.64×), 37.2 at 128k (0.42×) — NVIDIA RTX 5090 llama.cpp: Qwen3 14B (Q4_K) token generation at 4k context through NVIDIA RTX 5090 llama.cpp: Qwen3 14B (Q4_K) token generation at 128k context, all ÷ 89.6 (derived). A 14B dense file crossing the 20 GB floor is KV, not weights. The prefill siblings are not a decode ceiling and are not cited as one.

Refresh note (docs/fit-matrix-oct14, Oct-18, prior pass). Re-ran python tools/fit_matrix_35b.py on local main c104441 (230 accelerator records, 218 supply records, 4,306 benchmarks — #257, #258 and #259 on top of the #256 matrix, which was itself stamped on e525b48). The selection rule still yields 159 qualifying / 43 priced / 116 unpriced (69 of the unpriced fit 35B-A3B, 47 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match the table already below, so the ranked table does not move and no rank is renumbered. None of the three commits adds an accelerator or a purchase price. What they add is the named 35B-A3B checkpoint the 20 GB column was written for, plus two larger MoE checkpoints and one prefetch paper.

• The 20 GB column now has the model the brief names. Qwen3.6-35B-A3B is 35.952B total / 3B active, 256 experts with 8 routed plus one shared expert, and its KV is 20,480 B/token — the same geometry the previous note recorded for Ornith-1.5-35B-A3B. The record's own safetensors count is 35,951,822,704 BF16 parameters, so the checkpoint is 35,951,822,704 × 2 = 71,903,645,408 bytes = 71,903,645,408 / 1,073,741,824 = 66.97 GiB (derived). A 4-bit packing of the same count is a quarter of that, 66.97 / 4 = 16.74 GiB (derived), inside the 20 GB the ceiling divides by. The 4 GB headroom at 8,192 tokens is 20,480 × 8,192 = 167,772,160 bytes = 160 MiB (derived), so the 4 GB the fit flag reserves is runtime and the OS, not the KV. Same threshold as before: nothing in the new record moves the 24 GB line, and there is no benchmark of it, so it does not enter §4b.

• Two more MoE checkpoints, and neither is the 20 GB file. AliceAI-Foundation-80B-A3B-Base activates the same 3B per token (10 of 512 experts plus one shared expert) but the record's safetensors count is 81,286,433,408 BF16 parameters, so the checkpoint is 81,286,433,408 × 2 = 162,572,866,816 bytes = 151.41 GiB (derived) and a 4-bit packing is 151.41 / 4 = 37.85 GiB (derived). That clears the 35B-A3B 24 GB bar and misses the 70B-Q4 48 GB bar, and its KV is 24,576 B/token, so 8,192 tokens is 24,576 × 8,192 = 192 MiB (derived). Beam 1.0 501B-A23B is 501.3B total / 23.4B active, 8 of 256 experts and no shared expert. The record's BF16 count is 501,203,374,528 parameters, so a 4-bit packing is 501,203,374,528 / 2 = 250,601,687,264 bytes = 233.39 GiB (derived) — past every priced row except the 512 GB Apple M3 Ultra (Mac Studio, 2025), and past the 48 GB the 70B column tests. Its growing KV is only the 9 full-attention layers, 36,864 B/token, which at 8,192 tokens is 36,864 × 8,192 = 288 MiB (derived); the record's own note puts the 43 sliding layers at a fixed 172.5 MiB that does not grow with context. The active-byte read is the other way around: 23.4 / 20 = 1.17× (derived) the 20 GB the ceiling divides by, so for this checkpoint the dense-equivalent ceiling overstates the rate, where for the 3B-active models it understates it.

• ProMoE: Fast MoE-based LLM Serving using Proactive Caching is why a card the fit flag marks - can still serve the MoE, and it does not change a rank. With routed experts in host RAM the miss is only known when the expert GEMM is about to run, so the PCIe fetch sits on the critical path; the paper's fix is to predict the next layer's experts from the current hidden state and copy them while the current layer computes, then preempt with the real gate result. Every number in the record is the authors' own, on one NVIDIA GeForce RTX 4090 (rank 24, already Y/Y/- at 24 GB) at batch 1 over PCIe 4.0: decode 2.07× average and 5.02× best against transformers with an LRU expert cache, and only 1.34× average against llama.cpp, because a faster GPU-side MoE kernel leaves less idle compute to hide the copy behind. The same record says the gain falls further as the batch grows (batch 4 cuts the decode speedup from 2.28× to 1.59×). That is a software multiplier on an offload path, not a bandwidth or a price, so it does not move a ceiling, a fit flag, or the $/tok-s order. Ranks 1 and 2 stay -/-/-.

Refresh note (docs/fit-matrix-oct14, Oct-18, prior pass). Re-ran python tools/fit_matrix_35b.py on local main e525b48 (230 accelerator records, 218 supply records, 4,322 benchmarks — #249 through #255 on top of the #248 matrix, which was itself stamped on d72800f). The selection rule still yields 159 qualifying / 43 priced / 116 unpriced (69 of the unpriced fit 35B-A3B, 47 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match the table already below, so the ranked table does not move and no rank is renumbered. None of the seven commits adds an accelerator or a purchase price. What they add is model geometry, and one technique record that explains why a card under 24 GB can still serve the MoE the fit flag marks -.

• The 20 GB column is now the named checkpoint, not only the brief. Ornith-1.5-35B-A3B is 35.952B total / ~3B active, qwen3_5_moe, 256 experts with 8 routed, and its KV is 20,480 B/token — the same geometry as Qwen3.5-35B-A3B (35B / 3B, 20,480 B/token). BF16 of that checkpoint is 71,903,645,408 bytes, which is 71,903,645,408 / 1,073,741,824 = 66.97 GiB (derived). A 4-bit packing of the same count is about a quarter of that, 66.97 / 4 = 16.74 GiB (derived), inside the 20 GB the ceiling divides by. The 4 GB headroom at 8,192 tokens is 20,480 × 8,192 = 167,772,160 bytes = 160 MiB (derived), so the 4 GB the fit flag reserves is runtime and the OS, not the KV. Nothing in the new record changes the 24 GB threshold, and there is no benchmark of it, so it does not enter §4b.

• Three more records sit in the same ~3–4B-active class and none of them is the 20 GB file. Kolibri-1 is 78.103B / 3.458B active, KV 20,480 B/token, but 78.103 × 2 = 156.2 GB of BF16 (derived, one-pass 2 bytes/param) — a resident Q4 of that is closer to 40 GB than to 20 GB, so it is a 70B-column problem, not a 35B one. Xing4.0-29B-A4B is 31.215B / 4B active with KV 46,080 B/token, more than double the 20,480 of the named MoE, so its context headroom is the thing that moves, not the weight bar. Mellum2.1 Thinking 12B A2.5B is 12.15B / 2.5B active and the record's own note puts the BF16 checkpoint at about 24 GB; a Q4 of it is a small-model file and does not use the 20 GB denominator.

• The dense 27B sibling and the two Flash-class records do not move a byte assumption either. Qwen3.8-27B is dense, 27.781B, KV 65,536 B/token: 27.781 × 0.5 = 13.89 GB at 4-bit (derived), inside the 16 GB the 27B column divides by. MiMo-V2.6-Flash-RL is 309B / 15B active (KV 23,040 B/token) and GLM-5.3-Flash is 320B / 18B active (KV 11,264 B/token, geometry filled by #251). Both are expert-offload machines, not single-device 24 GB fits, so neither gets a fit flag here. Ornith-1.5-9B is a 9.653B dense model and is out of scope.

• #255 is why a - on 35B is not the same as "cannot run it," and it does not change a rank. MoE-Infinity: Efficient MoE Inference on Personal Machines with Sparsity-Aware Expert Cache and Sparsity-aware expert cache: request-level activation trace driving prefetch and eviction under host-RAM expert offload record a host-RAM expert cache: attention, shared experts and the KV stay resident, and a request-scoped activation trace picks which routed expert to DMA over PCIe. The authors' own range is 3.1–16.7× against other offload systems (abstract and Table 1) and 2.7–13.7× in the conclusion, with PCIe 4.0 stated as both 32 GB/s (Appendix B.2) and 24 GB/s (Table 1 caption) — vendor-paper figures, not a measurement in this corpus, and explicitly not a speedup against a fully resident GPU. A Mixtral-shaped MoE only gains 1.4×. That is the mechanism behind the sentence in §2 that the 20 GB ceiling is a conservative floor for MoE: the bytes moved per token can be the active experts, not the resident file. It does not add a tok/s, a price, or a Y, so NVIDIA Tesla P100 SXM2 16GB (GP100) and Intel Arc B580 stay -/-/- at ranks 1 and 2.

• #252 is a kernel constraint on two priced rows, not a new ceiling. TT-Metal Flash-Decode: split the KV cache across Tensix cores, reduce over the NoC says the TT-Metal decode SDPA splits KV across Tensix cores and only runs multi-query attention on a single token. Tenstorrent Wormhole n300 (rank 27, 24 GB, 576 GB/s, Y/Y/-, ceiling 576/20 = 28.8 tok/s (derived), $1,449, $50.31/tok-s (derived)) and Tenstorrent Wormhole Tensix (n150 / n300) (rank 33, 12 GB, -/-/-) are the parts that constraint lands on. Most models since Llama-2 are grouped-query, including the named 35B-A3B (2 KV heads, 16 query heads). The fit flag is unchanged because it is a byte test; the fast decode path does not exist yet for that attention shape.

Refresh note (docs/fit-matrix-oct14, Oct-17, this pass). Re-ran python tools/fit_matrix_35b.py on local main d72800f (230 accelerator records, 218 supply records, 4,305 benchmarks — #246 on top of 841d639). The selection rule still yields 159 qualifying / 43 priced / 116 unpriced (69 of the unpriced fit 35B-A3B, 47 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. #246 adds no accelerator and no purchase price. What it adds is the first Qwen3.5-35B-A3B and Qwen3.5-27B llama.cpp cells for the W7900 and a second protocol for the R9700, plus an MLX long-decode scorecard on the rank-41 Mac Studio.

• R9700 stays unpriced, and a second Q4 of the named MoE clears the same floor the earlier cell did. AMD Radeon AI PRO R9700 stays 32 GB / 640 GB/s, Y/Y/- (§4). Ceilings: 640/20 = 32.0, 640/16 = 40.0, 640/40 = 16.0 tok/s (derived). llama.cpp, batch 1, one request, ~2.9K prompt, unsloth UD-Q4_K_M (21 GB on disk), decodes at 133.0 on AMDVLK, 112.0 on RADV and 68.8 on ROCm (R9700 llama.cpp AMDVLK Vulkan: Qwen3.5-35B-A3B decode at 2.9K prompt (u/neuromacmd), R9700 llama.cpp RADV: Qwen3.5-35B-A3B decode at 2.9K prompt (u/neuromacmd, commit 48cda24), R9700 llama.cpp ROCm: Qwen3.5-35B-A3B decode at 2.9K prompt (u/neuromacmd)). 133.0/32.0 = 4.16×, 112.0/32.0 = 3.50×, 68.8/32.0 = 2.15× (derived). Prefill on the same three backends is 1,030, 2,987 and 1,190 (R9700 llama.cpp AMDVLK Vulkan: Qwen3.5-35B-A3B prefill at 2.9K prompt (u/neuromacmd), R9700 llama.cpp RADV: Qwen3.5-35B-A3B prefill at 2.9K prompt (u/neuromacmd, commit 48cda24), R9700 llama.cpp ROCm: Qwen3.5-35B-A3B prefill at 2.9K prompt (u/neuromacmd)); 2987/1030 = 2.90× and 2987/1190 = 2.51× (derived) — RADV leads prefill and AMDVLK leads decode, so the three are not averaged. Set against the §4b row already on file, 133.0/114.19 = 1.16× the Vulkan server-endpoint cell and 112.0/114.19 = 0.981× it (derived); 68.8/76.43 = 0.90× the ROCm 10.0 sibling (derived). Different harness, different Q4 file (Q4_K_M vs UD-Q4_K_XL), same card, and the RADV pair agrees to about 2%. The new AMDVLK cell is the faster of the two and is the one §4b now quotes; the 114.19 row stays. The dense sibling on the same machine is under its own floor again: Qwen3.5-27B Q4_K_M (16 GB) decodes at 30.6 / 30.4 / 25.2 (R9700 llama.cpp AMDVLK Vulkan: Qwen3.5-27B decode at 2.9K prompt (u/neuromacmd), R9700 llama.cpp RADV: Qwen3.5-27B decode at 2.9K prompt (u/neuromacmd, commit 48cda24), R9700 llama.cpp ROCm: Qwen3.5-27B decode at 2.9K prompt (u/neuromacmd)), 30.6/40.0 = 0.77×, 30.4/40.0 = 0.76×, 25.2/40.0 = 0.63× (derived). 133.0/30.6 = 4.35× (derived) on one card, two models, and again the MoE is the one above the floor. No street price, so §3 does not move.

• W7900 fits all three workloads and was missing a measured cell; it now has one, and it is still unpriced. AMD Radeon PRO W7900 stays 48 GB / 864 GB/s, Y/Y/Y (§4). Ceilings: 864/20 = 43.2, 864/16 = 54.0, 864/40 = 21.6 tok/s (derived). The same llama.cpp protocol decodes the 21 GB Q4_K_M at 123.7 AMDVLK, 114.3 RADV and 78.9 ROCm (W7900 llama.cpp AMDVLK Vulkan: Qwen3.5-35B-A3B decode at 2.9K prompt (u/neuromacmd), W7900 llama.cpp RADV: Qwen3.5-35B-A3B decode at 2.9K prompt (u/neuromacmd, commit 48cda24), W7900 llama.cpp ROCm: Qwen3.5-35B-A3B decode at 2.9K prompt (u/neuromacmd)). 123.7/43.2 = 2.86×, 114.3/43.2 = 2.65×, 78.9/43.2 = 1.83× (derived). Prefill is 948 / 2,326 / 1,001 (W7900 llama.cpp AMDVLK Vulkan: Qwen3.5-35B-A3B prefill at 2.9K prompt (u/neuromacmd), W7900 llama.cpp RADV: Qwen3.5-35B-A3B prefill at 2.9K prompt (u/neuromacmd, commit 48cda24), W7900 llama.cpp ROCm: Qwen3.5-35B-A3B prefill at 2.9K prompt (u/neuromacmd)); 2326/948 = 2.45× and 2326/1001 = 2.32× (derived) — same backend split as the R9700, not averaged. The dense 27B is under its floor on every backend: 31.8 / 31.8 / 24.4 (W7900 llama.cpp AMDVLK Vulkan: Qwen3.5-27B decode at 2.9K prompt (u/neuromacmd), W7900 llama.cpp RADV: Qwen3.5-27B decode at 2.9K prompt (u/neuromacmd, commit 48cda24), W7900 llama.cpp ROCm: Qwen3.5-27B decode at 2.9K prompt (u/neuromacmd)), 31.8/54.0 = 0.59× and 24.4/54.0 = 0.45× (derived). 123.7/31.8 = 3.89× (derived). AMDVLK on the 48 GB card versus AMDVLK on the 32 GB card is 123.7/133.0 = 0.93× (derived) — the wider bus (864/640 = 1.35×, (derived)) does not show up in the MoE decode, which is what the ~3B-active caveat in §2 predicts. No purchase price, so no $/tok-s and nothing enters §3. The AMDVLK cell is now in §4b.

• The two-card sweep is a combined rate, and it is slower than either card alone. W7900 + R9700, tensor-split, AMDVLK, same 21 GB file, decodes at 82.1 at 8,824 tokens of context and 34.3 at 215,917 (W7900+R9700 llama.cpp AMDVLK Vulkan: Qwen3.5-35B-A3B decode at 8824 context (u/neuromacmd), W7900+R9700 llama.cpp AMDVLK Vulkan: Qwen3.5-35B-A3B decode at 215917 context (u/neuromacmd)). The record says the figure covers both cards and the split ratio is unpublished, so it is not divided by 2 and it is not a per-card rate. Adding the two ceilings is 32.0+43.2 = 75.2 tok/s (derived); 82.1/75.2 = 1.09× and 34.3/75.2 = 0.46× (derived). The ROCm sibling at the short depth is 74.8 (W7900+R9700 llama.cpp ROCm: Qwen3.5-35B-A3B decode at 8824 context (u/neuromacmd)), 74.8/75.2 = 0.99× (derived). 82.1/133.0 = 0.62× the single R9700 and 82.1/123.7 = 0.66× the single W7900 (derived) — splitting this MoE across the two cards costs decode at every depth the sweep prints. The short cell is inside 10% of a summed floor and the long one is under it; neither replaces the single-card §4b rows, and neither is a price.

• M3 Ultra stays rank 41, and the new long-decode cell is the named MoE on a 256 GB machine the $9,000 sticker is not. Apple M3 Ultra (Mac Studio, 2025) stays 512 GB / 819 GB/s, Y/Y/Y, $9,000 (eBay used Mac Studio M3 Ultra 512GB (Oct 2026)), $219.78/(tok/s) (derived). The record's bandwidth is 819 GB/s for both the 60-core and the 80-core SKU, so the ceiling does not change with the core count: 819/20 = 40.95, rounded to the 41.0 the table prints; 819/16 = 51.2, 819/40 = 20.5 (derived). rapid-mlx on a 256 GB M3 Ultra decodes Qwen3.5-35B-A3B 4-bit at 95.2 tok/s long and Qwen3.5-27B 4-bit at 37.7 (M3 Ultra 256GB rapid-mlx: Qwen3.5-35B-A3B 4bit long decode, M3 Ultra 256GB rapid-mlx: Qwen3.5-27B 4bit long decode); cold TTFT is 322 ms and 453 ms (M3 Ultra 256GB rapid-mlx: Qwen3.5-35B-A3B 4bit cold TTFT, M3 Ultra 256GB rapid-mlx: Qwen3.5-27B 4bit cold TTFT). 95.2/41.0 = 2.32× the 35B floor and 37.7/51.2 = 0.74× the 27B floor (derived). 95.2/37.7 = 2.52× (derived) — same machine, and again the MoE is the one above its floor. The 8-bit sibling of the same MoE is 80.0 (M3 Ultra 256GB rapid-mlx: Qwen3.5-35B-A3B 8bit long decode), 80.0/41.0 = 1.95× and 80.0/95.2 = 0.84× (derived). The token count behind "long" is unpublished, so this is not a tg128 cell and it is not averaged with anything. The scorecard machine is 256 GB; the rank-41 price is the 512 GB Studio. Pairing $9,000 with 95.2 tok/s would be the cross-capacity join §6 already refuses, so $9,000/95.2 = $94.54 is not a re-rank (derived, and not used). The 256 GB row at $6,250 is the one the capacity matches (eBay used Mac Studio M3 Ultra 256GB (Oct 2026)), and it stays unjoined for the reason already in §6. Qwen3-Coder-Next 4-bit, the Next-class file, decodes at 73.5 (M3 Ultra 256GB rapid-mlx: Qwen3-Coder-Next 4bit long decode), 73.5/41.0 = 1.79× (derived); the 6-bit sibling is 65.6, 65.6/41.0 = 1.60× and 65.6/73.5 = 0.89× (derived) (M3 Ultra 256GB rapid-mlx: Qwen3-Coder-Next 6bit long decode). GLM-4.7-Flash 8-bit is 57.2, 57.2/41.0 = 1.40× (derived) (M3 Ultra 256GB rapid-mlx: GLM-4.7-Flash 8bit long decode). None of those is the 20 GB denominator, so only the 4-bit 35B-A3B cell enters §4b, and the rank does not move.

Refresh note (docs/fit-matrix-oct14, Oct-17, this pass). Re-ran python tools/fit_matrix_35b.py on local main 267e65c (230 accelerator records, 218 supply records, 4,203 benchmarks — #242 and #243, the same Xeon 6 MLPerf v6.1 ingest, on top of 3c5cf8e). The selection rule still yields 159 qualifying / 43 priced / 116 unpriced (69 of the unpriced fit 35B-A3B, 47 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. No new supply record, so no new price. What landed is four Granite Rapids host CPUs plus sixteen llama3.1-8b INT8 aggregate rows and one whisper samples/s row. None of the four enters the 159.

• The four Xeon 6 records are the same shape as the unpriced Genoa siblings, and they stay out for the same reason. Intel Xeon 6787P (Granite Rapids-SP, 86-core, 8-channel DDR5 host CPU) and Intel Xeon 6788P (Granite Rapids-SP, 86-core, 8-channel DDR5 host CPU) are 86-core, 8-channel, 409.6 GB/s (record). Intel Xeon 6972P (Granite Rapids-AP, 96-core, 12-channel DDR5 host CPU) (96-core) and Intel Xeon 6980P (Granite Rapids-AP, 128-core, 12-channel DDR5 host CPU) (128-core) are 12-channel, 614.4 GB/s (record). All four set unified_memory: true and leave vram_gb null — capacity is how many RDIMMs are fitted, and ARK's max is 3 TB, which is not a configured pool. effective_vram returns None unless the record has a vram_gb or a UNIFIED_MEM_GB override, and the only overrides in the tool are the Strix Halo 128 GB pool and the Genoa 384 GB self-build. There is no Xeon supply record to read a configured capacity off, and no purchase price at <= $3,000, so the 8 GB gate drops them before the bandwidth floor is even tested. The same drop already applies to AMD EPYC 9454P (Genoa, 48-core, 12-channel DDR5 host CPU) (460.8 GB/s) and AMD EPYC 9334 (Genoa, 32-core, dual-socket, 12-channel DDR5 host CPU) (921.6 GB/s): bandwidth alone does not qualify a host CPU whose pool is unpublished. Inventing a 512 GB or 1 TB fit flag so the 614.4 GB/s parts could show Y/Y/Y would pair a ceiling with a machine the corpus does not price. Not done.
• The bandwidth arithmetic, stated and not ranked. If a configured pool large enough for the 24 GB bar were on file, the 35B ceilings would be 409.6/20 = 20.48 tok/s on the 8-channel parts and 614.4/20 = 30.72 tok/s on the 12-channel parts (derived). The 27B column would be 409.6/16 = 25.6 and 614.4/16 = 38.4 (derived); the 70B column 409.6/40 = 10.24 and 614.4/40 = 15.36 (derived). Set against the one host CPU this doc does rank, AMD EPYC 9554 (Genoa, 64-core, 12-channel DDR5 host CPU) at 460.8 GB/s and a 23.0 tok/s ceiling, the 12-channel Xeon bus is 614.4/460.8 = 1.33× that floor (derived) and the 8-channel bus is 409.6/460.8 = 0.89× (derived). None of those ceilings is a row: there is no pool and no sticker, so there is no $/tok-s and nothing enters §3.
• The new cells are node-level INT8 aggregates of an 8B, not a 35B-A3B decode. Intel's own 2S 6980P box does llama3.1-8b INT8 at 1,916.74 tok/s Offline and 1,139.51 Server (MLPerf v6.1 Intel 2x Xeon 6980P: llama3.1-8b INT8 Offline aggregate tok/s, MLPerf v6.1 Intel 2x Xeon 6980P: llama3.1-8b INT8 Server aggregate tok/s). Supermicro's 2S 6980P is 1,833.97 / 1,101.19 (MLPerf v6.1 Supermicro 2x Xeon 6980P: llama3.1-8b INT8 Offline aggregate tok/s, MLPerf v6.1 Supermicro 2x Xeon 6980P: llama3.1-8b INT8 Server aggregate tok/s). 1833.97/1916.74 = 0.957× (derived) — about 4% apart, two submitters, not averaged. Red Hat's 2S 6972P is 1,507.21 / 644.788 (MLPerf v6.1 Red Hat 2x Xeon 6972P: llama3.1-8b INT8 Offline aggregate tok/s, MLPerf v6.1 Red Hat 2x Xeon 6972P: llama3.1-8b INT8 Server aggregate tok/s); 1507.21/1916.74 = 0.786× (derived) the 128-core box at the same 12-channel width. Dell's 2S 6787P (vLLM, FP8 KV, 16×64 GB DDR5-6400) is 1,227.99 / 516.012 (MLPerf v6.1 Dell R770 2x Xeon 6787P: llama3.1-8b INT8 Offline aggregate tok/s, MLPerf v6.1 Dell R770 2x Xeon 6787P: llama3.1-8b INT8 Server aggregate tok/s); Supermicro's 2S 6787P is 1,189.89 (MLPerf v6.1 Supermicro 2x Xeon 6787P: llama3.1-8b INT8 Offline aggregate tok/s), 1189.89/1227.99 = 0.969× (derived). Quanta's 2S 6788P is 1,170.51 / 516.012 (MLPerf v6.1 Quanta 2x Xeon 6788P: llama3.1-8b INT8 Offline aggregate tok/s, MLPerf v6.1 Quanta 2x Xeon 6788P: llama3.1-8b INT8 Server aggregate tok/s). The 4-socket 6788P rows do not double the 2-socket ones: Cisco Offline 2,068.63 and Supermicro Offline 2,063.24 (MLPerf v6.1 Cisco 4x Xeon 6788P: llama3.1-8b INT8 Offline aggregate tok/s, MLPerf v6.1 Supermicro 4x Xeon 6788P: llama3.1-8b INT8 Offline aggregate tok/s), 2068.63/1170.51 = 1.77× (derived) and 2063.24/2068.63 = 0.997× (derived). Server on those 4S boxes is 1,030.45 and 966.241 (MLPerf v6.1 Cisco 4x Xeon 6788P: llama3.1-8b INT8 Server aggregate tok/s, MLPerf v6.1 Supermicro 4x Xeon 6788P: llama3.1-8b INT8 Server aggregate tok/s). Dividing any of these by the 20 GB ceiling is the wrong comparison twice over: the metric is tps_aggregate across 2 or 4 sockets, and the model is llama3.1-8b INT8, not a Q4 35B-A3B. They do not enter §4b.
• The whisper row is not tokens. Dell's 2S 6980P whisper INT8 Offline is 2,275.93 samples/s (MLPerf v6.1 Dell 2x Xeon 6980P: whisper INT8 Offline aggregate samples/s), stored under tps_aggregate because that is the aggregate-throughput member of the metric enum. The unit field says samples/s. It is not divided into a tok/s ceiling.

Refresh note (docs/fit-matrix-oct14, Oct-17, prior pass). Re-ran python tools/fit_matrix_35b.py on local main 3c5cf8e (226 accelerator records, 218 supply records, 4,187 benchmarks — #240 on top of 80aee58). The selection rule still yields 159 qualifying / 43 priced / 116 unpriced (69 of the unpriced fit 35B-A3B, 47 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. #240 adds no accelerator and no purchase price. What it adds is the Next/Flash class itself, at Q4 and with MTP, on the rank-37 box, plus two larger Flash MoEs at Q2 and a 1.2B calibration on a card §4 already lists.

• Strix Halo 8060S stays rank 37, and the named Next/Flash Q4 sits just above the 20 GB floor. AMD Radeon 8060S (Ryzen AI Max+ integrated GPU) stays 128 GB unified / 256 GB/s, Y/Y/Y, $1,900 → $148.44/tok-s (derived) (New Minisforum MS-S1 MAX 128GB (Strix Halo)). The 35B ceiling is 256/20 = 12.8 tok/s (derived). Qwen3.8-Flash-Next is 180B total / 6B active, not the 35B-A3B and not the 27B. DwarfStar on ROCm 10.0, speculative decoding off, decodes the Q4 at 19.5233 tok/s at depth 0 and 19.3232 at depth 65536 (Strix Halo 8060S DwarfStar ROCm 10.0 Qwen3.8-Flash-Next Q4 depth-0 decode, Strix Halo 8060S DwarfStar ROCm 10.0 Qwen3.8-Flash-Next Q4 64k-depth decode); prefill at depth 0 is 504.5011 (Strix Halo 8060S DwarfStar ROCm 10.0 Qwen3.8-Flash-Next Q4 depth-0 prefill). 19.5233/12.8 = 1.53× and 19.3232/12.8 = 1.51× the 35B floor (derived). 19.3232/19.5233 = 0.990× (derived) — filling 64k of KV costs about a percent of the decode. The llama.cpp ROCm 10.0 sibling on UD-IQ4_XS is 20.5757 at depth 0 and 12.2638 at depth 65536 (Strix Halo 8060S ROCm 10.0 Qwen3.8-Flash-Next UD-IQ4_XS depth-0 decode, Strix Halo 8060S ROCm 10.0 Qwen3.8-Flash-Next UD-IQ4_XS 64k-depth decode); prefill 470.2468 (Strix Halo 8060S ROCm 10.0 Qwen3.8-Flash-Next UD-IQ4_XS depth-0 prefill). 20.5757/12.8 = 1.61× and 12.2638/12.8 = 0.96× (derived). 20.5757/19.5233 = 1.05× at depth 0 (derived), and 12.2638/20.5757 = 0.596× (derived) — the same file class loses two fifths of its decode by 64k on llama.cpp and essentially none on DwarfStar. Different engine, different quant label, so the two depth-0 numbers are not averaged. $1,900/19.5233 = $97.32 per measured tok/s (derived) and $1,900/20.5757 = $92.34 (derived). Both price a measured Q4 of a 180B/6B model, not the 12.8 ceiling the $148.44 rank uses, so the rank does not move. Neither cell is the 35B-A3B §4b ranks, so neither enters that table.
• MTP on the same model roughly doubles the decode, and it is still not the floor. The retained-PM4 llama-server build (shared MTP head, n-gram draft, draft-n-max 3) decodes Qwen3.8-Flash-Next at 43.9 tok/s and prefills pp2048 at 1,207.0 (Strix Halo retained-PM4 llama-server Qwen3.8-Flash-Next MTP decode, Strix Halo retained-PM4 llama-server Qwen3.8-Flash-Next MTP pp2048). 43.9/12.8 = 3.43× the 35B floor (derived). 43.9/19.5233 = 2.25× the DwarfStar Q4 baseline and 43.9/20.5757 = 2.13× the llama.cpp IQ4_XS cell (derived) — the record's own note calls it roughly double the 19.5–20.6 tok/s no-speculation baseline, and the arithmetic agrees. Speculation is a second draft model, not a bandwidth change, so 43.9 does not replace either baseline and does not enter §4b. $1,900/43.9 = $43.28 per measured tok/s (derived).
• The two larger Flash MoEs are the wrong byte count, and one of them is not fully resident. DeepSeek-V4-Flash-0731 UD-IQ2_XXS (DeepSeek-V4-Flash, 284B / 13B active) decodes at 13.3312 at depth 0 and 10.4007 at depth 65536 (Strix Halo 8060S ROCm 10.0 DeepSeek-V4-Flash-0731 UD-IQ2_XXS depth-0 decode, Strix Halo 8060S ROCm 10.0 DeepSeek-V4-Flash-0731 UD-IQ2_XXS 64k-depth decode); prefill 216.9735 (Strix Halo 8060S ROCm 10.0 DeepSeek-V4-Flash-0731 UD-IQ2_XXS depth-0 prefill). 13.3312/12.8 = 1.04× and 10.4007/12.8 = 0.81× (derived). 10.4007/13.3312 = 0.78× (derived). IQ2 is not the Q4 the 20 GB denominator assumes. DeepSeek-V4.1-Flash Q2 (DeepSeek-V4.1-Flash-1017, 552B backbone, 8B active at prefill) on DwarfStar decodes at 14.7622 and 14.3066 (Strix Halo (gorgon-halo) DwarfStar ROCm 10.0 DeepSeek-V4.1-Flash Q2 depth-0 decode, Strix Halo (gorgon-halo) DwarfStar ROCm 10.0 DeepSeek-V4.1-Flash Q2 64k-depth decode); prefill 240.8634 (Strix Halo (gorgon-halo) DwarfStar ROCm 10.0 DeepSeek-V4.1-Flash Q2 depth-0 prefill). 14.7622/12.8 = 1.15× and 14.3066/12.8 = 1.12× (derived). 14.3066/14.7622 = 0.969× (derived). The prefill record says the Q2 main weights are 151.77 GiB plus 188.83 GiB of Engram tables streamed from disk, so this is not a resident run and the Y on the 128 GB pool does not cover it. GLM-5.3-Flash Q2 (GLM-5.3-Flash, 320B / 18B active) decodes at 11.9241 and 9.159 (Strix Halo (gorgon-halo) DwarfStar ROCm 10.0 GLM-5.3-Flash Q2 depth-0 decode, Strix Halo (gorgon-halo) DwarfStar ROCm 10.0 GLM-5.3-Flash Q2 64k-depth decode); prefill 227.1627 (Strix Halo (gorgon-halo) DwarfStar ROCm 10.0 GLM-5.3-Flash Q2 depth-0 prefill). 11.9241/12.8 = 0.93× and 9.159/12.8 = 0.72× (derived). 9.159/11.9241 = 0.77× (derived). A Q2 of a 320B file is under the dense-equivalent floor on this bus. None of the three enters §4b, and none has a street price of its own, so §3 does not move. $1,900/13.3312 = $142.52, $1,900/14.7622 = $128.71, $1,900/11.9241 = $159.34 per measured tok/s (derived) — all three price the wrong byte count.
• R9700, still unpriced, and the new cell is a 1.2B, not the 35B-A3B. AMD Radeon AI PRO R9700 stays 32 GB / 640 GB/s, Y/Y/- (§4). Ceiling 640/20 = 32.0 tok/s (derived). LFM2.5-1.2B-Instruct BF16 on ROCm 10.0 decodes at 184.234 at depth 0 and 155.8044 at depth 65536 (R9700 ROCm 10.0 LFM2.5-1.2B-Instruct BF16 depth-0 decode, R9700 ROCm 10.0 LFM2.5-1.2B-Instruct BF16 64k-depth decode); prefill 20,717.405 (R9700 ROCm 10.0 LFM2.5-1.2B-Instruct BF16 depth-0 prefill). 184.234/32.0 = 5.76× and 155.8044/32.0 = 4.87× the 35B floor (derived). 155.8044/184.234 = 0.846× (derived). The same file on the Strix Halo Vulkan path already cited is 74.49 (Strix Halo 8060S Vulkan RADV LFM2.5-1.2B-Instruct BF16 depth-0 decode); 184.234/74.49 = 2.47× (derived) — different silicon, different backend, a 1.2B BF16 file, so the multiple is not a speedup and it does not enter §4b. A 1.2B is the wrong byte count for the 20 GB denominator, same caveat the prior note already gives for this model on the 8060S.

Refresh note (docs/fit-matrix-oct14, Oct-17, this pass). Re-ran python tools/fit_matrix_35b.py on local main 80aee58 (226 accelerator records, 218 supply records, 4,167 benchmarks — #238 and #237 on top of 2d884b8). The selection rule still yields 159 qualifying / 43 priced / 116 unpriced (69 of the unpriced fit 35B-A3B, 47 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. Neither commit adds an accelerator or a purchase price.

• LLaDA2.2-mini is a model, not a row. LLaDA2.2-mini is a 16.256B diffusion MoE with 1.4B active and kv_cache_bytes_per_token 0. The BF16 checkpoint is about 32 GB, which clears the 24 GB 35B-A3B bar as a weight footprint and is not the 20 GB Q4 denominator this doc divides by. The card publishes no absolute tok/s for the mini, so there is no measured cell to set against a ceiling and nothing enters §3 or §4b.
• The OpenRouter Llama 3.3 70B re-read is a hosted service, not a card. #237 moves the Groq row on OpenRouter hosted API router from 0.51 s / 198 tok/s to 0.52 s / 191 tok/s. List prices did not move. A hosted tok/s is not memory_bandwidth_gbps / 20, so it does not re-rank §3. The stretch goal this doc prices is still Llama-3.3-70B Q4 resident on a single device at 48 GB.

Refresh note (docs/fit-matrix-oct14, Oct-17, prior pass). Re-ran python tools/fit_matrix_35b.py on local main 2d884b8 (226 accelerator records, 218 supply records, 4,167 benchmarks — #235 and #234 on top of a641583). The selection rule still yields 159 qualifying / 43 priced / 116 unpriced (69 of the unpriced fit 35B-A3B, 47 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. Neither commit adds an accelerator or a purchase price. #234 files a vendor headline for the Hailo-10H (9.9 tok/s on Llama-3.2-1B, Hailo-10H Pi AI HAT+ 2 Llama-3.2-1B compiled-model TPS); that part is under the 8 GB gate and was never a row. #235 is the one that lands on parts this doc already lists.

• M4 Max stays rank 40, and the new llama.cpp cells sit next to the MLX ones already in §4b. Apple M4 Max (MacBook Pro 14-inch, 2024) stays 128 GB / 546 GB/s, Y/Y/Y, $4,700 (eBay used MacBook Pro M4 Max 16in 128GB (Oct 2026)), $172.16/(tok/s) (derived). Ceilings: 546/20 = 27.3, 546/16 = 34.125, 546/40 = 13.65 tok/s (derived). llama.cpp b8353, Metal, llama-bench pp512/tg128, batch 1, decodes Qwen3.5-35B-A3B Q4_K_M at 91.47 tok/s at depth 0 and 82.31 at depth 8192 (M4 Max 128GB llama.cpp b8353 Qwen3.5-35B-A3B Q4_K_M tg128 depth 0, M4 Max 128GB llama.cpp b8353 Qwen3.5-35B-A3B Q4_K_M tg128 depth 8192); prefill is 1,125.86 and 903.14 (M4 Max 128GB llama.cpp b8353 Qwen3.5-35B-A3B Q4_K_M pp512 depth 0, M4 Max 128GB llama.cpp b8353 Qwen3.5-35B-A3B Q4_K_M pp512 depth 8192). 91.47/27.3 = 3.35× and 82.31/27.3 = 3.02× the 35B floor (derived). 82.31/91.47 = 0.900× (derived) — filling 8192 of KV costs about a tenth of the decode, and 903.14/1125.86 = 0.802× of the prefill. Set against the two rows already in §4b, 91.47/113.33 = 0.807× the MLX 4-bit cell and 91.47/92.09 = 0.993× the community MLX Q4 cell (derived). The two MLX numbers and this llama.cpp one are different engines and different files, so they are not averaged; the new cell is the one within 1% of the lower MLX reading. The dense sibling on the same machine is under its own floor: Qwen3.5-27B Q4_K_M decodes at 24.18 at depth 0 and 22.64 at depth 8192 (M4 Max 128GB llama.cpp b8353 Qwen3.5-27B Q4_K_M tg128 depth 0, M4 Max 128GB llama.cpp b8353 Qwen3.5-27B Q4_K_M tg128 depth 8192), 24.18/34.125 = 0.71× and 22.64/34.125 = 0.66× the 27B floor (derived). 91.47/24.18 = 3.78× (derived) on one machine, two models, and again the MoE is the one above the floor. On the $4,700 sticker that is $51.38/measured-tok/s at the depth-0 MoE cell (derived, 4700/91.47) and $194.38 at the dense one (derived, 4700/24.18) — a whole laptop, not a bare card, so neither figure re-ranks §3.

• M1 Max stays unpriced, and the new cells are prompt-set averages rather than llama-bench. Apple M1 Max stays 64 GB / 400 GB/s, Y/Y/Y, no bare-part price (§4). Ceiling: 400/20 = 20.0 tok/s (derived). llama-server b8139, one prompt at a time inside a 4096 context cap, reports generation averages of 58.5 on Qwen3-Coder-30B-A3B IQ4_XS, 43.7 on Nemotron-3-Nano-30B-A3B Q4_K_XL and 36.8 on GLM-4.7-Flash Q4_K_XL (M1 Max 64GB llama-server b8139 Qwen3-Coder-30B-A3B IQ4_XS generation average, M1 Max 64GB llama-server b8139 Nemotron-3-Nano-30B-A3B Q4_K_XL generation average, M1 Max 64GB llama-server b8139 GLM-4.7-Flash Q4_K_XL generation average). 58.5/20.0 = 2.93×, 43.7/20.0 = 2.19×, 36.8/20.0 = 1.84× the 35B floor (derived). The prompt length is unpublished, so none of the three is a pp512/tg128 cell and none is ranked against the llama-bench rows. The spread is the model, not the machine: 58.5/43.7 = 1.34× and 43.7/36.8 = 1.19× (derived) on one author, one prompt set. No street price, so §3 does not move. The row is now in §4b.

Refresh note (docs/fit-matrix-oct14-p232, Oct-17, this pass). Re-ran python tools/fit_matrix_35b.py on local main a641583 (226 accelerator records, 218 supply records, 4,152 benchmarks — #232 on top of 46c83ee). The selection rule still yields 159 qualifying / 43 priced / 116 unpriced (69 of the unpriced fit 35B-A3B, 47 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. #232 adds no accelerator and no purchase price. What it adds is the named model itself, Qwen3.6-35B-A3B UD-Q4_K_XL, at depth 0 and concurrency 1, on two parts this doc already lists (Local LLM Benchmarks speed dataset (Donato Capitella, results.json)). The protocol is a server endpoint (2048-token prompt, 128 generated, TTFT excluded), not llama-bench tg128, so the new cells are not averaged with the llama-bench rows already on file.

• R9700, still unpriced, and the named Q4 MoE clears the 20 GB floor where the dense 27B does not. AMD Radeon AI PRO R9700 stays 32 GB / 640 GB/s, Y/Y/- (§4). Ceilings: 640/20 = 32.0, 640/16 = 40.0, 640/40 = 16.0 tok/s (derived). Vulkan RADV decodes the named file at 114.19 tok/s (R9700 Vulkan RADV Qwen3.6-35B-A3B UD-Q4_K_XL depth-0 decode) and prefills at 3,792.51 (R9700 Vulkan RADV Qwen3.6-35B-A3B UD-Q4_K_XL depth-0 prefill). The same weights on ROCm 10.0 decode at 76.43 (R9700 ROCm 10.0 Qwen3.6-35B-A3B UD-Q4_K_XL depth-0 decode); that run has no prefill sibling. 114.19/32.0 = 3.57× and 76.43/32.0 = 2.39× the 35B floor (derived). 114.19/76.43 = 1.49× (derived) — the backend gap the commit records, not an average. The dense row already in §4 is the other way around: Qwen3.8-27B UD-Q4_K_XL is 30.66 on ROCm and 28.39 on Vulkan (R9700 1-GPU ROCm Qwen3.8-27B UD-Q4_K_XL depth-0 decode, R9700 1-GPU Vulkan RADV Qwen3.8-27B UD-Q4_K_XL depth-0 decode), 30.66/40.0 = 0.77× and 28.39/40.0 = 0.71× the 27B floor (derived). 114.19/30.66 = 3.72× (derived) on one card, two different models, and the MoE is the one above the floor. No street price, so §3 does not move. The row is now in §4b.

• Strix Halo 8060S stays rank 37, and the new depth-0 cell matches the llama-bench one. AMD Radeon 8060S (Ryzen AI Max+ integrated GPU) stays 128 GB unified / 256 GB/s, Y/Y/Y, $1,900 → $148.44/tok-s (derived) (New Minisforum MS-S1 MAX 128GB (Strix Halo)). The 35B ceiling is 256/20 = 12.8 tok/s (derived). Vulkan RADV decodes the same UD-Q4_K_XL file at 60.68 tok/s (Strix Halo 8060S Vulkan RADV Qwen3.6-35B-A3B UD-Q4_K_XL depth-0 decode) and prefills at 1,218.71 (Strix Halo 8060S Vulkan RADV Qwen3.6-35B-A3B UD-Q4_K_XL depth-0 prefill). ROCm 10.0 decodes at 51.26 (Strix Halo 8060S ROCm 10.0 Qwen3.6-35B-A3B UD-Q4_K_XL depth-0 decode). 60.68/12.8 = 4.74× and 51.26/12.8 = 4.00× (derived). 60.68/51.26 = 1.18× (derived) — Vulkan ahead on decode, the same direction the R9700 shows and the opposite of that card's dense-27B pair. The llama-bench tg128 cells already cited are 60.43 on RADV (kyuz0 Strix Halo sweep: Qwen3.6-35B-A3B-UD-Q4_K_XL vulkan_radv decode_tok_s on Radeon 8060S (gfx1151)) and 51.47 on the ROCm 7 nightlies (kyuz0 Strix Halo 8060S Qwen3.6-35B-A3B-UD-Q4_K_XL ROCm 7 nightlies tg128). 60.68/60.43 = 1.004× and 51.26/51.47 = 0.996× (derived). Different harness, same weights, same box class, and the two numbers agree to under half a percent, so neither replaces the other and §4b keeps the 72.35 best-of-20 row. $1,900/60.68 = $31.31 per measured tok/s (derived) and $1,900/51.26 = $37.07 (derived). Both price a measured Q4 of the named model, not the 12.8 ceiling the $148.44 rank uses, so the rank does not move.

• The small dense sibling on the same box is not the 20 GB job. LFM2.5-1.2B-Instruct BF16 decodes at 74.49 and prefills at 2,843.09 (Strix Halo 8060S Vulkan RADV LFM2.5-1.2B-Instruct BF16 depth-0 decode, Strix Halo 8060S Vulkan RADV LFM2.5-1.2B-Instruct BF16 depth-0 prefill). 74.49/12.8 = 5.82× the 35B floor (derived) and 74.49/60.68 = 1.23× the named Q4 MoE on the same backend (derived). A 1.2B BF16 file is the wrong byte count for the 20 GB denominator, so it does not enter §4b.

Refresh note (docs/fit-matrix-oct14-p230, Oct-17, this pass). Re-ran python tools/fit_matrix_35b.py on local main 46c83ee (226 accelerator records, 218 supply records, 4,144 benchmarks — #230 on top of 0947ba3). The selection rule still yields 159 qualifying / 43 priced / 116 unpriced (69 of the unpriced fit 35B-A3B, 47 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. #230 adds no accelerator and no purchase price. What it adds is a 2-card gptj-99 box on a part §4 already lists:

• L40S, still unpriced, and the new number is a 6B FP8 aggregate, not the 35B-A3B job. NVIDIA L40S stays 48 GB / 864 GB/s, Y/Y/Y (§4). The ceilings are 864/20 = 43.2, 864/16 = 54.0, 864/40 = 21.6 tok/s (derived). #230 is Cisco's MLPerf v5.0 closed submission, a UCS X215 M8 with 2× L40S and TensorRT 10.8, on gptj-99 (GPT-J 6B, FP8) (MLPerf Inference v5.0 — Cisco UCS X215 M8 2x L40S gptj-99 submission (raw logs)). Offline is 1,754.97 tok/s aggregate (MLPerf v5.0 Cisco X215 M8 2x L40S: gptj-99 Offline aggregate tokens/s) and Server is 1,745.78 (MLPerf v5.0 Cisco X215 M8 2x L40S: gptj-99 Server aggregate tokens/s). 1745.78/1754.97 = 0.995× (derived) — the Server row sits inside 1% of Offline, which is the record's own reading that the submission is latency-bound near 25 queries/s rather than compute-bound. 1754.97/2 = 877.49 tok/s per card (derived), and the record says that division is not a per-request decode. Set against the single-card llama.cpp Q4_K_M 70B cell already stored, 15.31 tok/s (NVIDIA L40S 48GB llama.cpp cuBLAS: LLaMA-3 70B Q4_K_M tg1024 decode tok/s), 15.31/21.6 = 0.71× the 70B floor (derived) (15.31/21.6 = 0.709). 877.49/15.31 = 57.3× (derived) (877.49/15.31 = 57.31) — two cards, a concurrent closed-division run of a 6B FP8 file, and a different engine, so the multiple is not a speedup and it does not enter §4b. The 8-card llama2-70b-99.9 boxes already on file are a different model: HPE Offline 3,655.89 (MLPerf v5.0 HPE DL380a Gen12 8x L40S: llama2-70b-99.9 Offline aggregate tokens/s) and Dell Offline 3,481.53 (MLPerf v5.0 Dell XE7745 8x L40S: llama2-70b-99.9 Offline aggregate tokens/s). 3655.89/1754.97 = 2.08× (derived) for four times the cards and a 70B workload, which is not a per-card comparison. No street price, so §3 does not move.

Refresh note (docs/fit-matrix-oct14-p228, Oct-17, this pass). Re-ran python tools/fit_matrix_35b.py on local main 5c30835 (226 accelerator records, 218 supply records, 4,142 benchmarks — #228 on top of 3b34129; #227 is a model-record KV fill and moves nothing here). The selection rule still yields 159 qualifying / 43 priced / 116 unpriced (69 of the unpriced fit 35B-A3B, 47 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. #228 adds no accelerator and no purchase price. What it adds is one shared llama.cpp b7083 Vulkan run (Phoronix: llama.cpp Vulkan, Arc B580 vs RX 9000 vs RTX 50 (b7083, Dec 2025)) on three cards the matrix already lists, all of them -/-/-: the rank-2 Arc B580, the unpriced RX 9070, and the unpriced RX 9060 XT. Every cell is Q8_0, geometric mean of N=3, one request, tg128 or pp1024. Q8 moves about twice the bytes the 20 GB Q4 denominator assumes, so none of the ratios below is a Q4 measurement and none enters §4b.

• Arc B580 stays rank 2, and the new dense cells sit under the 20 GB floor. Intel Arc B580 is 12 GB / 456 GB/s, -/-/-, $199.99 → $8.77/tok-s (derived) (Newegg Refreshed (first-party refurbished) Intel Arc B580 12GB tier). The ceiling is 456/20 = 22.8 tok/s (derived). The four dense Q8_0 tg128 cells are DeepSeek-R1-Distill-Llama-8B 13.69 (llama.cpp b7083 Vulkan: DeepSeek-R1-Distill-Llama-8B-Q8_0 decode 128 tok on intel-arc-b580, 13.69/22.8 = 0.60× (derived)), Mistral 7B 14.37 (llama.cpp b7083 Vulkan: Mistral-7B-Instruct-v0.3-Q8_0 decode 128 tok on intel-arc-b580, 0.63×), Qwen3-8B 13.28 (llama.cpp b7083 Vulkan: Qwen3-8B-Q8_0 decode 128 tok on intel-arc-b580, 0.58×) and Tulu3-8B 12.85 (llama.cpp b7083 Vulkan: Llama-3.1-Tulu-3-8B-Q8_0 decode 128 tok on intel-arc-b580, 0.56×). 14.37/12.85 = 1.12× (derived) across the four, so the model is not the variable. $199.99/14.37 = $13.92 per measured tok/s (derived) on the best dense cell — still the wrong byte count for the $8.77 rank, which prices the 20 GB ceiling, and a 12 GB card cannot hold that file. Prefill siblings, not a decode ceiling: 508.54 / 509.09 / 463.79 / 507.15 (llama.cpp b7083 Vulkan: DeepSeek-R1-Distill-Llama-8B-Q8_0 prefill 1024 tok on intel-arc-b580, llama.cpp b7083 Vulkan: Mistral-7B-Instruct-v0.3-Q8_0 prefill 1024 tok on intel-arc-b580, llama.cpp b7083 Vulkan: Qwen3-8B-Q8_0 prefill 1024 tok on intel-arc-b580, llama.cpp b7083 Vulkan: Llama-3.1-Tulu-3-8B-Q8_0 prefill 1024 tok on intel-arc-b580).

• The one MoE cell on the B580 clears the floor, and it is still not the 35B-A3B job. gpt-oss-20b Q8_0 decodes at 26.64 tok/s (llama.cpp b7083 Vulkan: gpt-oss-20b-Q8_0 decode 128 tok on intel-arc-b580), 26.64/22.8 = 1.17× (derived), and 26.64/14.37 = 1.85× the best dense cell on the same card and the same build (derived). The record's own note says the MoE tok/s is not comparable to the dense rows. $199.99/26.64 = $7.51 per measured tok/s (derived). That does not re-rank: the section prices one card against a 20 GB Q4, and this is a 20B-class MoE at Q8_0. Prefill sibling 1,041.18 (llama.cpp b7083 Vulkan: gpt-oss-20b-Q8_0 prefill 1024 tok on intel-arc-b580).

• RX 9070, unpriced, and the new Q8 reading is well under the Q4_0 cell already on file. AMD Radeon RX 9070 is 16 GB / 640 GB/s, -/-/- (§4's does-not-fit set). Ceiling 640/20 = 32.0 tok/s (derived). The dense tg128 cells are 69.23 / 72.72 / 67.8 / 69.23 (llama.cpp b7083 Vulkan: DeepSeek-R1-Distill-Llama-8B-Q8_0 decode 128 tok on amd-radeon-rx-9070, llama.cpp b7083 Vulkan: Mistral-7B-Instruct-v0.3-Q8_0 decode 128 tok on amd-radeon-rx-9070, llama.cpp b7083 Vulkan: Qwen3-8B-Q8_0 decode 128 tok on amd-radeon-rx-9070, llama.cpp b7083 Vulkan: Llama-3.1-Tulu-3-8B-Q8_0 decode 128 tok on amd-radeon-rx-9070) → 2.16× / 2.27× / 2.12× / 2.16× the 32.0 floor (derived) (72.72/32.0 = 2.273). The Vulkan #10879 Llama-2-7B Q4_0 cell already cited for this card is 119.51 (llama.cpp Vulkan on amd-radeon-rx-9070: Llama-2-7B Q4_0 FA-on tg128 decode tok/s (scoreboard #10879)), 119.51/32.0 = 3.73× (derived), and 119.51/72.72 = 1.64× (derived). Q4_0 versus Q8_0 on a 7–8B file, different build, so the gap is the quant and not a second protocol to average. gpt-oss-20b Q8_0 is 170.89 (llama.cpp b7083 Vulkan: gpt-oss-20b-Q8_0 decode 128 tok on amd-radeon-rx-9070), 170.89/32.0 = 5.34× and 170.89/72.72 = 2.35× the best dense cell (derived). Prefill siblings 2,821.94 / 2,829.08 / 2,754.84 / 2,823.06 and 3,531.8 for the MoE (llama.cpp b7083 Vulkan: DeepSeek-R1-Distill-Llama-8B-Q8_0 prefill 1024 tok on amd-radeon-rx-9070, llama.cpp b7083 Vulkan: Mistral-7B-Instruct-v0.3-Q8_0 prefill 1024 tok on amd-radeon-rx-9070, llama.cpp b7083 Vulkan: Qwen3-8B-Q8_0 prefill 1024 tok on amd-radeon-rx-9070, llama.cpp b7083 Vulkan: Llama-3.1-Tulu-3-8B-Q8_0 prefill 1024 tok on amd-radeon-rx-9070, llama.cpp b7083 Vulkan: gpt-oss-20b-Q8_0 prefill 1024 tok on amd-radeon-rx-9070). No purchase price, so §3 does not move.

• RX 9060 XT tracks the 9070 at half the bandwidth, and the new Q8 cell is half the Q4_0 one. AMD Radeon RX 9060 XT 16GB is 16 GB / 320 GB/s, -/-/-. Ceiling 320/20 = 16.0 tok/s (derived). Dense tg128 is 37.04 / 38.95 / 36.53 / 37.04 (llama.cpp b7083 Vulkan: DeepSeek-R1-Distill-Llama-8B-Q8_0 decode 128 tok on amd-radeon-rx-9060-xt, llama.cpp b7083 Vulkan: Mistral-7B-Instruct-v0.3-Q8_0 decode 128 tok on amd-radeon-rx-9060-xt, llama.cpp b7083 Vulkan: Qwen3-8B-Q8_0 decode 128 tok on amd-radeon-rx-9060-xt, llama.cpp b7083 Vulkan: Llama-3.1-Tulu-3-8B-Q8_0 decode 128 tok on amd-radeon-rx-9060-xt) → 2.32× / 2.43× / 2.28× / 2.32× the 16.0 floor (derived) (38.95/16.0 = 2.434). The #10879 Q4_0 cell already on file is 74.01 (llama.cpp Vulkan on amd-radeon-rx-9060-xt: Llama-2-7B Q4_0 FA-on tg128 decode tok/s (scoreboard #10879)), 74.01/16.0 = 4.63× and 74.01/38.95 = 1.90× (derived) — the same Q4-versus-Q8 shape as the 9070, a little wider. gpt-oss-20b Q8_0 is 102.7 (llama.cpp b7083 Vulkan: gpt-oss-20b-Q8_0 decode 128 tok on amd-radeon-rx-9060-xt), 102.7/16.0 = 6.42× and 102.7/38.95 = 2.64× (derived). The 9070 has exactly twice the bandwidth (640/320 = 2), and the dense decode ratio is 72.72/38.95 = 1.87× (derived) — close to the bandwidth ratio, not equal to it. Prefill is further off: 2829.08/1792.13 = 1.58× (derived) (llama.cpp b7083 Vulkan: Mistral-7B-Instruct-v0.3-Q8_0 prefill 1024 tok on amd-radeon-rx-9060-xt and the other three dense pp1024 siblings at 1,787.9 / 1,745.08 / 1,788.06, plus the MoE pp1024 at 2,319.17, llama.cpp b7083 Vulkan: gpt-oss-20b-Q8_0 prefill 1024 tok on amd-radeon-rx-9060-xt). The B580 has more bandwidth than the 9060 XT (456/320 = 1.43× (derived)) and decodes the same dense Q8 file at 14.37/38.95 = 0.37× (derived) — the bandwidth roofline does not predict a Vulkan-on-ANV result, which is why the rank prices the ceiling and not this cell.

Refresh note (docs/fit-matrix-oct14-p225, Oct-17, this pass). Re-ran python tools/fit_matrix_35b.py on local main 77f4f6b (226 accelerator records, 218 supply records, 4,112 benchmarks — #225 on top of 19d2ccc). The selection rule still yields 159 qualifying / 43 priced / 116 unpriced (69 of the unpriced fit 35B-A3B, 47 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. #225 adds no accelerator and no purchase price. What it adds is a second box on a part §4 already lists, and a second reading of the rank-9 card plus the first time that card's stretch model is actually measured:

• L40S, still unpriced, and the new number is an 8-card aggregate, not a decode. NVIDIA L40S stays 48 GB / 864 GB/s, Y/Y/Y (§4). The ceilings are 864/20 = 43.2, 864/16 = 54.0, 864/40 = 21.6 tok/s (derived). #225 is HPE's MLPerf v5.0 closed submission, a DL380a Gen12 with 8× L40S and TensorRT-LLM, on llama2-70b-99.9 (MLPerf Inference v5.0 HPE ProLiant DL380a Gen12 8x L40S llama2-70b-99.9 raw logs). Offline is 3,655.89 tok/s aggregate (MLPerf v5.0 HPE DL380a Gen12 8x L40S: llama2-70b-99.9 Offline aggregate tokens/s) and Server is 3,181.65 (MLPerf v5.0 HPE DL380a Gen12 8x L40S: llama2-70b-99.9 Server aggregate tokens/s), with p99 first-token latency 525.5 ms at the 11 qps target (MLPerf v5.0 HPE DL380a Gen12 8x L40S: llama2-70b-99.9 Server p99 first-token latency). Dell's XE7745 in the same round, already on file, is 3,481.53 Offline and 3,201.13 Server (MLPerf v5.0 Dell XE7745 8x L40S: llama2-70b-99.9 Offline aggregate tokens/s, MLPerf v5.0 Dell XE7745 8x L40S: llama2-70b-99.9 Server aggregate tokens/s). 3655.89/3481.53 = 1.05× (derived) on Offline and 3181.65/3201.13 = 0.994× (derived) on Server — the two boxes agree on Server and differ by about 5% on Offline, which is the commit's own reading. 3655.89/8 = 457.0 tok/s per card (derived), and the record says that division is not a per-request decode. Set against the single-card llama.cpp Q4 cell already stored, 15.31 tok/s (NVIDIA L40S 48GB llama.cpp cuBLAS: LLaMA-3 70B Q4_K_M tg1024 decode tok/s), 15.31/21.6 = 0.71× the 70B floor (derived) (15.31/21.6 = 0.709). 457.0/15.31 = 29.8× (derived) — eight cards, a concurrent closed-division run, and a different engine, so the multiple is not a speedup and it does not enter §4b. No street price, so §3 does not move.

• RX 7900 XTX stays rank 9, and the new 7B cell is the low reading. AMD Radeon RX 7900 XTX is 24 GB / 960 GB/s, Y/Y/-, $875 → $18.23/tok-s (derived) (Used AMD RX 7900 XTX 24GB (eBay/peer market)). Ceilings: 960/20 = 48.0, 960/16 = 60.0, 960/40 = 24.0 tok/s (derived). The card does not fit 70B. #225 is llama-bench build 34ce48d97, ROCm 7.1.1, one card of a 2× box, Llama-2-7B Q4_0, FA off: tg128 118.82 and pp512 3,487.15 (RX 7900 XTX (GPU0) llama.cpp ROCm 7.1.1: Llama-2-7B Q4_0 FA-off tg128, RX 7900 XTX (GPU0) llama.cpp ROCm 7.1.1: Llama-2-7B Q4_0 FA-off pp512, 1337hero/rx7900xtx-llama-bench-rocm: llama.cpp ROCm 7.1.1 bench on 2x RX 7900 XTX). The maintainer scoreboard on the same model, quant, and tg128 test is 167.11 (llama.cpp ROCm on RX 7900 XTX: Llama-2-7B Q4_0 FA-off tg128 (#15021)). 167.11/118.82 = 1.41× (derived). The record keeps both: different build, different ROCm, and the two cards in this box already disagree (device 1 scored 130.67). Against the 20 GB ceiling, 118.82/48.0 = 2.48× and 167.11/48.0 = 3.48× (derived). A 3.56 GiB file is the wrong byte count for that denominator, same caveat as the other Llama-2-7B scoreboard rows, so neither enters §4b and neither re-ranks the $18.23 cell. pp512 against the scoreboard's 3,552.27 (llama.cpp ROCm on RX 7900 XTX: Llama-2-7B Q4_0 FA-off pp512 (#15021)) is 3487.15/3552.27 = 0.98× (derived) — the gap is in decode, not prefill.

• The stretch model, split across two of the same card, lands under the 70B floor. Llama 3.1 70B Q4_K_M is a 39.59 GiB file, so it does not fit the 24 GB the Y/Y/- flag describes. Split over both cards, FA on, the same build decodes at 13.37 tok/s (2x RX 7900 XTX llama.cpp ROCm 7.1.1: Llama 3.1 70B Q4_K_M FA-on tg128). One card's 70B ceiling is 24.0, and 13.37/24.0 = 0.56× (derived). Two cards would be 48.0 if bandwidth added, and 13.37/48.0 = 0.28× (derived) — it does not add. The link is host PCIe Gen4, which is the same failure the 2× tensor-split of Qwen3.8-27B Q4_K_XL already records at 47.06 tok/s on this pair (llama.cpp under ROCm on 2x RX 7900 XTX (gfx1100): Qwen3.8-27B Q4_K_XL batch-1 tensor-split decode throughput), 47.06/60.0 = 0.78× one card's 27B floor (derived). Different model, different build, same shape: the second card buys the fit and not the bandwidth. $875×2 / 13.37 = $130.89 per measured tok/s (derived) for a model the single card cannot hold. That is not a §3 rank — the section prices one card against the 20 GB ceiling, and this is two cards against a 39.59 GiB file.

Refresh note (docs/fit-matrix-oct14-p221, Oct-17, this pass). Re-ran python tools/fit_matrix_35b.py on local main 5ba1be0 (226 accelerator records, 218 supply records, 4,106 benchmarks — #221 and #222 on top of 58a57fa). The selection rule still yields 159 qualifying / 43 priced / 116 unpriced (69 of the unpriced fit 35B-A3B, 47 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. #221 adds no accelerator and no purchase price. #222 adds a model record, not a card. What #221 adds is the long-context half of a sweep §4b already cites at default context, on the same part:

• Strix Halo 8060S at 32k and 64k, and the named 35B-A3B Q4 cell holds. AMD Radeon 8060S (Ryzen AI Max+ integrated GPU) stays 128 GB unified / 256 GB/s, Y/Y/Y, rank 37 at $1,900 → $148.44/tok-s (derived) (35B ceiling 256/20 = 12.8 tok/s (derived), 27B ceiling 256/16 = 16.0 tok/s (derived)). #221 is kyuz0's llama-bench long-context rows (llama.cpp b9193 / b9187, -fa 1 -ngl 99 --mmap 0, one request, tg32) from the same results.json the default-context filing used (kyuz0 AMD Strix Halo toolboxes — llama-bench results.json (pp512/tg128 across ROCm 6.4.4/7.2.3/7-nightlies + Vulkan AMDVLK/RADV)). Qwen3.6-35B-A3B UD-Q4_K_XL is a 21.27 GiB file. Vulkan RADV decodes at 49.21 tok/s at depth 32768 and 43.23 at depth 65536 (kyuz0 Strix Halo 8060S Qwen3.6-35B-A3B-UD-Q4_K_XL Vulkan RADV 32k tg32, kyuz0 Strix Halo 8060S Qwen3.6-35B-A3B-UD-Q4_K_XL Vulkan RADV 64k tg32). Against the 12.8 floor that is 3.84× / 3.38× (derived) (49.21/12.8 = 3.845, 43.23/12.8 = 3.377). The same file on ROCm 7 nightlies is 44.45 and 39.30 (kyuz0 Strix Halo 8060S Qwen3.6-35B-A3B-UD-Q4_K_XL ROCm 7 nightlies 32k tg32, kyuz0 Strix Halo 8060S Qwen3.6-35B-A3B-UD-Q4_K_XL ROCm 7 nightlies 64k tg32) → 3.47× / 3.07× (derived). AMDVLK is 45.30 and 36.74 (kyuz0 Strix Halo 8060S Qwen3.6-35B-A3B-UD-Q4_K_XL Vulkan AMDVLK 32k tg32, kyuz0 Strix Halo 8060S Qwen3.6-35B-A3B-UD-Q4_K_XL Vulkan AMDVLK 64k tg32) → 3.54× / 2.87× (derived). 43.23/49.21 = 0.88× (derived) from 32k to 64k on RADV; 36.74/45.30 = 0.81× (derived) on AMDVLK. None of these is the §4b row. That row is the default-context tg128 cell at 60.43 (kyuz0 Strix Halo sweep: Qwen3.6-35B-A3B-UD-Q4_K_XL vulkan_radv decode_tok_s on Radeon 8060S (gfx1151), 4.7×), and 49.21/60.43 = 0.81× (derived) of it. tg32 at a primed 32k context is a different protocol from tg128 at the default depth, so it does not replace 60.43 and it does not replace the best-of-20 cell at 72.35. $1,900/49.21 = $38.61 per measured tok/s (derived) at 32k and $1,900/43.23 = $43.95 (derived) at 64k — both well above the rank-11 3090's derived $20.30 on the 20 GB ceiling, and a different context depth, so they do not re-rank. Prefill siblings, not a decode ceiling: RADV pp2048 is 715.52 at 32k and 524.7 at 64k (kyuz0 Strix Halo 8060S Qwen3.6-35B-A3B-UD-Q4_K_XL Vulkan RADV 32k pp2048, kyuz0 Strix Halo 8060S Qwen3.6-35B-A3B-UD-Q4_K_XL Vulkan RADV 64k pp2048).

• The same box, BF16, falls through the 20 GB floor. Qwen3.6-35B-A3B BF16 is a 64.6 GiB file, so the 20 GB denominator does not describe it. ROCm 7 nightlies still decodes at 23.96 tok/s at 32k and 22.60 at 64k (kyuz0 Strix Halo 8060S Qwen3.6-35B-A3B-BF16 ROCm 7 nightlies 32k tg32, kyuz0 Strix Halo 8060S Qwen3.6-35B-A3B-BF16 ROCm 7 nightlies 64k tg32) → 1.87× / 1.77× the 12.8 floor (derived) (23.96/12.8 = 1.872, 22.60/12.8 = 1.766). Vulkan does not: AMDVLK is 10.93 and 10.46 (kyuz0 Strix Halo 8060S Qwen3.6-35B-A3B-BF16 Vulkan AMDVLK 32k tg32, kyuz0 Strix Halo 8060S Qwen3.6-35B-A3B-BF16 Vulkan AMDVLK 64k tg32) → 0.85× / 0.82× (derived) (10.93/12.8 = 0.854, 10.46/12.8 = 0.817). 23.96/10.93 = 2.19× (derived), the same weights on two backends. Q8_K_XL sits between them: RADV 40.08 at 32k and 35.80 at 64k (kyuz0 Strix Halo 8060S Qwen3.6-35B-A3B-UD-Q8_K_XL Vulkan RADV 32k tg32, kyuz0 Strix Halo 8060S Qwen3.6-35B-A3B-UD-Q8_K_XL Vulkan RADV 64k tg32) → 3.13× / 2.80× (derived).

• The dense-27B cell on this box is Q8, not Q4, and it misses the 16 GB floor. Qwen3.6-27B UD-Q8_K_XL is a 33.31 GiB file. The best long-context decode is ROCm 6.4.4 at 6.13 tok/s at 32k and 5.81 at 64k (kyuz0 Strix Halo 8060S Qwen3.6-27B-UD-Q8_K_XL ROCm 6.4.4 32k tg32, kyuz0 Strix Halo 8060S Qwen3.6-27B-UD-Q8_K_XL ROCm 6.4.4 64k tg32). Against the 16.0 floor that is 0.38× / 0.36× (derived) (6.13/16.0 = 0.383, 5.81/16.0 = 0.363). RADV is 5.92 and 5.60 (kyuz0 Strix Halo 8060S Qwen3.6-27B-UD-Q8_K_XL Vulkan RADV 32k tg32, kyuz0 Strix Halo 8060S Qwen3.6-27B-UD-Q8_K_XL Vulkan RADV 64k tg32); AMDVLK is 5.42 and 5.03 (kyuz0 Strix Halo 8060S Qwen3.6-27B-UD-Q8_K_XL Vulkan AMDVLK 32k tg32, kyuz0 Strix Halo 8060S Qwen3.6-27B-UD-Q8_K_XL Vulkan AMDVLK 64k tg32). Q8 moves roughly twice the bytes the 16 GB denominator assumes, and the row is tg32 at 32k rather than a short-context Q4, so this is a bound on the floor, not a replacement for it. It does not enter §4b.

• Neighbours on the same sweep, none of them the 20 GB Q4 job. Gemma-4-26B-A4B UD-Q4_K_XL (15.9 GiB) on RADV is 42.40 at 32k and 37.46 at 64k (kyuz0 Strix Halo 8060S gemma-4-26B-A4B-it-UD-Q4_K_XL Vulkan RADV 32k tg32, kyuz0 Strix Halo 8060S gemma-4-26B-A4B-it-UD-Q4_K_XL Vulkan RADV 64k tg32) → 3.31× / 2.93× the 12.8 floor (derived). The dense sibling, Gemma-4-31B UD-Q4_K_XL (17.46 GiB), is 9.09 and 7.88 (kyuz0 Strix Halo 8060S gemma-4-31B-it-UD-Q4_K_XL Vulkan RADV 32k tg32, kyuz0 Strix Halo 8060S gemma-4-31B-it-UD-Q4_K_XL Vulkan RADV 64k tg32) → 0.57× / 0.49× the 16.0 floor (derived) (9.09/16.0 = 0.568, 7.88/16.0 = 0.493). gpt-oss-20b MXFP4 (11.27 GiB) is 61.12 and 50.13 (kyuz0 Strix Halo 8060S gpt-oss-20b-mxfp4 Vulkan RADV 32k tg32, kyuz0 Strix Halo 8060S gpt-oss-20b-mxfp4 Vulkan RADV 64k tg32) → 4.78× / 3.92× (derived). gpt-oss-120b MXFP4 (59.02 GiB) is 43.01 and 34.97 (kyuz0 Strix Halo 8060S gpt-oss-120b-mxfp4 Vulkan RADV 32k tg32, kyuz0 Strix Halo 8060S gpt-oss-120b-mxfp4 Vulkan RADV 64k tg32) → 3.36× / 2.73× (derived), a 120B-class file clearing the 20 GB floor because the active set is small. Qwen3.5-122B-A10B UD-Q5_K_XL (85.6 GiB) is 20.38 and 18.99 (kyuz0 Strix Halo 8060S Qwen3.5-122B-A10B-UD-Q5_K_XL Vulkan RADV 32k tg32, kyuz0 Strix Halo 8060S Qwen3.5-122B-A10B-UD-Q5_K_XL Vulkan RADV 64k tg32) → 1.59× / 1.48× (derived). GLM-4.7-Flash UD-Q8_K_XL (32.7 GiB) is 22.55 and 15.70 (kyuz0 Strix Halo 8060S GLM-4.7-Flash-UD-Q8_K_XL Vulkan RADV 32k tg32, kyuz0 Strix Halo 8060S GLM-4.7-Flash-UD-Q8_K_XL Vulkan RADV 64k tg32) → 1.76× / 1.23× (derived). Nemotron-3-Super-120B-A12B UD-Q4_K_XL (78.02 GiB) is 14.30 and 13.69 (kyuz0 Strix Halo 8060S NVIDIA-Nemotron-3-Super-120B-A12B-UD-Q4_K_XL Vulkan RADV 32k tg32, kyuz0 Strix Halo 8060S NVIDIA-Nemotron-3-Super-120B-A12B-UD-Q4_K_XL Vulkan RADV 64k tg32) → 1.12× / 1.07× (derived). MiniMax-M2.7 UD-Q3_K_S (87.2 GiB) is 14.65 and 8.84 (kyuz0 Strix Halo 8060S MiniMax-M2.7-UD-Q3_K_S Vulkan RADV 32k tg32, kyuz0 Strix Halo 8060S MiniMax-M2.7-UD-Q3_K_S Vulkan RADV 64k tg32) → 1.14× / 0.69× (derived) (14.65/12.8 = 1.145, 8.84/12.8 = 0.691) — the one large-MoE row in this sweep that drops under the floor between 32k and 64k. The dense control, Llama-2-7B Q4_0 (3.56 GiB), is 9.42 at 32k and 5.12 at 64k on AMDVLK (kyuz0 Strix Halo 8060S llama-2-7b.Q4_0 Vulkan AMDVLK 32k tg32, kyuz0 Strix Halo 8060S llama-2-7b.Q4_0 Vulkan AMDVLK 64k tg32), 9.42/12.8 = 0.74× (derived): a 7B file under a 20 GB ceiling is the KV, not the weights. None of these re-ranks §3.

• #222 is a model, not a row. MiniCPM-V 4.7 35B-A3B is a 35.213B-parameter multimodal MoE with a vendor 3B activated figure, so it sits in the same nameplate class as the 35B-A3B workload this matrix prices. No benchmark and no accelerator landed with it, so it changes no fit flag and no $/tok-s.

#223 re-reads OpenRouter's Llama-3.3-70B provider table. It moves no purchase price and no accelerator, so it does not enter this matrix.

Refresh note (docs/fit-matrix-oct14-p219, Oct-17, this pass). Re-ran python tools/fit_matrix_35b.py on local main d5c152f (226 accelerator records, 218 supply records, 3,765 benchmarks — #219 on top of 6eb566d). The selection rule still yields 159 qualifying / 43 priced / 116 unpriced (69 of the unpriced fit 35B-A3B, 47 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. #219 adds no accelerator and no purchase price. What it adds is a second protocol on a part §4 already lists, plus a two-card vLLM episode the single-card fit flag does not cover:

• 2× R9700, and the new dense-27B row is FP8 at concurrency 8, not the Q4 batch-1 cell already in §4. AMD Radeon AI PRO R9700 stays 32 GB / 640 GB/s, Y/Y/-, unpriced (35B ceiling 640/20 = 32.0 tok/s (derived), 27B ceiling 640/16 = 40.0 tok/s (derived)). §4 already quotes the single-card llama.cpp Q4 of this model: ROCm 30.66 tok/s, 0.77× the 40.0 floor (R9700 1-GPU ROCm Qwen3.8-27B UD-Q4_K_XL depth-0 decode). #219 is vLLM 0.27.1 on two of the same card, Qwen3.8-27B native FP8, concurrency 8, per-stream, TTFT excluded, a ~2048-token follow-up and 256 generated tokens (Local LLM Benchmarks R9700 episode CSV (Donato Capitella, r9700-episode.csv)). Depth 0 decodes at 135.853 tok/s and depth 8192 at 116.822 (2x R9700 vLLM 0.27.1 Qwen3.8-27B native FP8 decode at depth 0, concurrency 8, 2x R9700 vLLM 0.27.1 Qwen3.8-27B native FP8 decode at depth 8192, concurrency 8). Against one card's 40.0 floor that is 3.40× / 2.92× (derived) (135.853/40.0 = 3.396, 116.822/40.0 = 2.921). Both ratios use the wrong machine: two cards and a concurrent stream are not the single-device batch-1 the ceiling describes, and FP8 is not the Q4 the 16 GB denominator assumes. 116.822/135.853 = 0.86× (derived) from depth 0 to 8k. Prefill siblings, not a decode ceiling: 3,411.146 and 2,291.275 (2x R9700 vLLM 0.27.1 Qwen3.8-27B native FP8 prefill at depth 0, concurrency 8, 2x R9700 vLLM 0.27.1 Qwen3.8-27B native FP8 prefill at depth 8192, concurrency 8). The record's own notes drop the 24k-and-deeper rows because the prefix cache collapsed, so those are not a depth curve. It does not enter §4b and it does not re-rank §3: no street price, and a 2-GPU FP8 concurrency-8 cell is not the A3B the section ranks. Set next to the single-card Q4, 135.853/30.66 = 4.43× (derived) — two cards, a denser quant, eight streams, and a different engine, so the multiple is not a speedup.

• The other file is the Next/Flash class, and it only fits because the working set spills to NVMe. Same two cards, vLLM, Qwen3.8-Flash-Next MXFP4/FP8/GPTQ with MTP-3 (Qwen3.8-Flash-Next: 180B total, 6B active — not the 35B-A3B and not the 27B). Concurrency 1, TTFT excluded. Depth 0 decodes at 100.304 tok/s and depth 65536 at 106.684 (2x R9700 tcclaviger vLLM Qwen3.8-Flash-Next MXFP4/FP8/GPTQ MTP-3 decode at depth 0, 2x R9700 tcclaviger vLLM Qwen3.8-Flash-Next MXFP4/FP8/GPTQ MTP-3 decode at depth 65536). Against one card's 32.0 floor that is 3.13× / 3.33× (derived) (100.304/32.0 = 3.134, 106.684/32.0 = 3.334). 106.684/100.304 = 1.06× (derived) — the long context does not fall off, which is the MoE shape, on the wrong byte count and with speculative decoding on. The notes say the PLE rows live on NVMe (derived PLE file 41.60 GB), so the working set is larger than the 64 GB the two cards hold and the Y on one 32 GB card does not cover this file. Prefill siblings: 2,676.443 at depth 0 and 1,116.002 at 64k (2x R9700 tcclaviger vLLM Qwen3.8-Flash-Next MXFP4/FP8/GPTQ MTP-3 prefill at depth 0, 2x R9700 tcclaviger vLLM Qwen3.8-Flash-Next MXFP4/FP8/GPTQ MTP-3 prefill at depth 65536). The sustained batch on the same build is aggregate, not per-request: 100.75 / 81.057 / 99.263 / 94.933 / 121.014 tok/s at c1-distinct, c2-distinct, c2-shared, c4-distinct, c4-shared (2x R9700 tcclaviger vLLM Qwen3.8-Flash-Next MTP-3 aggregate decode, c1-distinct, 2x R9700 tcclaviger vLLM Qwen3.8-Flash-Next MTP-3 aggregate decode, c2-distinct, 2x R9700 tcclaviger vLLM Qwen3.8-Flash-Next MTP-3 aggregate decode, c2-shared, 2x R9700 tcclaviger vLLM Qwen3.8-Flash-Next MTP-3 aggregate decode, c4-distinct, 2x R9700 tcclaviger vLLM Qwen3.8-Flash-Next MTP-3 aggregate decode, c4-shared). 121.014/100.75 = 1.20× and 81.057/100.75 = 0.80× (derived) — sharing the prefix helps and a second distinct stream does not, and the page says C8/C16 were not tested. None of it enters §4b.

Refresh note (docs/fit-matrix-oct14-p217, Oct-17, this pass). Re-ran python tools/fit_matrix_35b.py on local main 84863d2 (226 accelerator records, 218 supply records, 3,752 benchmarks — #217 on top of 111572f). The selection rule still yields 159 qualifying / 43 priced / 116 unpriced (69 of the unpriced fit 35B-A3B, 47 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. #217 adds no accelerator and no purchase price. What it adds is a context sweep on a part §4 already lists and §4b already quotes at short context:

• M5 Max, unpriced, and the new dense-27B row lands under the 27B ceiling the way a dense read does. Apple M5 Max (MacBook Pro 14-inch, 2026) is 128 GB unified at 614 GB/s (Y/Y/Y, §4). The 35B ceiling is 614/20 = 30.7 tok/s (derived) and the 27B ceiling is 614/16 = 38.4 tok/s (derived, 614/16 = 38.375, printed 38.4). §4b already quotes the short-context A3B on this chip: MLX 4-bit 128.0 tok/s → 4.2× (ss-m5max-48-qwen35-35b-mlx-4bit decode) and llama.cpp Q4_K_M 89.4 → 2.9× (ss-m5max-48-qwen35-35b-llamacpp-q4km decode). #217 is a different protocol on the same 128 GB / 40-core machine: mlx_lm.generate, batch 1, one request, 128 generated tokens, version unprinted (r/LocalLLaMA: M5 Max 128GB mlx_lm.generate sweep (cryingneko, post 1rqnpvj)). The file is Qwen3.5-27B-Claude-4.6-Opus-Distilled MLX 6-bit, not Q4, and it decodes at 23.648 / 20.311 / 14.908 / 14.225 tok/s at ~4k / 16k / 32k / 64k (M5 Max 128GB mlx_lm: Qwen3.5-27B-Claude-4.6-Opus-Distilled-MLX-6bit decode at 4k (128 generated tokens), M5 Max 128GB mlx_lm: Qwen3.5-27B-Claude-4.6-Opus-Distilled-MLX-6bit decode at 16k (128 generated tokens), M5 Max 128GB mlx_lm: Qwen3.5-27B-Claude-4.6-Opus-Distilled-MLX-6bit decode at 32k (128 generated tokens), M5 Max 128GB mlx_lm: Qwen3.5-27B-Claude-4.6-Opus-Distilled-MLX-6bit decode at 64k (128 generated tokens)). Against the 38.4 floor that is 0.62× / 0.53× / 0.39× / 0.37× (derived) (23.648/38.375 = 0.616, 20.311/38.375 = 0.529, 14.908/38.375 = 0.388, 14.225/38.375 = 0.371). Under the floor at every depth, which is the same shape the 3090's Qwen3.5-27B Q4_K cell already shows at 0.57× (33.5/58.5) — different silicon, different quant, different engine, so the two ratios are not a ranking. 6-bit reads more bytes than the Q4 the floor assumes, so 0.62× is a lower bound on a Q4 reading of this file, not a Q4 measurement. Peak memory on the 4k row is 25.319 GB, above the 16 GB weight the ceiling divides by, which is why the file still clears the 20 GB fit bar and why the ratio sits where a dense read sits. 23.648/14.225 = 1.66× (derived) from 4k to 64k; the prefill siblings fall the same way, 811.134 → 475.828 (M5 Max 128GB mlx_lm: Qwen3.5-27B-Claude-4.6-Opus-Distilled-MLX-6bit prefill at 4k (4107 prompt tokens), M5 Max 128GB mlx_lm: Qwen3.5-27B-Claude-4.6-Opus-Distilled-MLX-6bit prefill at 64k (65547 prompt tokens), 811.134/475.828 = 1.70× (derived)). It does not enter §4b: a 6-bit dense 27B is not the A3B the section ranks, and there is still no purchase price, so §3 does not move. Set next to the MLX A3B cell already on this chip, 128.0/23.648 = 5.41× (derived) — the MoE-vs-dense gap, on one machine, at two different context depths.

• The other three files are larger than the 20 GB denominator, and two of them are the MoE class this doc is about. Same machine, same mlx_lm.generate protocol. Qwen3.5-122B-A10B MLX 4-bit decodes at 60.639 at ~16k and 54.923 at ~32k (M5 Max 128GB mlx_lm: Qwen3.5-122B-A10B-4bit decode at 16k (128 generated tokens), M5 Max 128GB mlx_lm: Qwen3.5-122B-A10B-4bit decode at 32k (128 generated tokens); peak 73.803 / 76.397 GB). Against the 30.7 floor that is 1.98× / 1.79× (derived) (60.639/30.7 = 1.975, 54.923/30.7 = 1.789). There is no 4k cell for this file. Qwen3-Coder-Next MLX 8-bit — the Next/Flash class, not a 35B-A3B — decodes at 74.293 / 68.624 / 48.212 at ~16k / 32k / 64k (M5 Max 128GB mlx_lm: Qwen3-Coder-Next-8bit decode at 16k (60 generated tokens), M5 Max 128GB mlx_lm: Qwen3-Coder-Next-8bit decode at 32k (58 generated tokens), M5 Max 128GB mlx_lm: Qwen3-Coder-Next-8bit decode at 64k (61 generated tokens); peak 88.176 / 89.652 / 92.605 GB) → 2.42× / 2.24× / 1.57× (derived) (74.293/30.7 = 2.420, 68.624/30.7 = 2.235, 48.212/30.7 = 1.570). It stays above the dense-equivalent floor out to 64k, which is the MoE shape §4b already records, on a larger active model and an 8-bit file. gpt-oss-120b MXFP4-Q8 decodes at 87.873 / 75.963 / 64.469 at ~4k / 16k / 32k (M5 Max 128GB mlx_lm: gpt-oss-120b-MXFP4-Q8 decode at 4k (128 generated tokens), M5 Max 128GB mlx_lm: gpt-oss-120b-MXFP4-Q8 decode at 16k (128 generated tokens), M5 Max 128GB mlx_lm: gpt-oss-120b-MXFP4-Q8 decode at 32k (128 generated tokens); peak 64.408 / 64.857 / 65.461 GB) → 2.86× / 2.47× / 2.10× (derived) (87.873/30.7 = 2.862, 75.963/30.7 = 2.474, 64.469/30.7 = 2.100). 120B is not the 27B class and MXFP4-Q8 is not the 20 GB Q4, so the multiple is the wrong byte count; it is the file that uses half the 128 GB pool and it does not enter §4b either. Prefill siblings, not a decode ceiling: 122B 1,239.734 / 1,067.824 (M5 Max 128GB mlx_lm: Qwen3.5-122B-A10B-4bit prefill at 16k (16394 prompt tokens), M5 Max 128GB mlx_lm: Qwen3.5-122B-A10B-4bit prefill at 32k (32778 prompt tokens)), Coder-Next 1,802.144 / 1,887.158 / 1,432.73 (M5 Max 128GB mlx_lm: Qwen3-Coder-Next-8bit prefill at 16k (16393 prompt tokens), M5 Max 128GB mlx_lm: Qwen3-Coder-Next-8bit prefill at 32k (32777 prompt tokens), M5 Max 128GB mlx_lm: Qwen3-Coder-Next-8bit prefill at 64k (65545 prompt tokens)), gpt-oss-120b 1,325.062 / 2,710.46 / 2,537.42 (M5 Max 128GB mlx_lm: gpt-oss-120b-MXFP4-Q8 prefill at 4k (4164 prompt tokens), M5 Max 128GB mlx_lm: gpt-oss-120b-MXFP4-Q8 prefill at 16k (16452 prompt tokens), M5 Max 128GB mlx_lm: gpt-oss-120b-MXFP4-Q8 prefill at 32k (32836 prompt tokens)). No street price on the M5 Max, so none of the four files produces a $/tok-s cell.

Refresh note (docs/fit-matrix-oct14-p215, Oct-17, this pass). Re-ran python tools/fit_matrix_35b.py on local main c45efa7 (226 accelerator records, 218 supply records, 3,728 benchmarks — #214 and #215 on top of 2ae5d01). The selection rule now yields 159 qualifying / 43 priced / 116 unpriced (69 of the unpriced fit 35B-A3B, 47 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. #214 adds sixteen accelerators and 123 llama.cpp scoreboard rows; #215 re-reads six service rows. None of the new accelerators has a purchase-price supply record, so the priced set stays 43.

• Five of the sixteen clear the gate. One fits 35B-A3B, and the fit is the board, not the GPU the benchmark measured. NVIDIA Tesla K80 is a dual-GK210 board, vram_gb 24 and memory_bandwidth_gbps 480 (the record's own basis: 240 GB/s and 12 GB per GK210, summed). 24 ≥ 20+4 and 24 ≥ 16+4, so the flags are Y/Y/- and the ceilings are 480/20 = 24.0, 480/16 = 30.0, 480/40 = 12.0 tok/s (derived). It enters §4 between Huawei Ascend 950DT NPU (per-accelerator, Atlas 650E / 950 SuperPoD) (25.0) and Huawei Atlas 300I Duo inference card (dual 310-series, 48/96 GB) (20.4). The record's notes say the scoreboard row is one GPU of the pair, and the benchmark notes print 12 GB / 384-bit. One GK210 is 12 GB, which misses the 24 GB bar, so the Y is a board total a single context cannot see. No purchase price.
• The other four clear the bandwidth floor and fit nothing. All -/-/-, all unpriced, all in the does-not-fit list: NVIDIA TITAN Xp (12 GB, 547.7 GB/s, 35B ceiling 547.7/20 = 27.4 tok/s (derived), between the Arc A770 at 28.0 and the RX 6800 XT at 25.6), AMD Radeon Instinct MI25 (16 GB HBM2, 484 GB/s, ceiling 24.2 (derived) — tied with AMD Radeon RX Vega 64, and the record notes that ROCm support for gfx900 ended years ago), NVIDIA GeForce RTX 2060 SUPER (8 GB, 448 GB/s, ceiling 22.4 (derived) — tied with the RX 5700 XT, the RTX 2070 SUPER / 3060 Ti / 3070, the RTX 5060 Ti and the RTX A4000) and NVIDIA Quadro RTX 4000 (8 GB, 416 GB/s, ceiling 20.8 (derived), between the RX 6750 XT at 21.6 and the RX Vega 56 at 20.5). 27B ceilings are 547.7/16 = 34.2, 484/16 = 30.3, 448/16 = 28.0, 416/16 = 26.0 tok/s (derived); none of the four reaches the 20 GB the 27B column needs.
• The other eleven never reach the gate. Below 8 GB: NVIDIA GeForce GTX 1060 6GB, NVIDIA GeForce GTX 1660, NVIDIA P106-100 and NVIDIA GeForce RTX 4050 Laptop GPU (all 6 GB), NVIDIA Quadro P2000 (5 GB), NVIDIA Quadro P1000 and NVIDIA Quadro T1000 (both 4 GB). At or above 8 GB but under 300 GB/s and with no <= $3,000 purchase price: NVIDIA GeForce GTX 1070 Ti (8 GB, 256 GB/s), NVIDIA Jetson AGX Orin 64GB (64 GB unified LPDDR5, 204.8 GB/s) and NVIDIA Tesla P4 (8 GB, 192 GB/s). AMD Radeon 880M is unified_memory with both vram_gb and memory_bandwidth_gbps null, and it is not in the generator's configured-pool override, so the gate drops it. None enters §3 or §4.
• Every new scoreboard cell is Llama-2-7B Q4_0, 3.56 GiB, not the 35B-A3B. llama-bench, pp512/tg128, batch 1, full offload, per-request (llama.cpp Discussion #15013 — Performance of llama.cpp on Nvidia CUDA (community scoreboard), and the ROCm half llama.cpp Discussion #15021 — Performance of llama.cpp on AMD ROCm/HIP (community scoreboard)). 3.56/20 = 0.18 of the weight the ceiling divides by (derived), so a ratio above 1× is the small-file case, not evidence the 20 GB model runs. The five that cleared the gate, FA where both exist: K80 14.27 tok/s (llama.cpp CUDA Tesla K80 llama-2-7b Q4_0 tg128 FA, no-FA 13.80 llama.cpp CUDA Tesla K80 llama-2-7b Q4_0 tg128 noFA). 14.27/24.0 = 0.59× the board ceiling (derived) and 13.80/24.0 = 0.58×; 14.27/13.80 = 1.03× (derived). Read against one GK210 the same cells are 14.27/12.0 = 1.19× and 13.80/12.0 = 1.15× (derived, 240/20 = 12.0) — still a 7B file, and the prefill pair is 133.36 vs 133.14, 1.00× (derived). TITAN Xp no-FA 76.08 (llama.cpp CUDA Titan Xp llama-2-7b Q4_0 tg128 noFA, FA 73.84 llama.cpp CUDA Titan Xp llama-2-7b Q4_0 tg128 FA): 76.08/27.4 = 2.78× (derived), 73.84/27.4 = 2.70×, 76.08/73.84 = 1.03×. MI25 has no FA row: 63.94 (llama.cpp ROCm Instinct MI25 llama-2-7b Q4_0 tg128 noFA), 63.94/24.2 = 2.64× (derived). RTX 2060 SUPER FA 61.13 (llama.cpp CUDA RTX 2060 SUPER llama-2-7b Q4_0 tg128 FA, no-FA 60.04 llama.cpp CUDA RTX 2060 SUPER llama-2-7b Q4_0 tg128 noFA): 61.13/22.4 = 2.73×, 60.04/22.4 = 2.68×, 61.13/60.04 = 1.02× (derived). Quadro RTX 4000 FA 67.62 (llama.cpp CUDA Quadro RTX 4000 llama-2-7b Q4_0 tg128 FA, no-FA 65.62 llama.cpp CUDA Quadro RTX 4000 llama-2-7b Q4_0 tg128 noFA): 67.62/20.8 = 3.25×, 65.62/20.8 = 3.15×, 67.62/65.62 = 1.03× (derived). None of these is an A3B measurement, so none enters §4b, and none has a price, so none re-ranks §3.
• #215 moves hosted tok/s, which this table does not rank. The third OpenRouter re-read of Llama 3.3 70B touches OpenRouter hosted API router, DeepInfra serverless inference, Novita AI serverless endpoints, SambaNova Cloud inference, Google Cloud Vertex AI generative inference and Together AI serverless inference. Those are service records, not purchase prices, so no $/tok-s cell changes.

Refresh note (docs/fit-matrix-oct14, Oct-17, this pass). Re-ran python tools/fit_matrix_35b.py on local main 2e3d05f (210 accelerator records, 218 supply records, 3,605 benchmarks — #211 and #212 on top of add09bf). The selection rule still yields 154 qualifying / 43 priced / 111 unpriced (68 of the unpriced fit 35B-A3B, 43 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. #211 adds a model record and fills one active-param field; #212 re-reads six service rows. Neither adds an accelerator, a supply row, or a benchmark, so the gate is the same 154 the previous note counted.

• The new model is the dense 27B this table already uses 16 GB for, and it does not change the byte count. Qwen3.8-27B is Qwen3.8-27B, dense hybrid, params_b 27.781 and active_params_b 27.781 (safetensors total 27,781,427,952 BF16; the card rounds it to 27B). The 27B column stays weights = 16 GB + 4 GB KV, needing vram_gb >= 20. Nothing in the new record is a measured Q4 file size, so the 16 GB assumption is not revised here. The card's own comparison (Terminal Bench 2.1 73.0 vs 63.4, SWE-bench Pro 61.7 vs 53.5, GPQA Diamond 89.2 vs 87.8 against Qwen3.6-27B) is a quality claim, not a bandwidth one, and it does not enter §3.
• The Q4 measurement of that model was already on file, on a part §3 already prices. Apple M3 Max (MacBook Pro 16-inch, Nov 2023) is rank 39 (128 GB, 400 GB/s, 27B ceiling 400/16 = 25.0 tok/s (derived), $3,400 → $170.00/tok-s (derived) from eBay used MacBook Pro M3 Max 16in 128GB (Oct 2026)). Ollama's llama.cpp backend runs Qwen3.8-27B Q4_K_M at 12.0 tok/s (Ollama (llama.cpp backend) on MacBook Pro M3 Max 128GB: Qwen3.8-27B Q4_K_M decode tokens/s). 12.0/25.0 = 0.48× that ceiling (derived) — under the floor, the same shape the M2 Max IQ3_S cell already shows at 0.43× and the R9700 UD-Q4_K_XL cell at 0.77×, on different silicon and different protocols, so the three ratios are not a ranking. The MLX sibling on the same machine is 42.5 tok/s (Ollama (MLX backend) on MacBook Pro M3 Max 128GB: Qwen3.8-27B 4-bit decode tokens/s), format_id null, methodology says NVFP4. 42.5/25.0 = 1.70× (derived), and 42.5/12.0 = 3.54× (derived) between the two backends. NVFP4 is not Q4_K_M, so the 1.70× is not a Q4 reading and it does not enter §4b. $3,400/12.0 = $283.33 per measured Q4 tok/s (derived) — not a re-rank; the ranking stays on the 20 GB ceiling.
• GLM-5.3's filled active-param field is the wrong model class. GLM-5.3 is 753.3B total with active_params_b now 40 (the card's figure; the record's notes keep the 40.93B config arithmetic beside it). A 40B-active MoE is not the 35B-A3B, the 27B dense, or the 70B dense this table fits, and the fill adds no accelerator and no price.
• #212 moves hosted tok/s, which this table does not rank. The OpenRouter re-read moves the Groq row on Llama 3.3 70B from 0.50 s / 198 tok/s to 0.49 s / 209 tok/s, and the DeepInfra, Novita, SambaNova, Vertex and Together P50s move in notes. Those are service records (OpenRouter hosted API router, DeepInfra serverless inference, Novita AI serverless endpoints, SambaNova Cloud inference, Google Cloud Vertex AI generative inference, Together AI serverless inference), not purchase prices, so no $/tok-s cell changes.

Refresh note (docs/fit-matrix-oct17-p11, Oct-17, this pass). Re-ran python tools/fit_matrix_35b.py on local main 42edb33 (210 accelerator records, 218 supply records, 3,605 benchmarks — #209 on top of c9ca28e). The selection rule still yields 154 qualifying / 43 priced / 111 unpriced (68 of the unpriced fit 35B-A3B, 43 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. #209 adds one accelerator and no purchase price. The new chip does not clear the gate, and the one new row that is the named 35B-A3B target sits on a part §4 already lists:

• Base M4 does not qualify. Apple M4 (base, Mac mini) is a Mac mini base M4, 24 GB unified LPDDR5X at 120 GB/s. 24 GB meets the 35B-A3B bar and the 27B bar and misses the 48 GB 70B bar (Y/Y/-), but 120 GB/s misses the 300 GB/s bandwidth floor, and there is no purchase-price supply record at <= $3,000 (no data/supply/ row names this id), so the selection rule keeps it out of the 154 — the same shape as GB10 and Jetson AGX Thor, at less than half their bandwidth. The arithmetic the ceiling column uses gives 120/20 = 6.0 tok/s (derived), 120/16 = 7.5 tok/s (derived) on the 27B column, and 120/40 = 3.0 tok/s (derived) on the 70B column. It does not enter §3 or §4. The record's own note says the comment does not print a GPU-core count, so this is the base M4 and not the M4 Pro or the M4 Max already ranked at 40.
• The two rows on that chip are the wrong byte count. llama.cpp discussion #4167, Metal, build 2e97c5f, batch 1: llama 3.2 3B Q4_K_M (1.87 GiB) tg128 45.95 (llama.cpp Metal on Apple M4 (Mac mini, 10 CPU cores, 24 GB): llama 3.2 3B Q4_K_M tg128) and Qwen2.5-Coder 7B Q4_K_M (4.36 GiB) tg128 22.60 (llama.cpp Metal on Apple M4 (Mac mini, 10 CPU cores, 24 GB): Qwen2.5-Coder 7B Q4_K_M tg128). Against the 6.0 floor those are 45.95/6.0 = 7.66× and 22.60/6.0 = 3.77× (derived) — a 1.87 GiB file and a 4.36 GiB file, not a 20 GB one, so the multiples are small-model calibrations and not evidence the 120 GB/s pool decodes the 35B-A3B at those rates. Neither enters §4b. No purchase price, so no $/tok-s cell.
• M2 Max, already in §4, now has the named 35B-A3B at Q4, and the chip in the comment is the slower SKU. Apple M2 Max (MacBook Pro 14/16-inch & Mac Studio, 2023) (96 GB, record bandwidth 400 GB/s, 35B ceiling 400/20 = 20.0 tok/s (derived), Y/Y/Y, unpriced). The same discussion, comment 18555918, is an M2 Max 30-core GPU, 32 GB, not the 38-core / 96 GB the record's bandwidth is published for — the record's own basis says the 30-core configuration is listed at 300 GB/s. llama-bench, batch 1, build unprinted: Qwen3.6-35B-A3B Q4_K_M (20.22 GiB) decodes at 58.56 tok/s tg128 (llama.cpp Metal on Apple M2 Max (30-core GPU, 32 GB): Qwen3.6-35B-A3B Q4_K_M tg128) and prefills at 671.4 tok/s pp4096 (llama.cpp Metal on Apple M2 Max (30-core GPU, 32 GB): Qwen3.6-35B-A3B Q4_K_M pp4096). Against the record's 20.0 floor that is 58.56/20.0 = 2.93× (derived). Against the 300 GB/s the comment's own SKU is listed at, 300/20 = 15.0 tok/s (derived) and 58.56/15.0 = 3.90× (derived). The 20.22 GiB file is the byte count the floor assumes, which is why this row enters §4b and the 7B rows above do not. It does not enter §3: still no purchase price, and the 32 GB machine is not the 96 GB the fit flag is computed against — 20.22 GiB of weights plus the OS on 32 GB is the record's own "context headroom is small" caveat, not a 70B fit.
• The dense sibling on the same comment lands under the floor, and it is the wrong quant. Qwen3.8-27B IQ3_S (11.20 GiB) decodes at 10.82 tok/s (llama.cpp Metal on Apple M2 Max (30-core GPU, 32 GB): Qwen3.8-27B IQ3_S tg128) and prefills at 106.78 (llama.cpp Metal on Apple M2 Max (30-core GPU, 32 GB): Qwen3.8-27B IQ3_S pp4096). The 27B ceiling on the record is 400/16 = 25.0 tok/s (derived); 10.82/25.0 = 0.43× (derived). IQ3_S is not Q4 and 11.20 GiB is not 16 GB, so the ratio is a smaller file than the denominator, which makes 0.43× a lower bound on how far a Q4 dense 27B would sit under the floor, not a Q4 measurement. The MoE-vs-dense gap on one comment is 58.56/10.82 = 5.41× (derived) at decode and 671.4/106.78 = 6.29× (derived) at prefill — the same shape the 3090 note already records (179.0 vs 33.5 at 4k), now on a 400 GB/s unified pool. The comment's MTP sweep is an ollama draft-head run and is not these rows.
• The rest of #209 is Llama-2-7B, so it calibrates and does not re-rank. On the priced M1 Ultra (rank 28, 800 GB/s, ceiling 40.0 tok/s (derived)): Q4_0 tg128 113.79 (llama.cpp Metal on Apple M1 Ultra (64-core GPU, 128 GB): llama 7B Q4_0 tg128 (discussion #4167, c1d0e7a)), 113.79/40.0 = 2.84× (derived) — a 7B file, not a 20 GB one. On the priced M3 Max (rank 39, 400 GB/s, ceiling 20.0 (derived), tied with the M2 Max): Q4_0 tg128 81.57 (llama.cpp Metal on Apple M3 Max (MBP 16, 40-core GPU): llama 7B Q4_0 tg128 (discussion #4167, c1d0e7a)), 81.57/20.0 = 4.08× (derived), same caveat. The M5 / M5 Pro / M5 Max / M5 Ultra cells in the same ingest are the same 7B protocol on parts §4 already lists. None of them is an A3B, so none enters §4b, and none carries a purchase price, so §3 does not move.

Refresh note (docs/fit-matrix-oct17-p10, Oct-17, this pass). Re-ran python tools/fit_matrix_35b.py on local main c9ca28e (209 accelerator records, 218 supply records, 3,549 benchmarks — #206 on top of 5bebcc4). The selection rule now yields 154 qualifying / 43 priced / 111 unpriced (68 of the unpriced fit 35B-A3B, 43 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. #206 adds two accelerators and no purchase price. Both clear the bandwidth floor and fit none of the three workloads, so both land in the does-not-fit list and neither enters §3 or §4:

• Arc A750 qualifies and does not fit. Intel Arc A750 8GB is 8 GB GDDR6 at 512 GB/s, 225 W. 8 GB misses the 24 GB 35B-A3B bar, the 20 GB 27B bar and the 48 GB 70B bar (-/-/-); 512 GB/s clears the 300 GB/s floor, which is why it qualifies with no purchase price. Ceiling 512/20 = 25.6 tok/s (derived) — tied with AMD Radeon RX 6800 / AMD Radeon RX 6800 XT / AMD Radeon RX 6900 XT, which are 16 GB at the same bus. The 27B column is 512/16 = 32.0 tok/s (derived) and the 70B column is 512/40 = 12.8 tok/s (derived). The only measured rows are llama-2-7b Q4_0 (3.56 GiB), batch-1, Vulkan, build c1b1876: noFA pp512 1,075.94 / tg128 42.66 (llama.cpp Vulkan Intel Arc A750 llama-2-7b Q4_0 pp512 noFA, llama.cpp Vulkan Intel Arc A750 llama-2-7b Q4_0 tg128 noFA) and FA pp512 303.37 / tg128 43.96 (llama.cpp Vulkan Intel Arc A750 llama-2-7b Q4_0 pp512 FA, llama.cpp Vulkan Intel Arc A750 llama-2-7b Q4_0 tg128 FA). Against the 25.6 floor the faster decode is 43.96/25.6 = 1.72× (derived) — a 3.56 GiB file, not a 20 GB one, so the multiple is a small-model calibration and not evidence the 8 GB card holds the 24 GB model. FA regresses prefill hard on this chip (1,075.94/303.37 = 3.55× (derived) slower); decode barely moves (43.96/42.66 = 1.03× (derived)). Neither row enters §4b.
• Radeon Pro W5700 qualifies and does not fit. AMD Radeon Pro W5700 is 8 GB GDDR6 at 448 GB/s, 205 W, RDNA1 Navi 10. Same -/-/- miss; 448 GB/s clears the floor. Ceiling 448/20 = 22.4 tok/s (derived) — tied with AMD Radeon RX 5700 XT, NVIDIA GeForce RTX 2070 SUPER, NVIDIA GeForce RTX 3060 Ti, NVIDIA GeForce RTX 3070 and NVIDIA GeForce RTX 5060 Ti 16GB. The 27B column is 448/16 = 28.0 tok/s (derived) and the 70B column is 448/40 = 11.2 tok/s (derived). llama-2-7b Q4_0, batch-1, Vulkan, build 23bc779: noFA pp512 449.85 / tg128 68.55 (llama.cpp Vulkan Radeon Pro W5700 llama-2-7b Q4_0 pp512 noFA, llama.cpp Vulkan Radeon Pro W5700 llama-2-7b Q4_0 tg128 noFA) and FA pp512 446.98 / tg128 71.30 (llama.cpp Vulkan Radeon Pro W5700 llama-2-7b Q4_0 pp512 FA, llama.cpp Vulkan Radeon Pro W5700 llama-2-7b Q4_0 tg128 FA). Against the 22.4 floor the faster decode is 71.30/22.4 = 3.18× (derived), again a 3.56 GiB file. FA is a wash on prefill (449.85/446.98 = 1.01× (derived)) and a small decode gain (71.30/68.55 = 1.04× (derived)). The W5700X (40 CU, 16 GB) is a different board and is not this record. Neither row enters §4b. No purchase price on either card, so no $/tok-s cell.

Refresh note (docs/fit-matrix-oct17-p9, Oct-17, this pass). Re-ran python tools/fit_matrix_35b.py on local main 6b863a4 (207 accelerator records, 218 supply records, 3,541 benchmarks — #203 on top of e68a507). The selection rule still yields 152 qualifying / 43 priced / 109 unpriced (68 of the unpriced fit 35B-A3B, 41 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. #203 adds one accelerator and no purchase price. The new card does not clear the gate:

• Arc Pro B50 does not qualify. Intel Arc Pro B50 is 16 GB GDDR6 at 224 GB/s, 70 W, half-height. 16 GB misses the 24 GB 35B-A3B bar and the 20 GB 27B bar (-/-/-), and 224 GB/s misses the 300 GB/s bandwidth floor. There is no purchase-price supply record at <= $3,000 (no data/supply/ row names this id), so the selection rule keeps it out of the 152 — the same shape as RTX PRO 2000 Blackwell, except that card clears the bandwidth floor and this one does not. The arithmetic the ceiling column uses gives 224/20 = 11.2 tok/s (derived), 224/16 = 14.0 tok/s (derived) on the 27B column, and 224/40 = 5.6 tok/s (derived) on the 70B column. It does not enter §3 or §4. The record's own note says a 4-bit 14B fits and a 4-bit 32B dense does not, which is the same 16 GB miss the fit flags encode.
• The two new rows are a 4-card aggregate, and they are the wrong byte count. MLPerf Inference v6.0 closed, PyTorch vLLM, Llama-3.1-8B INT8, vendor-measured on 4× MAXSUN Arc Pro B50: Offline 1,721.84 tok/s aggregate (Intel Arc Pro B50 x4, MLPerf v6.0 Offline: Llama-3.1-8B INT8 aggregate tok/s) and Server 1,187.13 tok/s aggregate (Intel Arc Pro B50 x4, MLPerf v6.0 Server: Llama-3.1-8B INT8 aggregate tok/s). The record says the submission does not publish a per-card split; dividing by 4 is an average only, 1,721.84/4 = 430.46 and 1,187.13/4 = 296.78 tok/s per card (derived). Against the 11.2 floor those are 38.4× and 26.5× (derived) (430.46/11.2 = 38.43, 296.78/11.2 = 26.50) — an INT8 8B, not a Q4 20 GB file, so the multiple is a small-model calibration and not evidence the 16 GB card holds the 24 GB model. Offline is a throughput scenario and Server is latency-bounded; neither is a batch-1 decode, and neither enters §4b. The same submitter's 4× Arc Pro B60 Offline row on Llama-3.1-8B FP8 is 3,347.43 (Intel Arc Pro B60 24GB x4, MLPerf v6.0 Offline: Llama-3.1-8B aggregate tok/s); 3,347.43/1,721.84 = 1.94× (derived), different silicon, different quant (FP8 vs INT8), same 4-card shape, not a re-rank. Intel Arc Pro B60 stays at rank 16 ($650, 24 GB, Y/Y/-). No purchase price on the B50, so no $/tok-s cell.

Refresh note (docs/fit-matrix-oct14, Oct-17, this pass). Re-ran python tools/fit_matrix_35b.py on local main af598e8 (206 accelerator records, 218 supply records, 3,539 benchmarks — #201 on top of c49175c). The selection rule still yields 152 qualifying / 43 priced / 109 unpriced (68 of the unpriced fit 35B-A3B, 41 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. #201 adds no accelerator and no purchase price. What it adds is three Database Mart tables on parts the previous note already placed: one priced, one unpriced-and-fits, one that qualifies and does not fit.

• A100 40GB PCIe, rank 32, and the new rows are aggregate. NVIDIA A100 40GB PCIe (40 GB, 1,555 GB/s, 35B ceiling 1,555/20 = 77.8 tok/s (derived), 27B ceiling 1,555/16 = 97.2 tok/s (derived), $7,000 → $90.03/tok-s (derived) from eBay used A100 40GB PCIe card). vLLM, unpinned build, 50 concurrent, quant unnamed so format_id is null: Gemma 3 4B 3,385.72 aggregate (A100 40GB PCIe vLLM: google/gemma-3-4b-it aggregate output throughput, 50 concurrent), Gemma 3 12B 2,091.15 (A100 40GB PCIe vLLM: google/gemma-3-12b-it aggregate output throughput, 50 concurrent), Qwen2.5 7B 2,225.26 (A100 40GB PCIe vLLM: Qwen/Qwen2.5-7B-Instruct aggregate output throughput, 50 concurrent), Qwen2.5 14B 420.31 (A100 40GB PCIe vLLM: Qwen/Qwen2.5-14B-Instruct aggregate output throughput, 50 concurrent). Divided by the 50 streams that is 67.71 / 41.82 / 44.51 / 8.41 tok/s per stream (derived) (3385.72/50 = 67.714, 2091.15/50 = 41.823, 2225.26/50 = 44.505, 420.31/50 = 8.406). Against the 77.8 floor those are 0.87× / 0.54× / 0.57× / 0.11× (derived). An aggregate per-stream rate is not a batch-1 decode, and none of the four is the 35B-A3B the §4b row already quotes at 136 tok/s (A100 40GB PCIe llama.cpp b9935: Qwen3.6-35B-A3B UD-Q4_K_XL (MoE) decode, 1.7× that same floor). The 14B cell is the closest byte count to the 27B column and it lands at 8.41/97.2 = 0.09× (derived) — a concurrency effect, the record's own note says the 14B row needed --max-model-len 4096 to load, not a new ceiling. $7,000 stays on the 20 GB ceiling. None of the four enters §4b.
• A40 stays unpriced, and the new rows are the same aggregate shape the previous note already recorded. NVIDIA A40 (48 GB, 696 GB/s, 35B ceiling 696/20 = 34.8 tok/s (derived), Y/Y/Y, §4). Same vLLM page, six model rows the earlier filing did not have. DeepSeek-R1-Distill-Llama-8B is 1,155.8 aggregate at 50 concurrent (A40 vLLM: deepseek-ai/DeepSeek-R1-Distill-Llama-8B aggregate output throughput, 50 concurrent, 1,155.8/50 = 23.12 per stream (derived)) and 1,333.21 at 100 (A40 vLLM: deepseek-ai/DeepSeek-R1-Distill-Llama-8B aggregate output throughput, 100 concurrent, 1,333.21/100 = 13.33 per stream (derived)). DeepSeek-R1-Distill-Qwen-14B is 446.73 at 50 (A40 vLLM: deepseek-ai/DeepSeek-R1-Distill-Qwen-14B aggregate output throughput, 50 concurrent, 8.93 per stream (derived)). Gemma 3 12B is 546.93 at 50 (A40 vLLM: google/gemma-3-12b-it aggregate output throughput, 50 concurrent, 10.94 (derived)). Gemma 3 4B is 756.72 at 50 (A40 vLLM: google/gemma-3-4b-it aggregate output throughput, 50 concurrent, 15.13 (derived)) and 401.22 at 100 (A40 vLLM: google/gemma-3-4b-it aggregate output throughput, 100 concurrent, 4.01 (derived)). 23.12/34.8 = 0.66× and 13.33/34.8 = 0.38× (derived) — both under the floor, and both aggregates of an 8B, the same caveat the prior Llama 3.1 8B cells on this card already carry (20.33 per stream at 50, 17.05 at 100). The 4B row going from 15.13 per stream at 50 concurrent to 4.01 at 100 is a saturation drop, not a bandwidth reading. No purchase price, so §3 does not move, and none of the six is an A3B, so §4b does not either.
• RTX A4000 qualifies and does not fit, and the one dense-27B-shaped row is a CPU spill. NVIDIA RTX A4000 is 16 GB at 448 GB/s. 16 GB misses the 24 GB 35B-A3B bar and the 20 GB 27B bar (-/-/-), and 448 GB/s clears the bandwidth floor, so it stays in the 41-row does-not-fit set. The ceiling arithmetic gives 448/20 = 22.4 tok/s (derived) and 448/16 = 28.0 tok/s (derived) on the 27B column. Ollama 0.5.11, default 4-bit pull, exact GGUF quant unprinted, one short question, per-request: llama2:7b 65.06 (RTX A4000 Ollama 0.5.11: llama2:7b per-request eval rate, 65.06/22.4 = 2.90× the 20 GB floor (derived)), mistral:7b 64.16 (RTX A4000 Ollama 0.5.11: mistral:7b per-request eval rate, 2.86×), qwen2.5:7b 52.68 (RTX A4000 Ollama 0.5.11: qwen2.5:7b per-request eval rate, 2.35×), deepseek-r1:7b 52.61 (RTX A4000 Ollama 0.5.11: deepseek-r1:7b per-request eval rate, 2.35×), deepseek-r1:8b 51.6 (RTX A4000 Ollama 0.5.11: deepseek-r1:8b per-request eval rate, 2.30×), llama3.1:8b 51.35 (RTX A4000 Ollama 0.5.11: llama3.1:8b per-request eval rate, 2.29×), gemma2:9b 39.04 (RTX A4000 Ollama 0.5.11: gemma2:9b per-request eval rate, 1.74×), llama2:13b 38.46 (RTX A4000 Ollama 0.5.11: llama2:13b per-request eval rate, 1.72×), deepseek-r1:14b 30.2 (RTX A4000 Ollama 0.5.11: deepseek-r1:14b per-request eval rate, 1.35×), qwen2.5:14b 30.05 (RTX A4000 Ollama 0.5.11: qwen2.5:14b per-request eval rate, 1.34×), deepseek-coder-v2:16b 22.89 (RTX A4000 Ollama 0.5.11: deepseek-coder-v2:16b per-request eval rate, 1.02×). All eleven are the wrong byte count for a 20 GB Q4, and the 16B cell sitting at 1.02× the 20 GB floor is a coincidence of denominators, not evidence the 16 GB card holds the 24 GB model. gemma2:27b evals at 2.38 (RTX A4000 Ollama 0.5.11: gemma2:27b per-request eval rate). 2.38/28.0 = 0.09× the 27B ceiling (derived), and the record flags it: 1% GPU, 70–86% CPU, the weight does not fit in 16 GB. That is a spill, not a dense-27B measurement, and it does not enter §4b. The Vulkan #10879 Llama-2-7B Q4_0 cell already cited for this card is 76.07 (llama.cpp Vulkan on nvidia-rtx-a4000: Llama-2-7B Q4_0 FA-on tg128 decode tok/s (scoreboard #10879)); 65.06 against 76.07 is a free-length default pull versus a fixed tg128, not a second protocol to average. No purchase price.

Refresh note (docs/fit-matrix-oct14, Oct-17, this pass). Re-ran python tools/fit_matrix_35b.py on local main b0ff5fd (206 accelerator records, 218 supply records, 3,507 benchmarks — #199 on top of 35d3ae1). The selection rule still yields 152 qualifying / 43 priced / 109 unpriced (68 of the unpriced fit 35B-A3B, 41 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. #199 adds one accelerator and no purchase price. The new card does not clear the gate:

• RTX PRO 2000 Blackwell does not qualify. NVIDIA RTX PRO 2000 Blackwell is 16 GB GDDR7 ECC at 256 GB/s, 70 W. 16 GB misses the 24 GB 35B-A3B fit bar (-/-/-) and 256 GB/s misses the 300 GB/s bandwidth floor, and there is no purchase-price supply record at <= $3,000, so the selection rule keeps it out of the 152 — the same shape as GB10 and Jetson AGX Thor, except this one also fails the fit flag. The arithmetic the ceiling column uses gives 256/20 = 12.8 tok/s (derived), 256/16 = 16.0 tok/s (derived) on the 27B column, and 256/40 = 6.4 tok/s (derived) on the 70B column. It does not enter §3 or §4.
• The Ollama rows on that card are per-request, and none of them is the 35B-A3B. Default-pull INT4, prompt "What is GPU?", batch 1, exact GGUF quant unprinted so format_id is null. gpt-oss-20b evals at 61.51 tok/s (RTX PRO 2000 Blackwell Ollama: gpt-oss:20b eval rate, 14 GB resident — a 20B file on a 16 GB card). 61.51/12.8 = 4.81× the 20 GB floor (derived), which is the wrong byte count for a 20B, the same caveat the GB10 gpt-oss-20b note already uses. llama3.1:8b is 59.05 (RTX PRO 2000 Blackwell Ollama: llama3.1:8b eval rate, 59.05/12.8 = 4.61× (derived)) and qwen3:8b is 48.75 (RTX PRO 2000 Blackwell Ollama: qwen3:8b eval rate, 3.81× (derived)). Both are small-model calibrations. None enters §4b.
• The vLLM rows on the same card are aggregate, and they are not divided into the floor. 1000 concurrent, input 100 / output 600, quant column only says "16". Llama 3.1 8B is 943.99 tok/s aggregate (RTX PRO 2000 Blackwell vLLM: meta-llama/Llama-3.1-8B-Instruct aggregate output throughput, 1000 requests), Qwen3 8B 941.32 (RTX PRO 2000 Blackwell vLLM: Qwen/Qwen3-8B aggregate output throughput, 1000 requests), gpt-oss-20b 1,037.61 (RTX PRO 2000 Blackwell vLLM: openai/gpt-oss-20b aggregate output throughput, 1000 requests), Gemma 3 12B 512.92 (RTX PRO 2000 Blackwell vLLM: google/gemma-3-12b-it aggregate output throughput, 1000 requests). 943.99/1000 = 0.94 tok/s per stream (derived) — an aggregate split, not a per-request rate, and not comparable to the 59.05 Ollama cell.
• PRO 6000 Blackwell, already priced, gets a per-request 70B and it lands under the floor. NVIDIA RTX PRO 6000 Blackwell Workstation Edition (rank 33, 96 GB, 1,792 GB/s, 70B ceiling 1,792/40 = 44.8 tok/s (derived), $8,650). Ollama 0.13.5, default 4-bit pull, same one-question prompt: Llama 3.3 70B evals at 31.96 tok/s (RTX PRO 6000 Blackwell Ollama 0.13.5: llama3.3:70b eval rate, 43 GB resident). 31.96/44.8 = 0.71× (derived). A 43 GB file is larger than the 40 GB denominator, and the quant is unnamed, so the ratio is a lower bound on a Q4 read rather than a Q4 measurement. $8,650/31.96 = $270.65 per measured tok/s (derived) — not a re-rank, the ranking stays on the 20 GB ceiling ($96.54/tok-s). gpt-oss-20b on the same card is 185.09 (RTX PRO 6000 Blackwell Ollama 0.13.5: gpt-oss:20b eval rate, 14 GB), 185.09/89.6 = 2.07× the 35B ceiling (derived), wrong byte count. gpt-oss-120b is 134.28 (RTX PRO 6000 Blackwell Ollama 0.13.5: gpt-oss:120b eval rate, 65 GB), 134.28/89.6 = 1.50× (derived). Neither is a Q4 35B-A3B, so neither enters §4b. Not comparable to the Hardware Corner llama-bench rows already cited for this card: fixed pp/tg lengths and a named GGUF quant there, a free-length default pull here.
• PRO 5000 Blackwell, A6000 and A40 stay unpriced, and the new rows are aggregate. NVIDIA RTX PRO 5000 Blackwell Workstation Edition (48 GB, ceiling 67.2 (derived), §4) gets its first vLLM table, 1000 concurrent, quant column "16", Server Edition not the 72 GB SKU: Llama 3.1 8B 2,878.77 aggregate (RTX PRO 5000 Blackwell vLLM: meta-llama/Llama-3.1-8B-Instruct aggregate output throughput, 1000 requests), Qwen3 8B 2,743.63 (RTX PRO 5000 Blackwell vLLM: Qwen/Qwen3-8B aggregate output throughput, 1000 requests), gpt-oss-20b 2,113.77 (RTX PRO 5000 Blackwell vLLM: openai/gpt-oss-20b aggregate output throughput, 1000 requests), Gemma 3 12B 1,706.59 (RTX PRO 5000 Blackwell vLLM: google/gemma-3-12b-it aggregate output throughput, 1000 requests). 2,878.77/1000 = 2.88 per stream (derived). Not a decode ceiling and not a price, so §3 and §4b do not move. NVIDIA RTX A6000 (ceiling 38.4) is recorded only at 50 concurrent — the page's 100-concurrent table contradicts itself and was dropped: Llama 3.1 8B 1,218.63 aggregate (RTX A6000 vLLM: meta-llama/Llama-3.1-8B-Instruct aggregate output throughput, 50 concurrent), 1,218.63/50 = 24.37 per stream (derived). NVIDIA A40 (ceiling 34.8) is consistent at both depths: Llama 3.1 8B 1,016.6 at 50 (A40 vLLM: meta-llama/Llama-3.1-8B-Instruct aggregate output throughput, 50 concurrent, 20.33 per stream (derived)) and 1,705.49 at 100 (A40 vLLM: meta-llama/Llama-3.1-8B-Instruct aggregate output throughput, 100 concurrent, 17.05 per stream (derived)). All three are aggregates of small dense models. None is a purchase price.

Refresh note (docs/fit-matrix-oct14, Oct-17, this pass). Re-ran python tools/fit_matrix_35b.py on local main 0e0a10b (205 accelerator records, 218 supply records, 3,467 benchmarks — #197 on top of 684bd5c). The selection rule still yields 152 qualifying / 43 priced / 109 unpriced (68 of the unpriced fit 35B-A3B, 41 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. #197 adds no accelerator and no purchase price. What it adds is the first MLPerf LLM rows and the first vendor vLLM table for a part this doc already keeps outside the gate:

• Jetson AGX Thor still does not qualify, and the new vendor row is a Qwen3-30B-A3B decode on the same 273 GB/s bus. NVIDIA Jetson AGX Thor 128GB is 128 GB unified LPDDR5X at 273 GB/s. That clears the 24 GB fit bar and misses the 300 GB/s bandwidth floor, and there is still no purchase-price supply record at <= $3,000 (the record's accelerator_ids are joined to nothing in data/supply/), so the selection rule keeps it out of the 152. The same arithmetic the ceiling column uses gives 273/20 = 13.7 tok/s (derived), 273/16 = 17.1 tok/s (derived) on the 27B column, and 273/40 = 6.8 tok/s (derived) on the 70B column. NVIDIA's Jetson Benchmarks page (vLLM, JetPack 7.0, input 2048 / output 128, concurrency 1, weight dtype not stated so format_id is null) measures Qwen3 30B-A3B at 61.0 tok/s per request (NVIDIA Jetson page: Thor vLLM Qwen3 30B-A3B, concurrency 1, 61 tok/s). 61.0/13.7 = 4.5× that 20 GB floor (derived) (61.0 ÷ (273/20) = 4.47). The same page's concurrency-8 column is 226.4 tok/s aggregate (NVIDIA Jetson page: Thor vLLM Qwen3 30B-A3B, concurrency 8, 226.4 tok/s aggregate), 226.4/8 = 28.3 per stream (derived) — an aggregate, not a second per-request rate, and not comparable to the 61.0.
• The dense rows on the same page land under the floor, which is what a dense read looks like on this bus. Qwen3 32B decodes at 13.19 tok/s (NVIDIA Jetson page: Thor vLLM Qwen3 32B, concurrency 1, 13.19 tok/s) against the 27B ceiling of 17.1 → 13.19/17.1 = 0.77× (derived). Llama 3.3 70B decodes at 4.7 tok/s (NVIDIA Jetson page: Thor vLLM Llama 3.3 70B, concurrency 1, 4.7 tok/s) against the 70B ceiling of 6.8 → 4.7/6.8 = 0.69× (derived). A 32B file is larger than the 16 GB denominator and a 70B file is larger than the 40 GB one, and the page does not state the quant, so both ratios are a lower bound on a Q4 read rather than a Q4 measurement. Neither enters §4b: the part is outside the selection rule, and the dtype is unnamed. Llama 3.1 8B at 41.3 tok/s (NVIDIA Jetson page: Thor vLLM Llama 3.1 8B, concurrency 1, 41.3 tok/s) is the wrong byte count for the 20 GB ceiling (41.3/13.7 = 3.0× (derived)) and stays a small-model calibration.
• The named 35B-A3B target was already on this part, and the new row is not it. The third-party SGLang cell already cited below is Qwen3.6-35B-A3B FP8 at 14.6 tok/s (Jetson AGX Thor SGLang: Qwen3.6-35B-A3B FP8 decode (Multimodalflow)), 14.6/13.7 = 1.07× the same floor (derived). 61.0/14.6 = 4.2× (derived) between the vendor's unnamed-dtype 30B-A3B and that FP8 35B-A3B. Different model, different quant, different engine (vLLM vs SGLang), same bus — the gap is not a claim that Thor got faster, and neither number is a Q4. The 30B-A3B row does not enter the §4b proxy set for the same reason the GB10 note gives: no price, and the part is outside the selection rule. It does corroborate the direction that note already records, that an A3B on 273 GB/s sits several times above the dense-equivalent floor when the quant is not FP8.
• The MLPerf rows are aggregate throughput, not a decode ceiling. v5.0 llama2-70b-99 FP16 Offline is 573.765 tok/s aggregate on one Thor (MLPerf v5.0 Jetson AGX Thor: llama2-70b-99 Offline 573.765 tok/s aggregate, TensorRT 10.8, open division). Offline is a throughput scenario; the record says so, and 573.765 is not divided into the 6.8 tok/s per-request floor. v6.0 llama3.1-8b-edge FP4 Offline is 350.605 tok/s aggregate (MLPerf v6.0 Jetson AGX Thor: llama3.1-8b-edge Offline 350.605 tok/s aggregate); the SingleStream siblings are TTFT and TPOT, not tok/s, so they are not a ceiling either. None of the 18 new rows is a purchase price, so none moves a $/tok-s rank.

Refresh note (docs/fit-matrix-oct14, Oct-17, this pass). Re-ran python tools/fit_matrix_35b.py on local main 4170507 (205 accelerator records, 218 supply records, 3,449 benchmarks — #192, #193, #194 and #195 on top of 166dd51). The selection rule still yields 152 qualifying / 43 priced / 109 unpriced (68 of the unpriced fit 35B-A3B, 41 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. #193 is a 7B model-parameter fill, #194 is the consumer-hardware ingest already reflected in the counts above, and #195 is a services telemetry re-read; none of the three adds an accelerator or a purchase price. #192 adds no accelerator and no purchase price either. What it adds is the flash-attention half of the llama.cpp Vulkan #10879 Llama-2-7B Q4_0 table, 82 rows (41 tg128 + 41 pp512) on cards the selection rule already covers:

• 23 of the 41 already had the no-flash-attention sibling, and flash attention barely moves a 7B decode. Same build (d05fe1d), same file (llama-2-7b.Q4_0.gguf, 3.56 GiB), -fa 1 against -fa 0. The ratio of the two tg128 cells runs from 0.957× to 1.061× on 22 of the 23 (derived): NVIDIA GeForce RTX 3090 171.61 vs 164.05 (llama.cpp Vulkan on nvidia-rtx-3090: Llama-2-7B Q4_0 FA-on tg128 decode tok/s (scoreboard #10879), llama.cpp Vulkan on RTX 3090: Llama-2-7B Q4_0 FA-off tg128 decode tok/s (scoreboard #10879), 171.61/164.05 = 1.046); NVIDIA GeForce RTX 5090 273.68 vs 263.63 (1.038); AMD Radeon RX 7900 XTX 190.92 vs 182.63 (1.045); NVIDIA Tesla P100 SXM2 16GB (GP100) 66.48 vs 63.16 (1.053); Apple M4 Max (MacBook Pro 14-inch, 2024) 71.79 vs 75.02 (0.957, flash attention slower); AMD Radeon RX 9070 XT 131.54 vs 137.11 (0.959, also slower). The one card outside that band is NVIDIA GeForce RTX 5070 Ti: 176.18 with flash attention against 135.63 without (llama.cpp Vulkan on nvidia-rtx-5070-ti: Llama-2-7B Q4_0 FA-on tg128 decode tok/s (scoreboard #10879), llama.cpp Vulkan on RTX 5070 Ti: Llama-2-7B Q4_0 FA-off tg128 decode tok/s (scoreboard #10879)), 176.18/135.63 = 1.299× (derived). Both rows are status: verified and both cite the same discussion page, so the split is kept rather than averaged — the same treatment the 3090 Ti's two Hardware Corner pages get. 176.18/44.8 = 3.93× the 20 GB ceiling (derived) and 135.63/44.8 = 3.03×; the 3090's own FA cell is 171.61/46.8 = 3.67× and the 5090's is 273.68/89.6 = 3.05×.
• 18 cards get their first Vulkan #10879 row, and every one of them is still a 7B file. The new tg128 cells are NVIDIA A100 80GB SXM4 170.56 (170.56/102.0 = 1.67× the 20 GB ceiling (derived) — the lowest ratio in the pass, and the one place a 7B decode does not clear the 35B floor), AMD Radeon Instinct MI50X 117.94 (2.30×), AMD Radeon PRO V620 79.24 (3.10×), AMD Radeon Pro VII 108.45 (2.12×), NVIDIA GeForce RTX 2080 Ti 100.99 (3.28×), NVIDIA RTX A4000 76.07 (3.40×), NVIDIA RTX A5000 140.43 (3.66×), AMD Radeon RX 5700 XT 74.43 (3.32×), AMD Radeon RX 6800 XT 100.50 (3.93×), AMD Radeon RX 6900 XT 106.15 (4.15×), AMD Radeon RX 7600 XT 56.11 (3.90×), AMD Radeon RX 7800 XT 124.86 (4.00×), AMD Radeon RX 7900 GRE 110.50 (3.84×), AMD Radeon RX 7900 XT 120.62 (3.02×), AMD Radeon RX 9060 XT 16GB 74.01 (4.63×), AMD Radeon RX 9070 119.51 (3.73×), AMD Radeon 8060S (Ryzen AI Max+ integrated GPU) 54.38 (4.25×) and NVIDIA V100 SXM2 32GB 142.13 (3.16×). Record ids are llamacpp-vulkan-<slug>-llama27b-q40-fa-tg128 with the matching pp512 sibling. The 2080 Ti (11 GB), 5700 XT (8 GB), 7600 XT (16 GB at 288 GB/s) and the rest of the sub-24 GB cards stay in the does-not-fit set; none of them gains a price.
• None of the 82 rows is the 35B-A3B workload, so none enters §4b and none moves a rank. Dividing a 3.56 GiB Llama-2-7B decode by the 20 GB denominator is a small-model calibration, the same caveat the Radeon VII note already states. The pp512 siblings (for example 5,200.7 on the 3090 and 11,796.4 on the 5090) are prefill and are not a decode ceiling. The R9700's dense 27B row from #188 is unchanged by this pass: its new cell is the 7B scoreboard at 152.70, next to the 145.67 no-FA sibling already cited, not the Qwen3.8-27B measurement.

Refresh note (docs/fit-matrix-oct14, Oct-17, this pass). Re-ran python tools/fit_matrix_35b.py on local main 166dd51 (205 accelerator records, 218 supply records, 3,367 benchmarks — #190 on top of b739d00). The selection rule still yields 152 qualifying / 43 priced / 109 unpriced (68 of the unpriced fit 35B-A3B, 41 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. #190 adds no accelerator and no purchase price. What it adds is the depth sweep §4b said the 3090 Ti did not have, on a part the table already prices at rank 10:

• RTX 3090 Ti, rank 10, now has the 4k cell, and the new page disagrees with the old one. NVIDIA GeForce RTX 3090 Ti (24 GB, 1,008 GB/s, 35B ceiling 1,008/20 = 50.4 tok/s (derived), $950 → $18.85/tok-s (derived) from Used RTX 3090 Ti 24GB - Oct-2026 band). Hardware Corner's March 2026 per-card llama.cpp table measures Qwen3 30B-A3B Q4_K, batch 1, at 166.9 / 121.9 / 92.0 tok/s for 4k / 16k / 32k (RTX 3090 Ti llama.cpp: Qwen3 30B-A3B (MoE) decode at 4k (Hardware Corner per-card page), RTX 3090 Ti llama.cpp: Qwen3 30B-A3B (MoE) decode at 16k (Hardware Corner per-card page), RTX 3090 Ti llama.cpp: Qwen3 30B-A3B (MoE) decode at 32k (Hardware Corner per-card page)). Against the 50.4 floor that is 3.3× / 2.4× / 1.8× (derived) (166.9/50.4 = 3.31, 121.9/50.4 = 2.42, 92.0/50.4 = 1.83). The 16k cell already in §4b is 98.25 (Hardware Corner: RTX 3090 Ti, llama.cpp CUDA, Qwen3 30B-A3B (MoE) Q4_K XL GGUF, decode 98.25 t/s @16k ctx) and the 32k cell already cited is 91.97 (Hardware Corner: RTX 3090 Ti 24GB, llama.cpp CUDA, Qwen3 30B-A3B (MoE) Q4_K XL GGUF, decode 91.97 t/s @32k ctx). The new 32k number is 0.03 tok/s off that one; the new 16k number is 23.65 tok/s higher. Both pages are kept, neither is averaged, and the new rows are status: contested in the records themselves because the same lab's ranking page and per-card page disagree on shared cells. At the new 4k cell the $950 sticker is $5.69 per measured tok/s (derived, 950/166.9). The 3090's own verified 4k cell is 179.0 on the same $950, which is $5.31 (derived, 950/179.0) — the Ti does not take the short-context measured lead, and the two 4k pages are different cards, so that gap is not a protocol split.
• The dense 32B row on the same card lands under the 27B ceiling. Qwen3 32B Q4_K decodes at 38.0 tok/s at 4k and 32.6 at 16k (RTX 3090 Ti llama.cpp: Qwen3 32B decode at 4k (Hardware Corner per-card page), RTX 3090 Ti llama.cpp: Qwen3 32B decode at 16k (Hardware Corner per-card page)). The 27B ceiling is 1,008/16 = 63.0 tok/s (derived), so those cells are 38.0/63.0 = 0.60× and 32.6/63.0 = 0.52× (derived). A 32B file is larger than the 16 GB denominator, so both ratios are a lower bound on how far under the floor a dense Q4 read sits — the same shape the R9700 note above records, not a new rank.
• The rest of #190 is the wrong byte count for this table. gpt-oss 20B MXFP4 on the 3090 Ti (160.3 at 4k down to 61.5 at 128k) and on the RTX 5070 Ti (156.0 at 4k down to 66.6 at 128k), plus the Qwen3 8B and 14B depth rows, are smaller than the 20 GB denominator. None of them enters the §4b proxy set and none moves a $/tok-s rank. The 5070 Ti still has no 30B-A3B cell beyond the 111.11 already quoted.

Refresh note (docs/fit-matrix-oct14, Oct-17, this pass). Re-ran python tools/fit_matrix_35b.py on local main d8214c6 (205 accelerator records, 218 supply records, 3,305 benchmarks — #186, #187 and #188 on top of 2c7fa57). The selection rule still yields 152 qualifying / 43 priced / 109 unpriced (68 of the unpriced fit 35B-A3B, 41 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. #186 is a 4B model record and #187 is a services price; neither touches an accelerator, a supply row, or a benchmark. #188 adds no accelerator and no purchase price. What it adds is the first current-generation dense 27B decode on a part §4 already lists and cannot price:

• Radeon AI PRO R9700, unpriced, fits 35B and 27B, not 70B, and the dense row lands under the 27B ceiling. AMD Radeon AI PRO R9700 is 32 GB GDDR6 at 640 GB/s (same bus as AMD Radeon RX 9070 XT, which is priced and does not fit). 35B ceiling 640/20 = 32.0 tok/s (derived); 27B ceiling 640/16 = 40.0 tok/s (derived). Flags stay Y/Y/- (32 ≥ 24, 32 ≥ 20, 32 < 48). local-llm-benchmarks.dev measures Qwen3.8-27B UD-Q4_K_XL, concurrency 1, depth 0, a 2048-token prompt and 256 generated tokens, TTFT excluded, llama.cpp 0.5.0-dev. ROCm 10.0 decodes at 30.66 tok/s (R9700 1-GPU ROCm Qwen3.8-27B UD-Q4_K_XL depth-0 decode; repeat means 30.826 and 30.485) and prefills at 3,048.78 tok/s (R9700 1-GPU ROCm Qwen3.8-27B UD-Q4_K_XL depth-0 prefill). Vulkan on the same host and the same file decodes at 28.39 (R9700 1-GPU Vulkan RADV Qwen3.8-27B UD-Q4_K_XL depth-0 decode) and prefills at 1,016.06 (R9700 1-GPU Vulkan RADV Qwen3.8-27B UD-Q4_K_XL depth-0 prefill). Against the 27B ceiling that is 30.66/40.0 = 0.77× and 28.39/40.0 = 0.71× (derived). Under the floor, which is what a dense Q4 read looks like on this bus — the same shape the 3090's Qwen3.5-27B Q4_K cell already shows at 0.57× (33.5/58.5), on a different card and a different protocol, so the two ratios are not a ranking. Dividing the same ROCm decode by the 20 GB 35B denominator instead gives 30.66/32.0 = 0.96× (derived), and that division is the wrong byte count for a 27B file; it is not an A3B measurement and it does not enter §4b. The older rows on this card stay where they were: Llama-2-7B Q4_0 tg128 is a small-model calibration (Vulkan 145.67 no-flash-attention, ROCm 97.98 with flash attention), and the Q1_0 Bonsai-27B row is a 3.53 GiB file, not this Q4. No street price, so the card cannot enter §3. The ROCm-vs-Vulkan gap on one file is 30.66 vs 28.39 on decode and 3,048.78 vs 1,016.06 on the 2048-token prefill — a backend gap, not a bandwidth one.

Refresh note (docs/fit-matrix-oct14, Oct-17, prior pass). Re-ran python tools/fit_matrix_35b.py on local main 2c7fa57 (205 accelerator records, 218 supply records, 3,301 benchmarks — #183 and #184 on top of 6180a58). The selection rule now yields 152 qualifying / 43 priced / 109 unpriced (68 of the unpriced fit 35B-A3B, 41 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. #184 is a services price refresh and touches no accelerator, supply, or benchmark. #183 adds no purchase price. What it adds to the gate is four bandwidth-qualified cards that fit nothing in this doc's model class, all -/-/-, all unpriced, so they extend the §4 close rather than §3 or §4:

• Radeon VII, the interesting near-miss. AMD Radeon VII is 16 GB of HBM2 at 1,024 GB/s — the same bus the record's notes give AMD Radeon Pro VII, and the same 35B ceiling, 1024/20 = 51.2 tok/s (derived). 16 GB is under the 24 GB the 35B-A3B fit needs and under the 20 GB the 27B fit needs, so the flags stay -/-/- and the card does not enter §3. The record stores the page's bandwidth row and not its "4 Gbps" speed row: those two disagree (4 Gbps × 4096 bit / 8 = 2,048 GB/s), and the record leaves that disagreement in its notes rather than averaging it. llama.cpp Vulkan #10879, Llama-2-7B Q4_0, tg128 is 101.19 with flash attention (llama.cpp Vulkan AMD Radeon VII llama-2-7b Q4_0 tg128 FA) and 110.86 without (llama.cpp Vulkan AMD Radeon VII llama-2-7b Q4_0 tg128 noFA). Dividing by the 20 GB ceiling — the wrong byte count for a 3.56 GiB model — gives 101.19/51.2 = 2.0× and 110.86/51.2 = 2.2× (derived). That is a small-model calibration, not a claim the card decodes a 35B-A3B at 100 tok/s; 16 GB cannot hold the 24 GB the fit flag requires. pp512 is 1,059.14 / 1,007.19 (llama.cpp Vulkan AMD Radeon VII llama-2-7b Q4_0 pp512 FA, llama.cpp Vulkan AMD Radeon VII llama-2-7b Q4_0 pp512 noFA).
• Arc A770 16GB, same shape, half the bus. Intel Arc A770 16GB is 16 GB GDDR6 at 560 GB/s, ceiling 560/20 = 28.0 tok/s (derived), also -/-/-. tg128 is 52.56 with flash attention (llama.cpp Vulkan Intel Arc A770 16GB llama-2-7b Q4_0 tg128 FA) and 53.07 without (llama.cpp Vulkan Intel Arc A770 16GB llama-2-7b Q4_0 tg128 noFA) → 52.56/28.0 = 1.9× (derived), same 7B caveat. The 8 GB A770 is a different card and is not this record. pp512 is 1,073.85 / 1,119.68 (llama.cpp Vulkan Intel Arc A770 16GB llama-2-7b Q4_0 pp512 FA, llama.cpp Vulkan Intel Arc A770 16GB llama-2-7b Q4_0 pp512 noFA).
• Vega 64 and Vega 56, 8 GB, fit nothing here. AMD Radeon RX Vega 64 is 8 GB HBM2 at 484 GB/s, ceiling 484/20 = 24.2 tok/s (derived). AMD Radeon RX Vega 56 is 8 GB at 410 GB/s, ceiling 410/20 = 20.5 (derived). Both are under every fit threshold in §2. tg128 is 63.43 / 67.70 on the 64 (llama.cpp Vulkan AMD Radeon RX Vega 64 llama-2-7b Q4_0 tg128 FA, llama.cpp Vulkan AMD Radeon RX Vega 64 llama-2-7b Q4_0 tg128 noFA) and 50.23 / 52.66 on the 56 (llama.cpp Vulkan AMD Radeon RX Vega 56 llama-2-7b Q4_0 tg128 FA, llama.cpp Vulkan AMD Radeon RX Vega 56 llama-2-7b Q4_0 tg128 noFA). 63.43/24.2 = 2.6× and 50.23/20.5 = 2.5× (derived), again against Llama-2-7B, so neither enters §4b.
• TITAN V does not qualify, so it is not one of the 152. NVIDIA TITAN V is 12 GB HBM2 with memory_bandwidth_gbps: null — the product page and the launch note print no bandwidth, and the record refuses to substitute one. Null bandwidth fails the >= 300 arm, and there is no linked purchase-price record at <= $3,000, so the selection rule keeps it out. Its #10879 tg128 of 100.69 / 109.21 (llama.cpp Vulkan NVIDIA TITAN V llama-2-7b Q4_0 tg128 FA, llama.cpp Vulkan NVIDIA TITAN V llama-2-7b Q4_0 tg128 noFA) is therefore not divided into a ceiling this doc does not have a bandwidth for. All twenty new rows are Llama-2-7B Q4_0, so none of them is an A3B measurement and none moves a rank.

Refresh note (docs/fit-matrix-oct14, Oct-17, this pass). Re-ran python tools/fit_matrix_35b.py on local main 6180a58 (200 accelerator records, 218 supply records, 3,281 benchmarks — #181 on top of 2925c06). The selection rule is unchanged and still yields 148 qualifying / 43 priced / 105 unpriced (68 of the unpriced fit 35B-A3B, 37 do not). Every priced cell was re-diffed against the generator: id, VRAM, bandwidth, the three fit flags, the 35B ceiling, the supply id, the price, the derived $/tok-s, and the order all match, so the ranked table does not move and no rank below is renumbered. #181 adds no accelerator and no purchase price.

• The new rows are not the 35B-A3B workload, and they are not cited as measurements here. #181 is 231 third-party llama.cpp rows from geerlingguy/ai-benchmarks issues #36, #39, #40, #45 and #46, covering NVIDIA GeForce RTX 2080 Ti (11 GB, new to this doc), NVIDIA RTX A4000 (16 GB, new to this doc), NVIDIA GeForce RTX 3080 Ti (12 GB), NVIDIA GeForce RTX 3060 12GB (12 GB) and NVIDIA GeForce RTX 4070 Ti (12 GB). All five have vram_gb under 24, so none clears the 35B-A3B fit (20 GB weights + 4 GB KV) or the 70B fit; the 16 GB A4000 also misses the 27B fit (16 + 4). They sit in the 37-row does-not-fit set, not in §3 or §4. The models are TinyLlama 1.1B, Llama-3.2-3B, Llama-2-7B, Llama-2-13B, DeepSeek-R1-Distill-Qwen-14B and gpt-oss-20b, all Q4_K_M — every one of them is the wrong byte count for the 20 GB ceiling, so none enters §4b.

• Do not divide these records into the ceiling until they are re-transcribed. A spot check of the stored decode_tok_s rows against the tables those five issues actually print matches on 10 rows and disagrees on 46. The stored RTX A4000 gpt-oss-20b tg128 is 168.03 (Geerling llama.cpp Vulkan on rtxa4000 (pc host): gpt-oss-20b Q4_K_M tg128 168.03 tok/s); the issue prints 93.55. The stored RTX 3080 Ti number is 167.86 (Geerling llama.cpp Vulkan on rtx3080ti (pc host): gpt-oss-20b Q4_K_M tg128 167.86 tok/s) against a printed 167.47, and the two cards — 448 GB/s and 912 GB/s — were recorded within 0.2 tok/s of each other, which is the signature of a copied sibling table rather than two measurements. The 2080 Ti and 3060 batches have the same shape of error, including rows for a llama-2-7b run issue #36 does not contain. Quoting a ×-over-floor from them would launder a bad transcription into the ranking. Left for a data pass; not fixed in this doc pass.

Refresh note (docs/fit-matrix-oct17, Oct-17, this pass). Re-ran python tools/fit_matrix_35b.py on local main 2925c06 (200 accelerator records, 218 supply records, 3,050 benchmarks — #179 on top of cd120be). The selection rule still yields 148 qualifying / 43 priced / 105 unpriced (68 of the unpriced fit 35B-A3B, 37 do not). The ranked $/tok-s table is unchanged: every priced cell was re-diffed against the generator and matches (price, ceiling, fit flags, supply id, order). #179 adds no accelerator and no purchase price. 48 of the 55 new rows are llama.cpp Vulkan #10879 no-flash-attention Llama-2-7B Q4_0 pp512/tg128 pairs on cards this doc already covers (5090, 4090, 5080, 3090, 4080 Super, 5070 Ti, 4070 family, 5060 Ti, 3060, 3080 10GB, 5090 Laptop, 7900 XTX, 9070 XT, R9700, MI50, A100 80GB PCIe, GB10, M4 Max, Tesla P100/P40/M40). A 7B dense model is the wrong byte count for the 20 GB ceiling, so none of them enters §4b and none moves a rank. The seven that are the workload are a second protocol on a part the table already prices:

• Strix Halo 8060S, rank 37, and the long-context column does not collapse. AMD Radeon 8060S (Ryzen AI Max+ integrated GPU) (128 GB unified, 256 GB/s, 35B ceiling 256/20 = 12.8 tok/s (derived), $1,900 → $148.44/tok-s (derived)). The homelab page's single-run llama-bench (-fa 1 -r 1 --mmap 0) measures Qwen3-30B-A3B UD-Q4_K_XL under Vulkan AMDVLK 2025.Q2.1 at 81.79 tok/s at depth 0 and 85.11 at depth 130560 (Strix Halo 8060S llama.cpp: Qwen3-30B-A3B UD-Q4_K_XL tg128 81.79 tok/s (amdvlk, depth 0), Strix Halo 8060S llama.cpp: Qwen3-30B-A3B UD-Q4_K_XL tg128 85.11 tok/s (amdvlk, depth 130560)). Against the 12.8 floor that is 6.4× and 6.6× (derived) (81.79/12.8 = 6.39, 85.11/12.8 = 6.65). The same build's tuned-ROCm column (ROCm 7.0.0-0, hipBLASLt) is 64.17 at depth 0 and 67.66 at depth 130560 (Strix Halo 8060S llama.cpp: Qwen3-30B-A3B UD-Q4_K_XL tg128 64.17 tok/s (rocm-tuned, depth 0), Strix Halo 8060S llama.cpp: Qwen3-30B-A3B UD-Q4_K_XL tg128 67.66 tok/s (rocm-tuned, depth 130560)) → 5.0× and 5.3× (derived) (64.17/12.8 = 5.01, 67.66/12.8 = 5.29). Depth 130560 is ~128k of primed context, and both columns hold — AMDVLK is slightly faster at the long depth than at zero. That contradicts the 5-run wiki column already on file for the same model on the same box: AMDVLK there is 130.56 at depth 0 and 27.32 at depth 131072 (Strix Halo 8060S (Vulkan AMDVLK (2025.Q2.1)) Qwen3-30B-A3B UD-Q4_K_XL: tg128 at depth 0, Strix Halo 8060S (Vulkan AMDVLK (2025.Q2.1)) Qwen3-30B-A3B UD-Q4_K_XL: tg128 at depth 131072), and the plain ROCm b6702 column already filed is 64.17 at depth 0 collapsing to 4.98 at depth 130560 (Strix Halo 8060S llama.cpp b6702 ROCm: Qwen3-30B-A3B UD-Q4_K_XL tg128 4.98 tok/s (depth 130560)). Same silicon, three protocols, three answers at ~128k (85.11, 27.32, 4.98). The single-run rows are not averaged into the 5-run ones, and they do not replace the 72.35 cell §4b ranks on (lhl 8060S Qwen3-30B-A3B-UD-Q4_K_XL Vulkan tg1 decode, 5.7×) or the 130.56 depth-0 row the §4b prose calls 10.2× (130.56/12.8 = 10.20, (derived)). What they do is bound the claim: the floor stays a floor at 128k on this build, and the collapse §4b warns about is a backend effect, not a bandwidth one. $1,900/85.11 = $22.32 per measured tok/s (derived) at depth 130560 — still above the 3090's $5.31 at its 4k cell, and a different context depth, so it does not re-rank. pp512 on the new AMDVLK column is 741.6 at depth 0 and 755.14 at depth 130560 (Strix Halo 8060S llama.cpp: Qwen3-30B-A3B UD-Q4_K_XL pp512 741.60 tok/s (amdvlk, depth 0), Strix Halo 8060S llama.cpp: Qwen3-30B-A3B UD-Q4_K_XL pp512 755.14 tok/s (amdvlk, depth 130560)); the tuned-ROCm pp512 at depth 130560 is 659.07 (Strix Halo 8060S llama.cpp: Qwen3-30B-A3B UD-Q4_K_XL pp512 659.07 tok/s (rocm-tuned, depth 130560)). The depth-0 tuned pp512 of 651.93 was already filed and was not duplicated.

Refresh note (docs/fit-matrix-oct14, Oct-16, this pass). Re-ran python tools/fit_matrix_35b.py on local main bd282fe (200 accelerator records, 218 supply records, 2,995 benchmarks — #176 and #177 on top of 5b1cadc). The selection rule now yields 148 qualifying / 43 priced / 105 unpriced (68 of the unpriced fit 35B-A3B, 37 do not). The ranked $/tok-s table is unchanged: every priced cell was re-diffed against the generator and matches (price, ceiling, fit flags, supply id, order). #177 is a services telemetry re-read and touches no accelerator, supply, or benchmark. #176 adds no purchase price. What it does add is a bandwidth that was previously null: FuriosaAI RNGD (Renegade) moved from memory_bandwidth_gbps: null / vram_gb: null to 48 GB HBM3 at 1,500 GB/s (vendor product page, cited on the record). 48 ≥ 24, 48 ≥ 20 and 48 ≥ 48, so the fit flags are Y/Y/Y; the 35B ceiling is 1500/20 = 75.0 tok/s (derived), which slots it in §4 between Biren BR100 GPGPU (2-die OAM training accelerator) (80.0) and Huawei Ascend 910B (70.0). It stays unpriced: the only Furiosa supply row, FuriosaAI RNGD (Renegade) - eval-only, no retail channel, has price_usd: null, kind: oem-direct, and its accelerator_ids still point at the Warboy, not the RNGD. The seven new benchmarks are vendor ranges and multi-card aggregates (4× EXAONE 3.5 32B batch-1 decode, 4× Qwen3-32B aggregate TPS, a Hot Chips ~10B range, a GPT-J 6B queries/s figure). None is a single-card batch-1 A3B decode, so none enters §4b and none moves a rank.

Refresh note (docs/fit-matrix-oct16-3090-a3b, Oct-16, this pass). Re-ran python tools/fit_matrix_35b.py on local main 5b1cadc (200 accelerator records, 218 supply records, 2,988 benchmarks — the Phase-C Oct-16 p20 llama.cpp Vulkan #10879 rows for 12 previously uncovered cards, #172, on top of ece538c). The selection rule now yields 147 qualifying / 43 priced / 104 unpriced (67 of the unpriced fit 35B-A3B, 37 do not). The ranked $/tok-s table is unchanged: every priced cell was re-diffed against the generator and matches (price, ceiling, fit flags, supply id, order), and #172 adds no purchase price. What it does add to the gate is five bandwidth-qualified cards that fit nothing in this doc's model class, all -/-/-, all unpriced, so they extend the §4 close rather than the table: AMD Radeon RX 6800 (16 GB, 512 GB/s, 35B ceiling 25.6 tok/s (derived)), NVIDIA GeForce RTX 2070 SUPER and NVIDIA GeForce RTX 3060 Ti (both 8 GB, 448 GB/s, ceiling 22.4 (derived)), AMD Radeon RX 6750 XT (12 GB, 432 GB/s, ceiling 21.6 (derived)) and AMD Radeon RX 6700 XT (12 GB, 384 GB/s, ceiling 19.2 (derived)). The other seven new records miss the gate: AMD Radeon RX 7600 (288 GB/s), AMD Radeon RX 6650 XT (280), AMD Radeon RX 6600 XT (256) and AMD Radeon RX 6600 (224) are under 300 GB/s with no <= $3,000 purchase record, and Intel Arc B570, NVIDIA GeForce GTX 1080 Ti and NVIDIA RTX A2000 12GB have a null bandwidth. The 44 new benchmarks are Llama-2-7B Q4_0, so none of them is an A3B measurement. What this pass actually lands is a hole in §4b: the part the headline calls the honest mainstream buy, NVIDIA GeForce RTX 3090, had a full Qwen3-30B-A3B Q4_K depth sweep in the corpus since #147 and was never cited. The 4090, 6000 Ada and A6000 sweeps from the same ingest were in the same state. None of them moves a rank — they are the same lab, the same model, a longer context than the one cell §4b already quoted.

• RTX 3090, rank 11, now has the sweep. NVIDIA GeForce RTX 3090 (24 GB, 936 GB/s, 35B ceiling 46.8 tok/s (derived), $950 → $20.30/tok-s (derived)). Hardware Corner's March 2026 llama.cpp table measures Qwen3 30B A3B (Q4_K), batch 1, at 179.0 / 98.3 / 58.8 / 32.8 tok/s for 4k / 16k / 32k / 64k (Hardware Corner Mar 2026: RTX 3090 Qwen3 30B A3B (Q4_K) decode @ 4k ctx, Hardware Corner Mar 2026: RTX 3090 Qwen3 30B A3B (Q4_K) decode @ 16k ctx, Hardware Corner Mar 2026: RTX 3090 Qwen3 30B A3B (Q4_K) decode @ 32k ctx, Hardware Corner Mar 2026: RTX 3090 Qwen3 30B A3B (Q4_K) decode @ 64k ctx). Against the 46.8 floor that is 3.8× / 2.1× / 1.3× / 0.70× (derived) (179.0/46.8 = 3.82, 98.3/46.8 = 2.10, 58.8/46.8 = 1.26, 32.8/46.8 = 0.70). The floor is crossed between 32k and 64k — the same context shape §4b already records on the PRO 5000, just one bucket earlier. A second 4k cell on the same card, same quant, is 153.6 (NVIDIA RTX 3090 llama.cpp: Qwen3 30B A3B (Q4_K) token generation at 4k context, 153.6/46.8 = 3.3× (derived)); the two pages do not agree at 4k, so both are kept and neither is averaged. At the 179.0 cell the $950 sticker is $5.31 per measured tok/s (derived, 950/179.0), under the 3090 Ti's $9.67 at its 16k cell — different context depths, so that is not a re-rank, it is why the 3090 now leads the measured cheap-fit read at short context and loses it once the KV is deep.
• RTX 4090, same sweep, higher multiplier. NVIDIA GeForce RTX 4090 already had the 16k cell at 109.57 (2.2×). The March table's other depths are 207.3 / 139.2 / 105.1 / 68.2 at 4k / 16k / 32k / 64k (Hardware Corner Mar 2026: RTX 4090 Qwen3 30B A3B (Q4_K) decode @ 4k ctx, Hardware Corner Mar 2026: RTX 4090 Qwen3 30B A3B (Q4_K) decode @ 16k ctx, Hardware Corner Mar 2026: RTX 4090 Qwen3 30B A3B (Q4_K) decode @ 32k ctx, Hardware Corner Mar 2026: RTX 4090 Qwen3 30B A3B (Q4_K) decode @ 64k ctx) against the 50.4 ceiling → 4.1× / 2.8× / 2.1× / 1.4× (derived) (207.3/50.4 = 4.11, 139.2/50.4 = 2.76, 105.1/50.4 = 2.09, 68.2/50.4 = 1.35). The 16k cell (139.2) is a different page from the 109.57 already in §4b; both stand. The card never crosses the floor inside 64k. $2,000 / 207.3 = $9.65 per measured tok/s (derived) at 4k — essentially the 3090 Ti's 16k figure, at a quarter of the context.
• RTX 6000 Ada and RTX A6000, unpriced, same shape. NVIDIA RTX 6000 Ada Generation (48 GB, 960 GB/s, ceiling 48.0 (derived), §4) goes 199.6 / 120.1 / 74.3 / 40.8 at 4k / 16k / 32k / 64k (Hardware Corner Mar 2026: RTX 6000 Ada Qwen3 30B A3B (Q4_K) decode @ 4k ctx, Hardware Corner Mar 2026: RTX 6000 Ada Qwen3 30B A3B (Q4_K) decode @ 16k ctx, Hardware Corner Mar 2026: RTX 6000 Ada Qwen3 30B A3B (Q4_K) decode @ 32k ctx, Hardware Corner Mar 2026: RTX 6000 Ada Qwen3 30B A3B (Q4_K) decode @ 64k ctx) → 4.2× / 2.5× / 1.5× / 0.85× (derived) (199.6/48.0 = 4.16, 120.1/48.0 = 2.50, 74.3/48.0 = 1.55, 40.8/48.0 = 0.85). The floor is crossed between 32k and 64k, same as the 3090. NVIDIA RTX A6000 (48 GB, 768 GB/s, ceiling 38.4) already had 75.35 at 16k; the 4k cell is 131.2 (NVIDIA RTX A6000 llama.cpp: Qwen3 30B A3B (Q4_K) token generation at 4k context, 131.2/38.4 = 3.4× (derived)). Neither has a purchase price, so neither enters §3.
• Two siblings, not a new rank. The 3090 Ti's 32k cell is 91.97 (Hardware Corner: RTX 3090 Ti 24GB, llama.cpp CUDA, Qwen3 30B-A3B (MoE) Q4_K XL GGUF, decode 91.97 t/s @32k ctx, 91.97/50.4 = 1.8× (derived)), next to the 98.25 already quoted at 16k — the Ti does not have the 3090's 4k cell, so the two cards are not comparable at short context from this corpus. The 5090's second 4k page is 226.1 (NVIDIA RTX 5090 llama.cpp: Qwen3 30B A3B (Q4_K) token generation at 4k context, 226.1/89.6 = 2.5× (derived)) beside the 234.3 already in §4b; same quant, same depth, two pages, 4% apart. Not averaged.
• Dense 27B on the same 3090, under the floor, as the ceiling predicts. Qwen3.5 27B (Q4_K) decodes at 33.5 tok/s at 4k and 28.8 at 64k (NVIDIA RTX 3090 llama.cpp: Qwen3.5 27B (Q4_K) token generation at 4k context, NVIDIA RTX 3090 llama.cpp: Qwen3.5 27B (Q4_K) token generation at 64k context) against the 27B ceiling of 936/16 = 58.5 tok/s (derived) → 33.5/58.5 = 0.57× at 4k (derived). That is the dense half-the-floor §4b already records on the PRO 5000, now on the card this doc actually recommends. The MoE-vs-dense gap on one card is 179.0 vs 33.5 at the same 4k bucket — the reason the 35B-A3B column, not the 27B column, is what a 24 GB card is for. The named 35B target in MXFP4 on the same card is 111.2 at 4k (NVIDIA RTX 3090 llama.cpp: Qwen3.5 35B (MXFP4) token generation at 4k context, 111.2/46.8 = 2.4× (derived)), below the Q4_K 30B-A3B cell because MXFP4 is a different byte count, not because the card got slower.
• Arc Pro B60, priced, fits, and the measured 35B-A3B is slow. Intel Arc Pro B60 (rank 16, 24 GB, 456 GB/s, ceiling 22.8 (derived), $650 → $28.51/tok-s (derived)) has a SYCL llama.cpp row of Qwen3.5-35B-A3B at Q8 — not Q4 — decoding at 8.3 tok/s (Arc Pro B60 24GB Qwen3.5-35B-A3B Q8 token-gen, SYCL (Level1Techs)). 8.3/22.8 = 0.36× the Q4 floor (derived). Q8 reads about twice the bytes the floor assumes, so the ratio is a lower bound on a Q4 run and still well under it; the record's own note calls it a capacity win, not a speed win. It does not enter the §4b short-context table: wrong quant, and one number with no depth. $650/8.3 = $78.31 per measured tok/s (derived), an order of magnitude worse than the 3090's 4k cell.

Refresh note (docs/fit-matrix-oct14, Oct-16, prior pass). Re-ran python tools/fit_matrix_35b.py on local main a2e890e (188 accelerator records, 218 supply records, 2,944 benchmarks — the Phase-C Oct-16 p19 official DGX Spark llama-bench rows, #169, on top of 1b2f366). The selection rule still yields 142 qualifying / 43 priced / 99 unpriced (67 of the unpriced fit 35B-A3B, 32 do not). The ranked $/tok-s table is unchanged: every priced cell was re-diffed against the generator and matches (price, ceiling, fit flags, supply id, order). #169 adds no purchase price and no accelerator. What it does add is the first maintainer llama.cpp decode series on a part this doc already treats as outside the gate, and the model is the 30B-A3B class:

• DGX Spark GB10, measured, and it does not qualify. NVIDIA GB10 Grace Blackwell (DGX Spark) is the same shape as the Jetson AGX Thor row already in the note below: 128 GB unified LPDDR5x at 273 GB/s. That clears the 24 GB fit bar and misses the 300 GB/s bandwidth floor, and there is no purchase-price supply record at <= $3,000, so the selection rule keeps it out of the 142. The official ggml table (build 11fb327bf, CUDA, flash attention, batch 1) runs Qwen3-Coder-30B-A3B-Instruct Q8_0 — 30.25 GiB on disk, not the 20 GB Q4 the ceiling assumes — at tg32 61.06 tok/s with no prior context (ggml CUDA on DGX Spark GB10: Qwen3-Coder-30B-A3B Q8_0 tg32 d0 (b7941)), then 54.77 / 48.02 / 40.23 / 30.21 after 4k / 8k / 16k / 32k of depth (ggml CUDA on DGX Spark GB10: Qwen3-Coder-30B-A3B Q8_0 tg32 d4096 (b7941), ggml CUDA on DGX Spark GB10: Qwen3-Coder-30B-A3B Q8_0 tg32 d8192 (b7941), ggml CUDA on DGX Spark GB10: Qwen3-Coder-30B-A3B Q8_0 tg32 d16384 (b7941), ggml CUDA on DGX Spark GB10: Qwen3-Coder-30B-A3B Q8_0 tg32 d32768 (b7941)). The same arithmetic the ceiling column uses gives 273/20 = 13.7 tok/s (derived), so the depth-0 row lands at 61.06/13.7 = 4.5× that floor (derived) and the 32k row at 30.21/13.7 = 2.2× (derived). Q8 reads more bytes than the Q4 the floor assumes, so 4.5× is a lower bound on how far a Q4 A3B would sit above it, not a Q4 measurement. pp2048 at depth 0 is 2,986.97 tok/s (ggml CUDA on DGX Spark GB10: Qwen3-Coder-30B-A3B Q8_0 pp2048 d0 (b7941)). It does not enter §3 or §4b: no price, and the part is outside the selection rule. The bandwidth match with Thor is exact (both 273 GB/s) and the decode is not — Thor's third-party Qwen3.6-35B-A3B FP8 row is 14.6 tok/s, this Q8_0 A3B row is 61.06 — different quant, different engine, same bus.
• The same box, three other MoEs, none of them the 35B-A3B target. gpt-oss-20b MXFP4 decodes at 83.43 tg32 (ggml CUDA on DGX Spark GB10: gpt-oss-20b MXFP4 tg32 d0 (b7941), 11.27 GiB), 83.43/13.7 = 6.1× the 20 GB ceiling (derived) — the wrong byte count for a 20B MXFP4, same caveat the RTX 4080/5080 note already uses. gpt-oss-120b MXFP4 is 58.72 (ggml CUDA on DGX Spark GB10: gpt-oss-120b MXFP4 tg32 d0 (b7941), 59.02 GiB) → 4.3× (derived), and it is the model that actually uses the 128 GB pool. GLM-4.7-Flash Q8_0, which llama-bench labels deepseek2 30B.A3B, is 48.68 (ggml CUDA on DGX Spark GB10: GLM-4.7-Flash Q8_0 tg32 d0 (b7941), 29.65 GiB) → 3.6× (derived). None of the three is a Q4 35B-A3B, so none enters §4b. The earlier third-party gpt-oss-120b CUDA cell on this part stays where it was (52.87, a different build); the maintainer number is 58.72.

Refresh note (docs/fit-matrix-oct14, Oct-16, prior pass). Re-ran python tools/fit_matrix_35b.py on local main 2e37ebe (188 accelerator records, 218 supply records, 2,884 benchmarks — the Phase-C Oct-16 p18 llama.cpp #15013 rows, #167, on top of ed08dce). The selection rule now yields 142 qualifying / 43 priced / 99 unpriced (67 of the unpriced fit 35B-A3B, 32 do not). The ranked $/tok-s table is unchanged: every priced cell was re-diffed against the generator and matches (price, ceiling, fit flags, supply id, order). #167 adds no purchase price. What it does add is four accelerator records, all of which clear the 8 GB / 300 GB/s gate and none of which has a supply row, plus the first llama.cpp Llama-2-7B Q4_0 tg128 cell on a part §4 already listed:

• RTX 4090 D, new, unpriced, fits 35B/27B and not 70B. NVIDIA GeForce RTX 4090 D (China, AD102-250) is the China-only AD102-250 cut: 24 GB, the same 1,008 GB/s as NVIDIA GeForce RTX 4090, TGP 425 W instead of 450 W. 35B ceiling 1,008/20 = 50.4 tok/s (derived) — identical to the priced 4090 at rank 22. It slots into §4 between the MI50/MI60 pair (51.2) and the RTX 6000 Ada (48.0). The scoreboard row is Llama-2-7B Q4_0, not an A3B model, so it does not enter §4b: tg128 191.57 tok/s (llama.cpp CUDA on nvidia-rtx-4090-d: Llama-2-7B Q4_0 FA-on tg128 decode tok/s (scoreboard #15013)) and pp512 12,506.97 (llama.cpp CUDA on nvidia-rtx-4090-d: Llama-2-7B Q4_0 FA-on pp512 prefill tok/s (scoreboard #15013)). Dividing the 7B decode by the 20 GB 35B denominator — the only ceiling this doc computes, and the wrong byte count for a 3.56 GiB model — gives 191.57/50.4 = 3.80× (derived). That ratio is a small-model calibration, not a claim the card decodes a 35B-A3B at 192 tok/s. No street price, so it cannot take the 4090's rank.
• RTX 5090 Laptop, new, unpriced, same fit shape. NVIDIA GeForce RTX 5090 Laptop GPU (GB203, 24 GB) is a GB203 mobile part, 24 GB GDDR7, 896 GB/s, TGP up to 175 W. 35B ceiling 896/20 = 44.8 tok/s (derived), tied with NVIDIA RTX PRO 4500 Blackwell Workstation Edition (same bus, same bandwidth, a 32 GB workstation card). It does not fit 70B. tg128 158.14 (llama.cpp CUDA on nvidia-rtx-5090-laptop: Llama-2-7B Q4_0 FA-on tg128 decode tok/s (scoreboard #15013)), pp512 7,641.89 (llama.cpp CUDA on nvidia-rtx-5090-laptop: Llama-2-7B Q4_0 FA-on pp512 prefill tok/s (scoreboard #15013)). 158.14/44.8 = 3.53× the 20 GB ceiling (derived), same caveat: the model is Llama-2-7B. The scoreboard prints a 384-bit bus; the accelerator record keeps the 256-bit spec and does not trust that width. Not a desktop 5090 (1,792 GB/s, rank 12) and not priced as a bare GPU — the part ships inside a laptop.
• Quadro RTX 8000 and Quadro RTX 6000, new, unpriced, same 672 GB/s. Turing TU102, not the RTX 6000 Ada and not the RTX PRO 6000 Blackwell. NVIDIA Quadro RTX 8000 (Turing, 48 GB) is 48 GB so it fits 70B (Y/Y/Y); NVIDIA Quadro RTX 6000 (Turing, 24 GB) is 24 GB and does not (Y/Y/-). Both ceiling at 672/20 = 33.6 tok/s (derived), tied with NVIDIA RTX PRO 4000 Blackwell. The 8000's extra 24 GB buys the stretch model and buys no bandwidth. tg128 is 91.88 on the 8000 (llama.cpp CUDA on nvidia-quadro-rtx-8000: Llama-2-7B Q4_0 FA-on tg128 decode tok/s (scoreboard #15013)) and 93.11 on the 6000 (llama.cpp CUDA on nvidia-quadro-rtx-6000: Llama-2-7B Q4_0 FA-on tg128 decode tok/s (scoreboard #15013)) — within 2% of each other, which is what equal bandwidth predicts. 93.11/33.6 = 2.77× and 91.88/33.6 = 2.73× (derived), again against a 7B model, so neither enters §4b. pp512 is 2,934.33 and 2,941.85 (llama.cpp CUDA on nvidia-quadro-rtx-8000: Llama-2-7B Q4_0 FA-on pp512 prefill tok/s (scoreboard #15013), llama.cpp CUDA on nvidia-quadro-rtx-6000: Llama-2-7B Q4_0 FA-on pp512 prefill tok/s (scoreboard #15013)).
• RTX PRO 4500 Blackwell, already in §4, now has its tg128 cell. NVIDIA RTX PRO 4500 Blackwell Workstation Edition (32 GB, 896 GB/s, ceiling 44.8 (derived), unpriced). The pp512 sibling was already stored; #167 adds tg128 168.9 (llama.cpp CUDA on nvidia-rtx-pro-4500-blackwell: Llama-2-7B Q4_0 FA-on tg128 decode tok/s (scoreboard #15013)). 168.9/44.8 = 3.77× the 20 GB ceiling (derived). Same Llama-2-7B protocol as the three new cards, so the four tg128 numbers are comparable to each other and not to the §4b A3B column. 168.9 vs the 5090 Laptop's 158.14 on the same 896 GB/s is a power-cap gap (workstation card vs a 95–175 W mobile SKU), not a bandwidth gap.

Refresh note (docs/fit-matrix-oct14, Oct-16, prior pass). Re-ran python tools/fit_matrix_35b.py on local main 1c2a269 (184 accelerator records, 218 supply records, 2,876 benchmarks — the Phase-C Oct-16 p17 tt-metal matrix rows, #165, on top of 3438151). The selection rule still yields 138 qualifying / 43 priced / 95 unpriced (63 of the unpriced fit 35B-A3B, 32 do not). The ranked $/tok-s table is unchanged: every priced cell was re-diffed against the generator and matches (price, ceiling, fit flags, supply id, order). #165 adds no purchase price and no accelerator. What it does add is the first tt-metal decode numbers on three parts this doc already ranks, and none of them is an A3B-class model, so none of them enters §4b or moves a rank:

• Wormhole n300, rank 24, and the only one of the three that fits 35B-A3B. Tenstorrent Wormhole n300 (24 GB, 576 GB/s, 35B ceiling 28.8 tok/s (derived), $1,449 → $50.31/tok-s (derived)). The vendor's own tt-metal matrix measures Llama-3.2-11B-Vision at 18.1 tok/s per user, TP=2 across both chips, batch 16, input length 128 (Wormhole n300 tt-metal: Llama-3.2-11B-Vision TP2 batch-16 tok/s per user; aggregate 289.6 = 18.1 × 16, not a per-request rate). TTFT on that row is 714 ms (Wormhole n300 tt-metal: Llama-3.2-11B-Vision TP2 batch-16 TTFT). Dividing the per-user figure by the 20 GB 35B ceiling — the only ceiling this doc computes, and the wrong byte count for an 11B vision model at batch 16 — gives 18.1/28.8 = 0.63× (derived). That is under the floor, which is what a concurrent per-user rate on a smaller model looks like against a batch-1 dense-equivalent denominator. It does not re-rank the row.
• Wormhole n150, rank 30, does not fit 35B-A3B. Tenstorrent Wormhole Tensix (n150 / n300) is 12 GB / 288 GB/s, ceiling 14.4 tok/s (derived), $999 → $69.38/tok-s (derived), fits -/-/-. The vendor matrix's LLM row is Mamba-2.8B at 14.1 tok/s per user at batch 32 (Wormhole n150 tt-metal: Mamba-2.8B batch-32 tok/s per user; aggregate 451.2). 14.1/14.4 = 0.98× the 20 GB ceiling (derived) — a coincidence of denominators, not evidence the 12 GB card holds a 24 GB model. The speech row on the same card, Whisper distil-large-v3 at 105.0 tok/s batch 1 (Wormhole n150 tt-metal: Whisper distil-large-v3 batch-1 tok/s per user, TTFT 163 ms Wormhole n150 tt-metal: Whisper distil-large-v3 batch-1 TTFT), is an encoder-decoder audio model, not a 35B decode, so it is not divided into the ceiling.
• Blackhole p150, rank 25, fits 35B/27B and not 70B, and the new row is speech. Tenstorrent Blackhole Tensix (p100a / p150a / p150b) (32 GB, 512 GB/s, ceiling 25.6 tok/s (derived), $1,399 → $54.65/tok-s (derived)). Whisper distil-large-v3 decodes at 263.4 tok/s batch 1 (Blackhole p150 tt-metal: Whisper distil-large-v3 batch-1 tok/s per user, TTFT 63 ms Blackhole p150 tt-metal: Whisper distil-large-v3 batch-1 TTFT). The record's own notes flag the Blackhole software as still under active optimization, and a speech model is not the 35B-A3B workload, so 263.4 is not a measured override of the 25.6 tok/s floor and does not enter §4b.

Refresh note (docs/fit-matrix-oct14, Oct-16, prior pass). Re-ran python tools/fit_matrix_35b.py on local main 18b33eb (184 accelerator records, 218 supply records, 2,869 benchmarks — the Phase-C Oct-16 p16 Hardware Corner long-context cells, #163, on top of d4b79fb). The selection rule still yields 138 qualifying / 43 priced / 95 unpriced (63 of the unpriced fit 35B-A3B, 32 do not). The ranked $/tok-s table is unchanged: every priced cell was re-diffed against the generator and matches (price, ceiling, fit flags, supply id, order). #163 adds no purchase price and no accelerator. What it does add, and what this pass lands, is the rest of the context sweep on one §4 part and the first gpt-oss-20B depth rows on two parts that do not fit 35B-A3B:

• RTX PRO 5000 Blackwell, still unpriced, still in §4. NVIDIA RTX PRO 5000 Blackwell Workstation Edition (48 GB, 1,344 GB/s, 35B ceiling 67.2 tok/s (derived)). The §4b row already quotes the short-context end of the same lab: 188.8 tok/s at 4k (RTX PRO 5000 Blackwell llama.cpp: Qwen3 30B-A3B (MoE) decode at 4k (Hardware Corner), 2.8×) and 148.5 at 16k (RTX PRO 5000 Blackwell llama.cpp: Qwen3 30B-A3B (MoE) decode at 16k (Hardware Corner), 2.2×). #163 extends that one Qwen3-30B-A3B series: 123.2 at 32k (RTX PRO 5000 Blackwell llama.cpp: Qwen3 30B-A3B (MoE) tg at 32k (Hardware Corner), 123.2/67.2 = 1.8× (derived)), 91.9 at 64k (RTX PRO 5000 Blackwell llama.cpp: Qwen3 30B-A3B (MoE) tg at 64k (Hardware Corner), 1.4×), 61.4 at 128k (RTX PRO 5000 Blackwell llama.cpp: Qwen3 30B-A3B (MoE) tg at 128k (Hardware Corner), 0.91×), 37.0 at 256k (RTX PRO 5000 Blackwell llama.cpp: Qwen3 30B-A3B (MoE) tg at 256k (Hardware Corner), 0.55×). The floor is crossed between 64k and 128k — a context effect, the same shape §4b already flags on the PRO 6000's 256k row, not a reason to reorder the bandwidth ranking. The dense cross-check on the same card: Qwen3-32B decode is 42.0 / 36.4 / 28.9 at 16k/32k/64k (RTX PRO 5000 Blackwell llama.cpp: Qwen3 32B tg at 16k (Hardware Corner), RTX PRO 5000 Blackwell llama.cpp: Qwen3 32B tg at 32k (Hardware Corner), RTX PRO 5000 Blackwell llama.cpp: Qwen3 32B tg at 64k (Hardware Corner)) against the 27B ceiling of 1,344/16 = 84.0 tok/s (derived), so 42.0/84.0 = 0.50× at 16k (derived) — the same half-the-floor the prior refresh already recorded for the Ollama qwen2.5:32b rows, and it keeps falling as context grows. The stretch 70B was already on the card at 24.0 (4k) and 22.4 (16k) against a 33.6 tok/s ceiling; #163 does not extend it. It does not enter §3: still no purchase price.
• RTX 4080 and RTX 5080, gpt-oss-20B, and neither fits 35B-A3B. Both are 16 GB, so both are in the 32-row near-miss list, not §3 or §4. NVIDIA GeForce RTX 4080 (16 GB) (716.8 GB/s, 35B ceiling 35.8 tok/s (derived)) now has a gpt-oss-20B MXFP4 sweep at 120.2 / 106.0 / 81.3 tok/s for 16k/32k/64k (RTX 4080 llama.cpp: gpt-oss 20B (MXFP4) tg at 16k (Hardware Corner), RTX 4080 llama.cpp: gpt-oss 20B (MXFP4) tg at 32k (Hardware Corner), RTX 4080 llama.cpp: gpt-oss 20B (MXFP4) tg at 64k (Hardware Corner)). Dividing by the 20 GB 35B denominator — the only ceiling this doc computes, and it is the wrong byte count for a 20B MXFP4 — gives 120.2/35.8 = 3.4× at 16k (derived). NVIDIA GeForce RTX 5080 (960 GB/s, ceiling 48.0) does the same at 161.3 / 149.3 / 130.9 for 16k/32k/64k (RTX 5080 llama.cpp: gpt-oss 20B (MXFP4) tg at 16k (Hardware Corner), RTX 5080 llama.cpp: gpt-oss 20B (MXFP4) tg at 32k (Hardware Corner), RTX 5080 llama.cpp: gpt-oss 20B (MXFP4) tg at 64k (Hardware Corner)), 161.3/48.0 = 3.4× at 16k (derived), down from the already-recorded 4k cell of 172.4. Neither card holds the 24 GB the 35B-A3B fit flag requires, so these rows calibrate the roofline, they do not add a rank.

Refresh note (docs/fit-matrix-oct14, Oct-16, prior pass). Re-ran python tools/fit_matrix_35b.py on local main 16ea04a (184 accelerator records, 218 supply records, 2,813 benchmarks — the Phase-C Oct-16 p13 Ollama / Thor backfill, #161, on top of 37c67d2). The selection rule still yields 138 qualifying / 43 priced / 95 unpriced (63 of the unpriced fit 35B-A3B, 32 do not). The ranked $/tok-s table is unchanged: every priced cell was re-diffed against the generator and matches (price, ceiling, fit flags, supply id, order). #161 adds no purchase price. What it does add, and what this pass lands, is the first Qwen3.6-35B-A3B measurement in the corpus plus dense cross-checks on two parts this doc already lists as unpriced:

• Jetson AGX Thor, measured, and it does not qualify. NVIDIA Jetson AGX Thor 128GB is 128 GB unified LPDDR5X at 273 GB/s. That clears the 24 GB fit bar and misses the 300 GB/s bandwidth floor, and there is no purchase-price supply record at <= $3,000, so the selection rule keeps it out of the 138. A third-party SGLang run of Qwen3.6-35B-A3B FP8 decodes at 14.6 tok/s (Jetson AGX Thor SGLang: Qwen3.6-35B-A3B FP8 decode (Multimodalflow), ~36 GB footprint, 24-in/200-out, batch 1). The same arithmetic the ceiling column uses gives 273/20 = 13.7 tok/s (derived), so the FP8 row lands at 14.6/13.7 = 1.07× that floor (derived). That is not a Q4-MoE comparison — FP8 reads roughly twice the bytes of the 20 GB Q4 the floor assumes — and it is the first time the named 35B-A3B target itself, not the Qwen3-30B-A3B stand-in, shows up against a part this doc would otherwise never mention. It does not enter §3 or §4b: no price, and the part is outside the selection rule. Warm TTFT on the same run is 101 ms (Jetson AGX Thor SGLang: Qwen3.6-35B-A3B FP8 TTFT warm (Multimodalflow)); prompt length is unknown, so that number is not a prefill rate.
• RTX PRO 4000 / PRO 5000 Blackwell, dense, not A3B. Still unpriced, still in §4. NVIDIA RTX PRO 4000 Blackwell (24 GB, 672 GB/s, 35B ceiling 33.6 (derived)) has an Ollama qwen2.5:32b eval of 24.6 tok/s (RTX Pro 4000 Blackwell Ollama: Qwen2.5 32B eval rate (Computingforgeeks)). The 27B-dense ceiling on that card is 672/16 = 42.0 tok/s (derived), so 24.6/42.0 = 0.59× (derived) — under the floor, which is what a ~19.4 GB 32B resident looks like against a 16 GB denominator, and the page's own quant is unnamed. NVIDIA RTX PRO 5000 Blackwell Workstation Edition (48 GB) gets the same tag at 42.9 / 42.92 tok/s from two labs (RTX Pro 5000 Blackwell Ollama: Qwen2.5 32B eval rate (Computingforgeeks), RTX PRO 5000 Blackwell Ollama: Qwen2.5 32B eval rate (Database Mart)) against a 27B ceiling of 1,344/16 = 84.0 (derived) → 0.51×. The stretch 70B is now measured on that card too: Ollama llama3.3:70b at 21.42 tok/s (RTX PRO 5000 Blackwell Ollama: Llama 3.3 70B eval rate (Database Mart), 42 GB resident) and qwen2.5:72b at 18.4 (RTX Pro 5000 Blackwell Ollama: Qwen2.5 72B eval rate (Computingforgeeks)), against a 70B ceiling of 1,344/40 = 33.6 (derived) → 0.64× and 0.55×. The 72 GB sibling still has no public row. None of these move §3.

Refresh note (docs/fit-matrix-oct14, Oct-16, prior pass). Re-ran python tools/fit_matrix_35b.py on local main 37c67d2 (184 accelerator records, 218 supply records, 2,801 benchmarks — the llama.cpp Llama-2-7B scoreboard backfill, #159, on top of b7921d5). The selection rule still yields 138 qualifying / 43 priced / 95 unpriced (63 of the unpriced fit 35B-A3B, 32 do not). The ranked $/tok-s table is unchanged: every priced cell was re-diffed against the generator and matches (price, ceiling, fit flags, supply id, order). #159 adds no purchase price and no A3B-class measurement — its rows are Llama-2-7B Q4_0 — so it cannot move this ranking. What this pass does land is one measured A3B row the previous table never cited:

• RTX PRO 5000 Blackwell, measured. NVIDIA RTX PRO 5000 Blackwell Workstation Edition (48 GB, 1,344 GB/s, §4, unpriced) has a third-party Qwen3-30B-A3B Q4_K decode of 188.8 tok/s at 4k (RTX PRO 5000 Blackwell llama.cpp: Qwen3 30B-A3B (MoE) decode at 4k (Hardware Corner)) against a 67.2 tok/s ceiling (derived) — 2.8× the floor (188.8/67.2 = 2.81, (derived)). The same card at 16k is 148.5 (RTX PRO 5000 Blackwell llama.cpp: Qwen3 30B-A3B (MoE) decode at 16k (Hardware Corner)), 2.2×. That is the first measured A3B number for a part this doc lists only as "fits, can't price it," and it sits between the RTX 5090 (234.3) and the RTX PRO 6000 Blackwell (198.7) on the same lab's 4k protocol. The 72 GB sibling still has no public row. It does not enter §3: there is still no bare-card price.

Refresh note (docs/fit-matrix-oct14, Oct-16, prior pass). Re-ran python tools/fit_matrix_35b.py on local main b7921d5 (184 accelerator records, 218 supply records, 2,742 benchmarks). The selection rule now yields 138 qualifying / 43 priced / 95 unpriced (63 of the unpriced fit 35B-A3B, 32 do not). Two things actually moved; the rest of the ranked order did not:

• One new priced row. AMD EPYC 9554 (Genoa, 64-core, 12-channel DDR5 host CPU) (vram_gb null, unified DDR5 host) is joined to the whole 384 GB self-build Used single-socket EPYC 9004 'Genoa' build (12ch DDR5, ~384GB) at $7,500. The 384 GB figure is the configured pool of that supply record (UNIFIED_MEM_GB in the tool), so the Y/Y/Y flag and the sticker describe the same machine. Ceiling 23.0 tok/s (derived) → $325.52/tok-s (derived), rank 42, one slot above the H100 PCIe. The $2,000 CPU-only row is not joined (it prices no RAM).
• Six fit-but-unpriced rows the previous table dropped. NVIDIA RTX PRO 5000 72GB Blackwell Workstation Edition, NVIDIA RTX PRO 5000 Blackwell Workstation Edition, NVIDIA L40 (Ada, 48 GB), NVIDIA RTX A5000, NVIDIA RTX PRO 4000 Blackwell, AMD Radeon PRO V620. They were already in data/accelerators/; §4 now lists them. The near-miss (does-not-fit) count went from 19 to 32 for the same reason — those ids are named in the §4 close, not given their own table.
• A stale generator join was removed before this table was written. The tool briefly joined eBay used Mac Studio M1 Ultra 64GB (Oct 2026) ($1,850) and eBay used Mac Studio M3 Ultra 256GB (Oct 2026) ($6,250) onto records whose vram_gb is 128 and 512. That would have ranked M1 Ultra at $46.25/tok-s and M3 Ultra at $152.63/tok-s for fit flags the sticker does not buy. Both stay dropped; the table keeps the capacity-matched rows ($2,500 / 128 GB and $9,000 / 512 GB).

Ranked order of the other 42 priced rows is unchanged from the note below, including the RTX 5070 Ti measured cell at 111.11 tok/s.

Refresh note (docs/fit-matrix-oct14, Oct-16). Re-ran python tools/fit_matrix_35b.py on local main 069e36f (162 accelerator records, 214 supply records). Selection rule still yields 118 qualifying / 42 priced / 76 unpriced (57 of the unpriced fit 35B-A3B, 19 do not). Ranked $/tok-s order is unchanged from the p6 table already on this tree. The only numeric edit is §4b: the RTX 5070 Ti measured decode cell is restored to the benchmark record's own value, 111.11 tok/s (Hardware Corner: RTX 5070 Ti, llama.cpp CUDA, Qwen3 30B-A3B (MoE) Q4_K XL GGUF, decode 111.11 t/s @16k ctx), not the rounded 111.1 the p6 prose left behind. 2.5× vs the 44.8 tok/s ceiling is unchanged (111.11/44.8 = 2.48, still 2.5× (derived)).

Refresh note (docs/fit-matrix-oct16-p6, Oct-16). Re-run of python tools/fit_matrix_35b.py against local main e128197 (162 accelerator records, 214 supply records, 1,595 benchmarks — the Phase-C Oct-16 p7 benchmark backfill on top of origin/main). The selection rule yields 118 qualifying accelerators, of which 42 have a usable purchase price and 76 are bandwidth-qualified but unpriced (57 of those fit 35B-A3B, 19 do not). The ranked $/tok-s table is unchanged from the Oct-16 docs/fit-matrix-oct14 refresh; what this pass does is land that table on the tree that actually contains the Atlas join and the four §4b rows, and quote two measured values at the record's own precision. Three things moved since the Oct-15 pass-3 doc:

• One new priced row. Huawei Atlas 300I Pro inference card (24 GB, 310-series) (24 GB, 204.8 GB/s) is the card-level record of the same 310P-class silicon as Huawei Ascend 310P (Da Vinci edge inference NPU). Joined at $265 from Taobao used Huawei Atlas 300I Pro (Ascend 310P) 24GB — identical sticker and identical derived ceiling, so the two tie at ranks 13/14.
• Two supply prices were re-read at their recorded low end, which reorders the top of §3. eBay used AMD Instinct MI100 32GB is $720 (the record's buy-target; the prior doc used the $900 top of the same band), so AMD Instinct MI100 moves from rank 5 at $14.66/tok-s to rank 4 at $11.73/tok-s (derived). Used Apple Mac Studio M1 Ultra 128GB is $2,500 (midpoint of the $2,200–3,000 band), so Apple M1 Ultra (Mac Studio, 2022) moves from $65.00 to $62.50/tok-s (derived).
• Capacity-mismatched Apple joins dropped. The prior doc priced Apple M3 Ultra (Mac Studio, 2025) (record vram_gb 512) off the 96 GB Used Apple Mac Studio M3 Ultra 96GB at $3,700. The generator now joins the 512 GB machine (eBay used Mac Studio M3 Ultra 512GB (Oct 2026), $9,000), which is the memory the fit flag is computed against. That moves M3 Ultra from rank 32 ($90.35) to rank 41 ($219.78/tok-s (derived)).

§4 grew from 52 to 57 fit-but-unpriced rows (DGX-1, RTX A6000, A40, RTX 5000 Ada, M1 Max, M2 Max). §4b adds four measured A3B decode rows that landed after the last refresh (RTX PRO 6000 Blackwell, A100 40 GB SXM4, A100 40 GB PCIe, RTX A6000).

Prior refresh (docs/fit-matrix-oct14, third refresh): re-run on the Phase-C Oct-15 pass-3 tree. Ranked table then 41 priced rows; §4 at 52.

Scope, set by the commissioning brief. The target class is a 35B-A3B Mixture-of-Experts model (~3B active parameters at decode), a ~27B dense model, and Llama-3.3-70B at Q4 as the stretch goal. The question is narrow: does it fit, and what is the cheapest $/(tok/s) for the 35B-A3B job. This doc does not try to rank serving quality, concurrency, or prefill — those need the measured-benchmark join in docs/08-cost-per-token.md, not a roofline.

Every number in the tables is either copied verbatim from a record or marked (derived). Derived means the arithmetic is shown below and a reader can re-run it. Nothing here is a measurement.

1. The selection rule (which records got a row)

This build is a re-run of tools/fit_matrix_35b.py on main 81b8382 (236 accelerators, 218 supply, 4,532 benchmarks). The selection rule is unchanged and the counts did not move with #286: 159 qualifying / 43 priced / 116 unpriced (69 of the unpriced fit 35B-A3B, 47 do not). #286 adds a model, not an accelerator and not a purchase price. NVIDIA Nemotron 3.5 Super VL 120B-A12B packs to 55.88 GiB at 4-bit — past the 24 GB bar and past the 48 GB the 70B column tests, inside a 96 GB card. The prior stamp, kept in the note above, was 84256e2 / 4,532 benchmarks. See the refresh note above.

An accelerator record gets a row when all of these hold:

• vram_gb >= 8 — below this there is no useful quantised-7B headroom either.
• memory_bandwidth_gbps >= 300 OR it has a linked supply record with a purchase price <= $3,000.

The <= $3,000 arm exists so that a cheap-but-slow card (the used-Tesla slice) is not filtered out by the bandwidth floor before its price can speak. That rule yields 159 qualifying accelerators, of which 43 have a usable purchase price and 116 are bandwidth-qualified but have no bare-card/system price in data/supply/ (69 of those fit 35B-A3B; 47 qualify and do not).

One exception to the vram_gb gate. A unified_memory part (an APU/SoC whose GPU shares the CPU's memory pool) leaves vram_gb null on purpose — there is no dedicated VRAM to report. The >= 8 gate would silently drop it. For those, tools/fit_matrix_35b.py substitutes the configured memory of the cheapest linked supply record (a whole system, so its RAM is the usable pool). Two records need the override: AMD Radeon 8060S (Ryzen AI Max+ integrated GPU) (Strix Halo, 128 GB configured) and AMD EPYC 9554 (Genoa, 64-core, 12-channel DDR5 host CPU) (the 384 GB DDR5 pool of Used single-socket EPYC 9004 'Genoa' build (12ch DDR5, ~384GB)). The Apple Ultra parts already carry an explicit vram_gb.

2. The three fit columns and the one ceiling

Model footprints used for the fit flags (these are the brief's numbers, kept as stated):

fit = vram_gb >= (weights + headroom), read off each accelerator record's own vram_gb. A Y means resident on a single device — no multi-GPU span, no offload. The 35B-A3B threshold of 24 GB is why every 24 GB card in the table is a Y by exactly zero margin.

The ceiling. tok/s ceiling (derived) = memory_bandwidth_gbps / model_weights_gb. This is the standard decode roofline: at batch-1 each output token must read the resident weights once, so a perfectly bandwidth-bound decode cannot exceed bandwidth / bytes-read. It is a ceiling, not a forecast — real decode lands well under it because of kernel inefficiency, KV reads, and the fact that a MoE reads only its active experts. It is also optimistic for MoE: a 35B-A3B that activates ~3B/token reads closer to the active-expert bytes than the full 20 GB resident, so the same ceiling formula on a dense-equivalent basis overstates the value of a big resident pool for MoE. The doc divides by the full resident weights (20 GB) anyway, which makes the ceiling a conservative lower bound on the achievable rate and keeps the arithmetic identical across the three columns.

How conservative shows up the moment a low-bandwidth part enters the table: AMD Radeon 8060S (Ryzen AI Max+ integrated GPU) (Strix Halo, 256 GB/s shared) derives to a 12.8 tok/s ceiling on the 20 GB column, but the corpus's own measurement of the exact target model — AMD Ryzen AI Max+ Radeon 8060S iGPU (RDNA 3.5) under Ollama: Qwen3.5 35B-A3B MoE generation throughput, 100% GPU offload — records 42.04 tok/s on it, ~3.3× the derived ceiling, because the MoE reads only ~3B active experts per token, not the 20 GB resident. The roofline ranks capex against a dense-equivalent number; it understates real MoE throughput most where bandwidth is scarcest. Treat every ceiling cell as the floor, not the forecast.

$/tok-s (derived) = cheapest purchase price ÷ the 35B-A3B ceiling. It ranks "how little silicon do I buy to move 20 GB of weights." It is a capex-efficiency figure, not a cost-per-token-served figure — it ignores power, the chassis/baseboard several of these parts need (§5), and that $/tok-s for a 14 tok/s card and a 240 tok/s card are not the same product.

A third calibration sweep landed with the Phase-C pass-5 ingest. The hardware-corner 16k-context sweep now holds five third-party measured decode rows on Qwen3-30B-A3B (MoE) Q4_K XL — the closest corpus proxy to the 35B-A3B target — at a ~16,384-token context, batch-1 per-request:

|| part | measured tok/s | derived ceiling (BW ÷ 20 GB) | measured ÷ ceiling | ||---|---|---|---| || NVIDIA GeForce RTX 3090 Ti | 98.25 (Hardware Corner: RTX 3090 Ti, llama.cpp CUDA, Qwen3 30B-A3B (MoE) Q4_K XL GGUF, decode 98.25 t/s @16k ctx) | 50.4 | 1.95× | || NVIDIA GeForce RTX 4090 | 109.57 (Hardware Corner: RTX 4090, llama.cpp CUDA, Qwen3 30B-A3B (MoE) Q4_K XL GGUF, decode 109.57 t/s @16k ctx) | 50.4 | 2.17× | || NVIDIA RTX 6000 Ada Generation | 104.88 (Hardware Corner: RTX 6000 Ada, llama.cpp CUDA, Qwen3 30B-A3B (MoE) Q4_K XL GGUF, decode 104.88 t/s @16k ctx) | 48.0 | 2.19× | || NVIDIA GeForce RTX 4080 (16 GB) | 85.04 (Hardware Corner: RTX 4080, llama.cpp CUDA, Qwen3 30B-A3B (MoE) Q4_K XL GGUF, decode 85.04 t/s @16k ctx) | 35.8 | 2.37× | || NVIDIA GeForce RTX 5070 Ti | 111.11 (Hardware Corner: RTX 5070 Ti, llama.cpp CUDA, Qwen3 30B-A3B (MoE) Q4_K XL GGUF, decode 111.11 t/s @16k ctx) | 44.8 | 2.48× |

Two of the five (3090 Ti, 4090) are also priced rows in §3 — the ranking on them can now be sanity-checked against a measured number: both beat their dense roofline by ~2× at 16k context, corroborating the direction the Strix Halo anchors set (MoE reads ~3B active bytes, not the resident 20 GB). The ordering is not pure bandwidth — the 5070 Ti (896 GB/s) edges the 4090 (1,008 GB/s) at 16k, so generation/architecture effects are real at long context — but every measured value sits above its derived ceiling, i.e. the ceilings in the table remain floors, and the $/tok-s ranking undervalues nothing below a measured anchor. Caveat: the hwcorner sweep measures 30B-A3B, not the 35B-A3B target — the active-parameter ratio is the same shape (~3B active of ~30B vs ~3B of ~35B resident), so the direction transfers, not the exact figure.

3. Ranked: purchase-priced parts by $/tok-s for the 35B-A3B job

Sorted by derived $/tok-s, lowest first. Fits columns are 35B/27B/70B. Only rows with a usable purchase price appear; the ← supply column is the record the price was read from. A Y on Fits is bolded. Every ceiling and every $/tok-s cell is (derived): ceiling = memory_bandwidth_gbps / 20 and $/tok-s = price / ceiling.

* Two rows are unified_memory and the capacity shown is not the record's vram_gb (that field is null). The 8060s Strix Halo 128 GB is the configured LPDDR5x pool of the cheapest linked supply record (New Minisforum MS-S1 MAX 128GB (Strix Halo)). The EPYC 9554 384 GB is the configured DDR5 pool of Used single-socket EPYC 9004 'Genoa' build (12ch DDR5, ~384GB) — a whole self-build, not a bare CPU. The sibling amd-ryzen-ai-max-395-framework-desktop record carries an explicit vram_gb=128 for the same Strix Halo silicon, priced at the Beelink whole-system row. See §1 and §6.

The headline read

• The cheapest fits answer is the used V100 SXM2 — and MI100 just passed the M40. NVIDIA Tesla P100 SXM2 16GB (GP100) at $115 and Intel Arc B580 at $199.99 still rank #1/#2 on $/tok-s and still fit nothing: 16 GB / 12 GB are both -/-/-. They are in the table because a sub-$3,000 price is allowed to speak. The first row that fits 35B-A3B is NVIDIA V100 SXM2 32GB at $500 (rank 3, 45.0 tok/s ceiling (derived)). Re-reading eBay used AMD Instinct MI100 32GB at its recorded $720 low end (was $900) moves AMD Instinct MI100 to rank 4 at $11.73/tok-s (derived) — ahead of the $189 M40, and still the cheapest part with both a 35B Y and >1 TB/s (61.4 tok/s ceiling (derived), faster than a 3090 for less money). All four are traps unless you read §5.
• The honest mainstream answer is still the RTX 3090, matched by its Ti. Both sit at $950 used (Used NVIDIA RTX 3090 24GB - Oct-2026 corrected street price, Used RTX 3090 Ti 24GB - Oct-2026 band) for 24 GB. The Ti's extra 72 GB/s buys a 50.4 vs 46.8 tok/s ceiling (derived) at the same price, so it edges the 3090 (ranks 10 and 11). Both are real PCIe cards with a fan. The AMD Radeon RX 7900 XTX still undercuts both ($875 used, rank 9) but needs the ROCm path.
• New priced row: the Atlas 300I Pro ties the Ascend 310P. Huawei Atlas 300I Pro inference card (24 GB, 310-series) and Huawei Ascend 310P (Da Vinci edge inference NPU) are die-vs-card records of the same 24 GB / 204.8 GB/s part, both at $265, both a 10.2 tok/s ceiling (derived) and $25.88/tok-s (derived). A Y on 35B here means the bytes fit, not that it serves — see §5.
• The stretch goal (70B, needs ≥48 GB) did not get a cheaper usable answer this pass. Cheapest single-accelerator 70B sticker is still the MLU370 at $450 (307 GB/s — a capacity fit, not a usable one), then Gaudi 2 at $2,000 (OAM, §5) and the MI250/X pair at $4,950 (same OAM caveat). The cheapest drop-in PCIe 70B card with >1 TB/s remains NVIDIA RTX PRO 6000 Blackwell Workstation Edition at $8,650 (rank 34, 89.6 tok/s ceiling (derived)). Apple M3 Ultra (Mac Studio, 2025) stays on the 512 GB Studio (eBay used Mac Studio M3 Ultra 512GB (Oct 2026), $9,000, rank 41, $219.78/tok-s (derived)) — the memory the Y/Y/Y flag is computed against. The new row is a host, not a GPU: AMD EPYC 9554 (Genoa, 64-core, 12-channel DDR5 host CPU) at $7,500 for a 384 GB DDR5 self-build (rank 42, 23.0 tok/s ceiling (derived), $325.52/tok-s (derived)). It fits 70B because the pool is 384 GB, and it is slow because 12-channel DDR5 is 460.8 GB/s. Strix Halo at $1,900 still fits 70B on paper and is still bandwidth-capped (§5).

4. Qualified but no purchase price — the "fits, can't price it" list

These records satisfy the bandwidth/fit rule and fit 35B-A3B but have no bare-card or system purchase price in data/supply/ (they may have a rental price in the cloud records — that is a different join, see tools/cost_per_token.py). Listed so a future supply pass can close them; sorted by the 35B ceiling (derived). 69 rows. This pass adds NVIDIA Tesla K80 (24 GB board total, 480 GB/s, ceiling 24.0 (derived), Y/Y/- — dual GK210, and the scoreboard row is one 12 GB GPU; see the refresh note). The prior pass added FuriosaAI RNGD (Renegade) (48 GB, 1,500 GB/s, ceiling 75.0 (derived), Y/Y/Y, no purchase price — see the refresh note). The prior pass added the four #167 records: NVIDIA GeForce RTX 4090 D (China, AD102-250) (1,008 GB/s, tied on ceiling with the priced 4090), NVIDIA GeForce RTX 5090 Laptop GPU (GB203, 24 GB) (896 GB/s, tied with the PRO 4500 Blackwell), NVIDIA Quadro RTX 8000 (Turing, 48 GB) (48 GB, fits 70B) and NVIDIA Quadro RTX 6000 (Turing, 24 GB) (24 GB, does not) — both 672 GB/s, tied with the PRO 4000 Blackwell.

Forty-seven further accelerators qualify on the bandwidth-or-price rule and do not fit 35B-A3B (all unpriced). The highest-bandwidth near-misses, all -/-/- except the two 20 GB parts which fit 27B only: AMD Radeon Pro VII (16 GB, 1024 GB/s), NVIDIA GeForce RTX 5080 (16 GB, 960), NVIDIA GeForce RTX 3080 Ti (12 GB, 912), NVIDIA GeForce RTX 5070 Ti (16 GB, 896), Google Cloud TPU v5e (16 GB, 819), AMD Radeon RX 7900 XT (20 GB, 800, -/Y/-), NVIDIA GeForce RTX 3080 10GB (10 GB, 760), NVIDIA GeForce RTX 4080 SUPER / NVIDIA GeForce RTX 4080 (16 GB), Google Cloud TPU v2 (16 GB, 700), NVIDIA GeForce RTX 5070 (12 GB, 672), NVIDIA GeForce RTX 4070 Ti SUPER, AMD Radeon RX 9070 (16 GB, 640), NVIDIA RTX A4500 (20 GB, 640, -/Y/-), and NVIDIA RTX 4000 Ada Generation (20 GB, 360 GB/s, -/Y/-). Five of the thirty-seven are new with #172 and sit at the bottom of that list, all -/-/-: AMD Radeon RX 6800 (16 GB, 512 GB/s, 35B ceiling 25.6 tok/s (derived)), NVIDIA GeForce RTX 2070 SUPER and NVIDIA GeForce RTX 3060 Ti (both 8 GB, 448 GB/s, 22.4), AMD Radeon RX 6750 XT (12 GB, 432 GB/s, 21.6) and AMD Radeon RX 6700 XT (12 GB, 384 GB/s, 19.2). Four of the forty-one are new with #183 and sit in that same does-not-fit list, all -/-/- and all unpriced: AMD Radeon VII (16 GB, 1,024 GB/s, 35B ceiling 51.2 tok/s (derived) — tied with AMD Radeon Pro VII on bandwidth, and the highest-bandwidth near-miss in the set), Intel Arc A770 16GB (16 GB, 560 GB/s, ceiling 28.0 (derived)), AMD Radeon RX Vega 64 (8 GB, 484 GB/s, ceiling 24.2 (derived)) and AMD Radeon RX Vega 56 (8 GB, 410 GB/s, ceiling 20.5 (derived)). NVIDIA TITAN V shipped in the same ingest and is not one of the 43: its bandwidth is null, so it never clears the gate. Two of the forty-three are new with #206 and sit in that same does-not-fit list, both -/-/- and both unpriced: Intel Arc A750 8GB (8 GB, 512 GB/s, 35B ceiling 25.6 tok/s (derived) — tied with the 16 GB RX 6800 / 6800 XT / 6900 XT on bandwidth) and AMD Radeon Pro W5700 (8 GB, 448 GB/s, ceiling 22.4 tok/s (derived) — tied with the RX 5700 XT and the RTX 2070 SUPER / 3060 Ti / 3070). Four of the forty-seven are new with #214 and sit in that same does-not-fit list, all -/-/- and all unpriced: NVIDIA TITAN Xp (12 GB, 547.7 GB/s, 35B ceiling 27.4 tok/s (derived)), AMD Radeon Instinct MI25 (16 GB, 484 GB/s, ceiling 24.2 (derived) — tied with the RX Vega 64), NVIDIA GeForce RTX 2060 SUPER (8 GB, 448 GB/s, ceiling 22.4 (derived) — tied with the RX 5700 XT and the RTX 2070 SUPER / 3060 Ti / 3070) and NVIDIA Quadro RTX 4000 (8 GB, 416 GB/s, ceiling 20.8 (derived)). The full 159-row list, including these 47, is python tools/fit_matrix_35b.py.

4b. Roofline vs. measured — the A3B-MoE cross-check

The 35B ceiling column divides by the full resident 20 GB, which makes it a dense-equivalent floor for a MoE that reads only ~3B active per token (§2). The Phase-C benchmark backfill landed 22+ third-party decode_tok_s records on real A3B-class MoE models (Qwen3-30B-A3B / Qwen3.5/3.6-35B-A3B, Q4-class quants, batch-1 per-request) — enough to check how far off the floor each part actually lands. measured ÷ ceiling below is derived arithmetic; the measured numbers themselves come from the cited records.

Three reads:

• The ordering is the same at one context depth, and it is not at 4k. The 16k cells this section was built on sit in a ~2–2.6× band and do not reorder the bandwidth ranking. The 4k cells now cited do reorder within a generation: the 3090 at 179.0 (3.8×) and the 6000 Ada at 199.6 (4.2×) both clear the 4090's 16k number and do not clear the 4090's own 4k number (207.3). The RTX 5090 still leads the short-context list (234.3). The RTX PRO 5000 Blackwell cell (188.8 tok/s, 2.8×) still slots between the 5090 and the PRO 6000 on bandwidth and on the 4k measurement, which is what the floor predicts. What the new sweep adds is the depth at which the floor stops being a floor: 3090 and 6000 Ada cross it between 32k and 64k, the PRO 5000 between 64k and 128k.
• The floor is ~2–6× low, and where it is lowest is systematic. The tightest-margin parts (Strix Halo now at 5.7× on its best run, M5 Max and M4 Max at 4.2×) are exactly the bandwidth-starved ones — the MoE reads only ~3B/token, so a 256 GB/s pool sustains ~72 tok/s real where the dense-equivalent floor says 12.8. The discrete GPUs cluster at ~2–2.6× because KV reads and kernel overhead bite closer to the roofline. The engine matters too: the same M5 Max measures 128.0 tok/s under MLX but 89.4 under llama.cpp — a 1.4× swing on one part, wider than some cross-part gaps. Do not use the ceiling column to compare a MoE target across architectures — it ranks $/tok-s correctly only because the error is monotonic in bandwidth.
• The cheapest measured A3B answer depends on the context depth. At the 16k cell this section used to quote, NVIDIA GeForce RTX 3090 Ti at $950 with 98.25 tok/s is still ~$9.67/measured-tok/s (derived), and its 1.9× margin was the worst of the set that had been cited. The 3090's own sweep, now in the table, changes the short-context read and not the deep one: 179.0 tok/s at 4k on the same $950 sticker is $5.31/measured-tok/s (derived, 950/179.0), the cheapest verified fit at that depth, at 3.8× its floor — no longer the worst margin. By 64k the same card is at 32.8 tok/s and under the floor (0.70×), while the 4090 is still above it (68.2, 1.4×) and costs $9.65/measured-tok/s at its 4k cell (derived, 2000/207.3). The Ti now has a 4k cell, and it does not win that comparison: 166.9 tok/s (RTX 3090 Ti llama.cpp: Qwen3 30B-A3B (MoE) decode at 4k (Hardware Corner per-card page)) on the same $950 sticker is $5.69/measured-tok/s (derived, 950/166.9), above the 3090's $5.31 at 179.0. The two pages are different cards, and the Ti's new 16k cell (121.9) disagrees with the 98.25 this paragraph's $9.67 is computed from, so neither page is promoted to a re-rank. Strix Halo's 5.7× multiplier (10.2× on the depth-0 wiki row, which is a different protocol and not the cell this paragraph ranks on) is why AMD Strix Halo is an MoE part, not a dense-70B part - the ~256 GB/s wall caps dense decode at ~5 tok/s calls the MoE target the workload that part is for.

5. Why the cheap rows are cheap — read before buying

The $/tok-s ranking is honest arithmetic on asking prices for the part as listed, and four of the top six are cheap for a reason the price doesn't show. These are the corpus's own gotcha records, not new claims:

• SXM/OAM is not a card. NVIDIA V100 SXM2 32GB (#3), Intel Gaudi 2 (#6), AMD Instinct MI250 and AMD Instinct MI250X (#18/#19) are modules. Per SXM2/SXM4 accelerator modules are bare - they need a carrier/baseboard + heatsink and have no PCIe edge, the SXM discount is a form-factor discount — the module needs a carrier baseboard + heatsink and has no PCIe edge. Gaudi2 is worse: Intel Gaudi 2 HL-225 is OAM-only with no PCIe SKU - needs a UBB chassis + 24x 100GbE fabric to light up records that HL-225 has no PCIe path at all and needs a UBB chassis + fabric. Add that cost to the $/tok-s before comparing it to a PCIe row.
• Pascal/Maxwell has no FP16 tensor path. NVIDIA Tesla M40 24GB (#5) and NVIDIA Tesla P40 24GB (#8) look like steals at $189/$290, but Tesla P40/M40 (Pascal/Maxwell) have no FP16 tensor cores - GGUF INT8 DP4A is the only usable LLM path is explicit: the M40's low sticker is a trap (no DP4A, slower than P40 on the only LLM path that matters), both are passive cards needing a shroud+blower, and their EPS-12V power connector is not a PCIe 8-pin. The P40's DP4A INT8 path is the floor buy; the M40 is not a cheaper P40.
• MI100's $720 needs a ROCm tax. AMD MI100 multi-GPU segfaults on bare-metal ROCm - needs PCIe-passthrough VM + HSA_AMD_SVM records that MI100 is the best $/(GB·TB/s) in the used slice but multi-GPU segfaults on bare-metal ROCm and wants a PCIe-passthrough VM — and gfx908 is maintenance-tier on the ROCm matrix.
• A "fits" flag is not a speed promise — and the new edge-NPU rows prove it. The M40 fits 35B-A3B (24 GB) but its 288 GB/s ceiling is 14.4 tok/s (derived) — barely interactive. The cheap edge-NPU fits are slower still: Huawei Ascend 310P (Da Vinci edge inference NPU) and Huawei Atlas 300I Pro inference card (24 GB, 310-series) (both 204.8 GB/s → 10.2 tok/s ceiling (derived)), Qualcomm Cloud AI 100 (family: HHHL-Pro / HHHL-Std / HHHL-Lite / Dual M.2 / Dual M.2e) (137 GB/s → 6.8 tok/s) and Cambricon MLU370 (Siyuan 370) AIDC accelerators (307.2 GB/s → 15.4 tok/s) are NPUs/AIDC parts whose low sticker reflects an immature or edge-targeted software stack, not a serving answer — a Y on 70B (MLU370) means "the bytes fit," not "it serves." Conversely the unified-memory parts (Apple, Strix Halo) fit large models but are bandwidth-capped; AMD Strix Halo is an MoE part, not a dense-70B part - the ~256 GB/s wall caps dense decode at ~5 tok/s measures a dense-70B at ~5 tok/s on the ~256 GB/s Strix Halo wall, while the same record class measures the 35B-A3B MoE target at 42 tok/s (AMD Ryzen AI Max+ Radeon 8060S iGPU (RDNA 3.5) under Ollama: Qwen3.5 35B-A3B MoE generation throughput, 100% GPU offload). The MoE target is exactly the workload those parts are good for.

6. Assumptions and what this table deliberately ignores

• Price class: every purchase price is an asking price (used-market, reseller, or MSRP), not a realised transaction price. The repo stamps each with a PRICE CLASS and a RE-VERIFY BY date; the Oct-2026 used records are freshest. $/tok-s inherits that uncertainty.
• Unit price ≠ system price. Prices are per bare card/module where the record gives one. Several rows price a whole system instead — the part is not sold bare: Used Apple Mac Studio M1 Ultra 128GB ($2,500), Used Apple Mac Studio M2 Ultra 192GB ($3,700), eBay used Mac Studio M3 Ultra 512GB (Oct 2026) ($9,000 — the 512 GB config the accelerator record's vram_gb describes; the 96 GB Studio at $3,700 is a different machine and is not joined) are whole Mac Studios, and all six Strix Halo rows (New Minisforum MS-S1 MAX 128GB (Strix Halo) $1,900, New Framework Desktop 128GB (Ryzen AI Max+ 395), New Beelink GTR9 Pro 128GB (Strix Halo), New GMKtec EVO-X2 128GB (Ryzen AI Max+ 395 Strix Halo), New Beelink GTR9 Pro 128GB, New GMKtec EVO-X3 128GB) are whole mini-PCs/boards, plus the DGX-1 (below), two whole MacBook Pro laptops (eBay used MacBook Pro M3 Max 16in 128GB (Oct 2026) $3,400 and eBay used MacBook Pro M4 Max 16in 128GB (Oct 2026) $4,700), and — new this pass — a whole EPYC self-build, Used single-socket EPYC 9004 'Genoa' build (12ch DDR5, ~384GB) ($7,500: used CPU + SP5 board + 384 GB DDR5 + chassis, joined to AMD EPYC 9554 (Genoa, 64-core, 12-channel DDR5 host CPU)). Dividing a whole-system price by a per-device ceiling is an apples/orange flagged here, not hidden — the laptop rows price the screen, battery and logic board as well as the unified pool, and the Genoa row prices the DRAM and the motherboard, which is why both sit at the bottom of the ranking despite a Y/Y/Y fit. These parts are listed because no bare-silicon price exists, and for the unified-memory rows the system is the product. Capacity-mismatched Apple SKUs stay unjoined. eBay used Mac Studio M1 Ultra 64GB (Oct 2026) ($1,850) is a 64 GB chassis against a record whose vram_gb is 128; eBay used Mac Studio M3 Ultra 256GB (Oct 2026) ($6,250) and Used Apple Mac Studio M3 Ultra 96GB ($3,700) price less pool than the M3 Ultra record's 512 GB. Joining either would rank a cheaper machine against a fit flag it does not have.
• Derived, not measured. docs/08-cost-per-token.md joins measured benchmarks to rental prices; this doc joins roofline ceilings to purchase prices. They answer different questions and neither substitutes for the other. A Y here and a cheap $/tok-s do not mean the part serves the model well at concurrency.
• Deliberately dropped joins (the raw accelerator_ids link would over-attribute, per the same trap tools/cost_per_token.py documents):
  • ebay-used-accelerator-market ($10,449.95) is one A100 80GB SXM4 listing; its H100/H200 ids are "obtainable," not "priced at $10,450." Not joined to H100/H200.
  • ebay-used-a100-40gb-sxm4 ($2,499) prices a 40GB module but points at the 80GB id (no 40GB record when written). Mapped to NVIDIA A100 40GB SXM4, not the 80GB part.
  • microcenter-refurb-rtx-3090 is two tiers: 3090 $1,099.99 / 3090 Ti $1,239.99 — each joined at its own price.
  • new-retail-rtx-5090-amazon headline is the 5090 band; the 5080 sibling band lives in notes and is not the headline price. 5080 not joined.
  • new-retail-rtx-4090-3090-consumer headline $3,299 is the 4090 median; the 3090 median is $1,514 in notes. 3090 not joined at $3,299.
• Whole-system rows kept for context but not mis-joined: used-dgx1-8x-v100 and used-dgx1-8xv100-32gb-chassis ($4,000) are 8-GPU servers → ~$500/GPU-equivalent (derived), consistent with the $500 bare-SXM2 row but system prices. gaudi2-8x-oam-tray-supermicro-820gh ($11,995) is an 8-module OAM tray → ~$1,500/module-equivalent (derived), a system price not a module price. ebay-used-a100-40gb-rack-liquidator ($249,000) is a 32-GPU rack; used-8x-h100-sxm-node-system ($180,000) is an 8-GPU server. None are bare-card prices.
• Edge/SBC whole-board prices are not accelerator prices. new-raspberry-pi-ai-hat-2 ($200, Hailo-10H + a Pi 5), new-rockchip-rk3588-sbc ($140, RK3588 board) and used-google-coral-edge-tpu ($90) price a whole single-board computer, not the NPU alone — and their accelerator records carry no vram_gb >= 8 anyway (Hailo-10H has 8 GB but no memory_bandwidth_gbps; the other two are vram_gb-null unified-memory SoCs), so they would not qualify even if joined. Left unjoined so the price column stays a GPU/part price.

7. Reproduce

python tools/fit_matrix_35b.py            # the full 159-row matrix, all three sections, re-run on 81b8382 (GB10, Jetson AGX Thor, TITAN V, RTX PRO 2000 Blackwell, Arc Pro B50, the base M4, the GTX 1070 Ti, Jetson AGX Orin 64GB, Tesla P4 and Radeon 880M are outside the gate; see the Oct-16 p19 note, the Oct-17 #183 note, the Oct-17 #197 note, the Oct-17 #199 note, the Oct-17 #203 note, the Oct-17 #209 note and the Oct-17 #214 note)
python tools/validate.py                  # record integrity (no new records added here)

Every figure above is generated by that tool against the committed data/ records; where the tool's arithmetic disagrees with a table cell, the tool is right and the cell is stale. The tool also carries the two judgement calls the table depends on, documented inline: the curated PRICE_JOIN (which corrects the raw accelerator_ids over-attribution the same way tools/cost_per_token.py does) and UNIFIED_MEM_GB (the configured-memory override that keeps vram_gb-null unified-memory parts like Strix Halo from being silently filtered out by the >= 8 GB gate).