id
gotchas/value-joules-per-token-and-why-the-cheapest-energy-is-not-the-cheapest-hardware
Symptom
The corpus has both TDP for 116 accelerators and decode_tok_s for 32 benchmark records, so joules per token looks like a straightforward join. The naive join produces absurd figures: the GB200 NVL72 row at 0.16 J/tok, which would make a 27kW rack-scale system the most energy-efficient device in the database by a factor of 25, and the Cerebras WSE-3 at 36.00 J/tok against its own 27,000W TDP on a 750 tok/s vendor peak figure.
Root cause
Three separate confounds, each of which invalidates the join for specific rows.
UNITS OF DEVICE. sglang-deepseek-v3-gb200-nvl72-decode-tps-per-node aggregates an entire NVL72 rack, but the TDP joined to it is a single 1200W accelerator figure. Dividing rack throughput by one card's power understates the true consumption by two orders of magnitude. The same defect hits comm-mi50-50x rows, which scale via HIP_VISIBLE_DEVICES across 2, 4, 8 and 16 GPUs while the accelerator record carries one MI50X's 300W.
MEASUREMENT BOUNDARY. The Cerebras figure is a vendor peak number on a 750 tok/s claim with no stated power at the measured operating point; 27,000W is a board TDP, not the draw during that benchmark. The apple m4-pro joules records are system- and package-level figures that must never be mixed with a GPU TDP - a different boundary entirely.
BATCH SIZE. comm-mi50-8x and comm-mi50-16x run at -b 2048 -ub 1024, i.e. micro-batch 2048. A batch-2048 throughput divided by a card's TDP is not comparable to a batch-1 row, because per-token energy falls as batch rises: the batch is doing the work of sharing fixed cost, and it also means the figure is not what a single user experiences.
UNSOURCED VALUE. local-rtx-5070-13b-q4-decode is marked UNSOURCED, DO NOT CITE and was decommissioned on 2026-10-04, retained only for history because the schema forbids nulling value. It produces an attractive 2.08 J/tok that is the second-best in the naive table and must not be quoted.
Workaround
Restrict to rows that are single-device, batch 1, and carrying a sourced value. Nine rows qualify:
J/tok $/Mtok accelerator W tok/s batch model
4.11 0.1017 nvidia-rtx-5090 575 140.00 b1 Qwen3-14B-AWQ
4.87 0.1204 amd-radeon-ai-pro-r9700 300 61.66 b1 Bonsai-27B Q1_0
5.54 0.1371 nvidia-rtx-5080 360 65.00 b1 Qwen3 32B
6.76 0.1674 nvidia-rtx-5090 575 85.00 b1 Qwen3 32B
7.54 0.1867 amd-radeon-rx-7900-xtx 355 47.06 b1 Qwen3.8-27B Q4_K_XL
10.91 0.2700 nvidia-rtx-pro-6000-blackwell 600 55.00 b1 Llama 3.3 70B
12.78 0.3163 nvidia-rtx-5090 575 45.00 b1 Llama 3.3 70B
13.71 0.3394 local-apple-m5-ultra 480 35.00 b1 Llama 3.3 70B
20.53 0.5081 amd-instinct-mi300x 750 36.53 b1 DeepSeek-V3 671B Q4_K_M
Electricity cost at the corpus's own $0.0891/kWh rate from supply/us-electricity-industrial-tariff-2024, using 1 kWh = 3.6 MJ.
WHAT IT MEANS: energy efficiency and cost of ownership disagree, and the corpus proves it. The RTX PRO 6000 Blackwell at 10.91 J/tok is 2.65x less energy-efficient than the RTX 5080 at 5.54, yet the 5080 is $6,550 against the PRO 6000's $16,624 and cannot fit a 70B model at all. The 5090's own rows span 4.11 to 12.78 J/tok depending purely on which model runs - a 3.1x range on identical hardware, which is larger than the spread between any two different cards here.
Energy is therefore a poor proxy for value on this corpus, and the joules-per-token framing should be read as a statement about heat and power draw, not about what to buy. value-tco-and-why-rent-vs-own-cannot-be-computed-from-this-corpus reaches the same conclusion from the cost side: power is 15-20% of three-year cost for $2,000 cards and under 5% above $12,000.
notes
FILED 2026-10-11 (parent, in place of a subagent that died on HTTP 429 after 6 API calls, before writing anything). Computed directly from the accelerator TDP fields and decode_tok_s benchmark records.
Rows excluded and why: local-rtx-5070-13b-q4-decode (UNSOURCED, decommissioned); comm-mi50-8x-gpt-oss-120b-mxfp4-decode-collapse and comm-mi50-16x-deepseek-v32-awq-reported (multi-GPU against one card's TDP, and batch 2048); the GB200 NVL72 decode record, benchmarks/sglang-deepseek-v3-gb200-nvl72-decode-tps-per-node (rack-scale against one card's TDP; omitted from affects because it has no source record, which is itself a gap worth noting); alt-cerebras-gpt56-ultrafast-tps (vendor peak with no measured power at that operating point).
The GB200 exclusion deserves a note: a per-node figure divided by a single accelerator's TDP is not merely imprecise, it is wrong by the number of cards in the node. Any future joules-per-token row in this corpus must state the device count the power figure covers, and the batch size, alongside the J/tok.