id
gotchas/w7z-onchip-sram-filed-as-vram-and-bandwidth
Concepts
rooflinearithmetic-intensitywafer-scalesystolic-arraydataflowin-memory-computememory-hierarchyfield-semanticsprose-versus-fields
Symptom
A query, join or sort over the accelerator corpus returns a number for a non-GPU part that is arithmetically right and semantically wrong. Concretely, three records in this repo held on-chip SRAM figures in the two fields that every GPU record uses to mean off-package memory capacity and off-package memory bandwidth: cerebras-wse-3 at vram_gb 44 and memory_bandwidth_gbps 21000 (44 GB of on-wafer SRAM, 21 PB/s of on-wafer SRAM bandwidth), groq-lpu at 0.23 and 80000 (230 MB SRAM, 80 TB/s SRAM bandwidth), and d-matrix-corsair at 2 and 150000 (2 GB on-chip Performance Memory at 150 TB/s, on a card whose actual off-chip LPDDR5 Capacity Memory tier runs at about 400 GB/s per card). The reader computes a roofline. For the WSE the roofline says memory bandwidth is 21000 GB/s, which would make weight streaming free; for d-Matrix it says 150000 GB/s, which would make the card 25x an H100 on bandwidth and hide a real 400 GB/s wall. Both answers are meaningless, and both are reached by following the repo's own field names.
Root cause
Two independent causes, and the second is the one that let the defect survive three passes.
(1) A FIELD-NAME COLLISION. vram_gb is defined in the schema as 'HBM or VRAM per device/package' and memory_bandwidth_gbps has no definition at all. On a GPU, and on any part with an off-package DRAM tier, those names and those quantities coincide, so nothing distinguishes them. On a part whose only memory is on-chip SRAM, the names still look right and the quantities are different things entirely. There is no value of vram_gb that is correct for a wafer-scale part, and no value of memory_bandwidth_gbps - not null, not zero, not the SRAM figure. A field whose meaning is defined by a convention rather than by a definition cannot be made honest by populating it.
(2) PROSE DOES NOT CONSTRAIN A FIELD. Every one of these three records ALREADY carried correct, careful prose in notes. cerebras-wse-3 said the two figures 'are false friends against every GPU field of the same name and must not be read as VRAM or as a memory-bandwidth roofline input'. groq-lpu said its 80 TB/s 'is SRAM bandwidth, not memory capacity bandwidth'. d-matrix-corsair said its two-tier system was 'the key innovation'. All three warnings were correct, all three were added by earlier passes, and all three fields were left populated anyway - because notes are invisible to every numeric query, join, sort and index in the repo, while the fields are the only thing any downstream consumer reads. A reader who greps notes is protected. A reader who filters vram_gb is not. Correcting the prose again would have been the third no-op; only emptying the field fixed anything.
A THIRD, RELATED CAUSE THAT PRODUCES NO WRONG NUMBER AT ALL, so it is easy to miss: the same GPU reasoning applied to a part whose FIELDS ARE ENTIRELY CORRECT. tenstorrent-wormhole-n150 and tenstorrent-blackhole-p150 both hold true GDDR6 capacity, bus width and bandwidth, and both then asserted a batch-1 roofline position - 'decisively bandwidth-bound at batch 1 (257 FLOP/byte FP16 does not save it)' and 'at 512 GB/s it is still memory-bound for batch-1 decode' - derived from a compute-to-byte balance point. On a Tensix part the tapeout is a compiled schedule over a RISC-V NoC, so there is no batch size at which a balance point predicts behaviour, and calling GDDR6 'the wall' inverts a deliberate cost decision into a deficiency. Finding this class produces pressure to change a number, and changing one would have introduced the error rather than removed it. The correct response to an interpretation-only finding is to correct the prose and leave every figure alone.
Workaround
For an SRAM-only or in-memory part: set vram_gb and memory_bandwidth_gbps to null and record WHY in the record's notes, naming the document that was read and the figure that was found not to be this quantity. Put a first-party on-chip capacity in accelerator.onchip_cache with kind 'sram-feed-tier' and sram_mb, which is what that field exists for - but ONLY if a vendor published it. Do not put an on-chip tier into a GPU-named field, do not convert it, and do not average two tiers into one number.
For a part with BOTH an on-chip tier and an off-package tier (d-Matrix is the case in this corpus): memory_bandwidth_gbps must hold the OFF-PACKAGE tier's bandwidth, because that is the only tier that bounds weight streaming and the only one any GPU record of that name means. The on-chip tier goes in onchip_cache. The off-package capacity, not the on-chip capacity, is what a reader asking 'how much model fits' wants.
For a part with a real off-package tier but non-GPU data movement (the Tenstorrent cards): change no field. Remove the roofline vocabulary - 'compute-to-byte', 'FLOP/byte', 'memory-bound at batch 1', 'balance point' - and replace it with a statement about weight-streaming traffic per token and about the serving regime the architecture targets. Preserve the useful facts underneath (the 108 MB and 180 MB SRAM feed tiers, the price band, the TDP), because those are what a reader actually needs.
General rule for anyone extending this corpus: on ANY part whose data placement is fixed at compile time - systolic, dataflow, wafer-scale, in-memory-compute - the roofline question does not have a well-formed answer, and the honest record says so rather than answering it with a ratio.
notes
WHY THIS IS A GOTCHA AND NOT JUST A FIX: all three wrong fields were populated by a pass that had ALREADY correctly diagnosed the problem in prose, and a fourth record pair had the same error in interpretation with every field correct. The corpus had four independent instances of one category error, and the first three were invisible to the only check that exists because the check was textual while the defect was structural. The durable lesson is that on a non-GPU record the question is not 'is this number right' but 'is this number about this part's memory system', and that adding a correct warning to notes is not a fix.
THE FIX, RECORDED SO A LATER PASS DOES NOT REVERT IT AS A REGRESSION: cerebras-wse-3 vram_gb 44 -> null, memory_bandwidth_gbps 21000 -> null, with 44 GB moved to onchip_cache.sram_mb (kind sram-feed-tier, per_scope per-package, basis vendor-spec-sheet, because the CS-3 system page and the CS-3 datasheet both publish it). groq-lpu vram_gb 0.23 -> null, memory_bandwidth_gbps 80000 -> null, and onchip_cache left NULL because Groq publishes no SRAM capacity figure at all and the schema forbids filling that field without a first-party source. d-matrix-corsair vram_gb 2 -> null, memory_bandwidth_gbps 150000 -> null, with 2 GB moved to onchip_cache.sram_mb (basis vendor-whitepaper) and an int16 600 TFLOPS row added from the white paper's numerics table.
WHAT WAS DELIBERATELY NOT CHANGED, and it belongs in this record because the temptation runs the other way on a 'found an error' pass: no number on either Tenstorrent record was altered. No number on tenstorrent-wormhole-n150's GDDR6 tier is wrong. Equally, the 230 MB Groq figure and the unverified 188 TFLOPS / 750 TOPS Groq compute rows were NOT deleted - they were already correctly marked basis 'unverified', vendor_claim false and source_id null, and deleting them would have destroyed the record of what was looked for.
THE DEEPER FORM OF THE SAME POINT, which no field can express and which every reader of a non-GPU record needs: on a GPU, memory bandwidth is the wall because there IS an off-package tier that weights must cross on every token. On a wafer-scale or SRAM-only part the weights are already in on-chip SRAM, so there is no weight-streaming cost for a model that fits, and the binding constraint is CAPACITY plus static compile-time scheduling. The same number - bandwidth - means the opposite thing on the two architectures, which is why a field named after the GPU sense of it cannot be filled honestly on the other kind of part no matter what value you choose.
A SIBLING DEFECT FOUND IN THE SAME PASS AND NOT COVERED HERE, because it is a units-and-provenance problem rather than an architecture problem: aws-inferentia2 filed 380 as FP8 TFLOPS on an admitted 2x inference from its 190 FP16 figure, when the vendor publishes that figure as 380 INT8 TOPS - so the number was right and the precision was invented. It was fixed in place (now precision int8, basis vendor-spec-sheet, first-party source) and the record also carries an unconverted GiB/s-vs-GB/s discrepancy. See the record's own notes.