inference/research
contestedlow · 0.55updated 2026-10-08

Cerebras WSE-3 (CS-3 system)

cerebras wafer-scale engine (WSE-3) 2024

Fields

id
accelerators/cerebras-wse-3
Vendor
cerebras
Architecture
wafer-scale engine (WSE-3)
Release year
2024
Process (nm)
5
Form factors
mdu
Memory type
sram (on-wafer, no DRAM tier; capacity recorded in onchip_cache, not vram_gb)
Bandwidth basis
NULL, AND HERE IS WHAT WAS LOOKED FOR (2026-10-05, non-GPU classification audit). The CS-3 system page and the CS-3 datasheet BOTH publish a '21 PB/s memory bandwidth' figure and this field used to hold it. It is NULL because 21 PB/s is ON-WAFER SRAM bandwidth on a part that has no DRAM tier and no memory bus: there is no memory-tier bandwidth for this field to hold. The figure is not lost - it is preserved verbatim in sources asic-cerebras-cs3 and vb-cerebras-cs3-datasheet-memory-power, and the on-wafer capacity is now in this record's onchip_cache.sram_mb. NO FIGURE WAS BACK-COMPUTED from any other quantity. See notes, section 'FIELDS CORRECTED 2026-10-05'.
On-chip cache
[object Object]
Peak FLOPS
precisionfp16tflops125,000densenovendor_claimyessource_idvb-cerebras-cs3-datasheet-125pf-sparsenotesWSE-3 wafer in the CS-3 system. CORRECTED 2026-10-04, twice, in the same field. DENSITY: the CS-3 datasheet prints 'FP16 125 petaFLOPS*' with the footnote '* FLOPS shown as sparse'. An earlier version of this record asserted 'Cerebras publishes no precision label and no sparsity statement for this figure' and set dense:true. That assertion is false and the flag encoded the OPPOSITE of the vendor's own label. Now dense:false, so any reader applying the repo's sparse=2x-dense convention halves it to 62.5 PFLOPS dense, which is the correct reading of a single sparse figure. The 'fp16' precision label is a schema-enum placeholder, NOT a vendor claim about FP16 arithmetic. This record previously contained a FABRICATED dense/sparse PAIR: 125 PFLOPS was recorded as the dense half of a pair whose sparse half was 250, when 125 is the CS-3 wafer and 250 is the WSE-3T in CS-4 - two different products. The narrative correction for that was already right; the flag was what still encoded the error.basisvendor-spec-sheetbasis_detailTranscribed from the vendor's own fp16 for Cerebras WSE-3 (CS-3 system) (source of record: 'Cerebras CS-3 / WSE-3 Datasheet - 125 petaFLOPS, FP16, shown as SPARSE', kind: spec-sheet). Recorded dense:false - the vendor prints this as a structured-sparsity figure; the dense companion row on this record is the halved value. See this record's notes for the sparsity-footnote rule. Full reasoning for this row retained verbatim in notes.
TDP (W)
27,000
Unified memory
no
Consumer part
no
notes
CORRECTED 2026-10-04: The previous version of this record contained a FABRICATED sparse/dense pair (125 PFLOPS dense + 250 PFLOPS sparse) that did not reflect any Cerebras publication. The 125 PFLOPS figure is the CS-3 wafer; the 250 PFLOPS figure belongs to the WSE-3 Turbo in CS-4, a DIFFERENT and faster wafer. They are NOT a sparse/dense pair of the same part. This record now covers only the CS-3 system. A separate record for CS-4 / WSE-3 Turbo should be created if needed. WHAT CEREBRAS DOES NOT DO WELL: (1) 44 GB SRAM capacity is tiny for modern LLMs — a 70B model in BF16 needs ~140 GB, so even the CS-3 cannot hold a single 70B model on-chip. (2) No dynamic batching — the wafer-scale architecture requires static compilation and scheduling. (3) Limited ecosystem — Cerebras SDK only, no PyTorch/JAX native support, no vLLM. (4) No MLPerf submission — Cerebras has never submitted to MLPerf, so there is no independent benchmark. (5) The 125 PFLOPS figure has no published precision label or sparsity statement, making it impossible to compare against GPU TFLOPS numbers. (6) TDP is 25 kW for the CS-3 system, which is enormous. (7) The wafer-scale approach means the entire wafer is a single device — if any part fails, the whole wafer is affected. WHAT CEREBRAS IS GOOD AT: (1) On-wafer SRAM bandwidth of 21 PB/s is orders of magnitude above HBM, eliminating the memory wall for models that fit in 44 GB. (2) Deterministic execution with no cache misses or variance. (3) The CS-4 system (3 wafers, 250 PFLOPS per wafer, 43.2 PB/s) is the fastest LLM inference system available for models that fit on-chip, with published results like 750 tok/s on GPT-5.6 Sol Ultrafast. (4) The wafer-scale approach eliminates inter-chip communication bottlenecks for models that fit on a single wafer. POWER: tdp_w corrected 25000 -> 27000. The CS-3 datasheet states 27 kW max system power for the 16-OU system. Also note vram_gb (44 GB) and memory_bandwidth_gbps (21000) hold ON-WAFER SRAM figures, not a DRAM tier: the WSE has no memory bus and no off-package memory. Those are false friends against every GPU field of the same name and must not be read as VRAM or as a memory-bandwidth roofline input. MEMORY_BUS_WIDTH: STILL NULL, AND HERE IS PRECISELY WHAT WAS LOOKED FOR (2026-10-04). The vendor publishes NO memory-bus width for this part, and none was back-computed from the published bandwidth. NO FIGURE WAS BACK-COMPUTED FROM BANDWIDTH.[REPAIRED 2026-10-08 by w8-reverify. This sentence previously read '(2026-10-04). %s NO FIGURE WAS BACK-COMPUTED FROM BANDWIDTH' — a literal printf placeholder left in the JSON by the wave-3 pass (commit 9968769) across 34 accelerator records, and invisible to every check in this repo because validate.py never reads notes. The substitution text is the same boilerplate the 7 uncorrupted records in this family already carry, and it is only valid here because memory_bus_bit is null on every one of the 34, so "the vendor does not publish it" is the true state. A record with a non-null memory_bus_bit must NOT receive this filler.] That is the standing rule for this field in this repo and it was applied to every null here rather than to a subset: a width derived by dividing a published aggregate bandwidth by an assumed or half-assumed per-pin rate is two inferences stacked on one number, and if either inference is wrong the record looks populated while being wrong. The correct state of this field is 'the vendor does not publish it', and the note above names the specific document that was read and found not to carry it, so a future pass does not have to re-establish that from scratch. MEMORY_BUS_WIDTH: STILL NULL, AND HERE IS PRECISELY WHAT WAS LOOKED FOR (2026-10-04). STRUCTURALLY NOT APPLICABLE, which is different from 'not found'. The WSE-3 has no DRAM tier at all - memory is the on-wafer SRAM, whose 'width' is a wafer-scale fabric rather than a bus. There is no memory bus to report a width for. Left null deliberately. NO FIGURE WAS BACK-COMPUTED FROM BANDWIDTH. That is the standing rule for this field in this repo and it was applied to every null here rather than to a subset: a width derived by dividing a published aggregate bandwidth by an assumed or half-assumed per-pin rate is two inferences stacked on one number, and if either is wrong the record looks populated while being wrong. The correct state of this field is 'the vendor does not publish it', and the text above names the specific document read and found not to carry it, so a future pass does not have to re-establish that from scratch. === FIELDS CORRECTED 2026-10-05 (non-GPU classification audit, w7-nonxpu-computers). THE PROSE ABOVE WARNED ABOUT THESE TWO FIELDS AND THE FIELDS STILL HELD THE FALSE-FRIEND VALUES. === A previous pass appended three paragraphs of prose to notes saying these two fields are on-wafer SRAM figures and not VRAM or a bandwidth roofline input, and then LEFT BOTH FIELDS POPULATED with them. Prose does not constrain a numeric field: any query, join or sort over vram_gb still returned 44 for this part, and any roofline computed from memory_bandwidth_gbps still used 21000. The notes were load-bearing for a human reader and invisible to every other reader in the repo, which is the worse of the two failures. vram_gb: 44 -> null. Cerebras's 44 GB is ON-WAFER SRAM, and the WSE-3 has no DRAM tier. There is no honest value for a field defined as 'HBM or VRAM per device/package'. The 44 GB is not lost to the corpus: schemas/accelerator.schema.json gained an onchip_cache object in 2026-10-05 whose sram_mb sub-field exists for exactly this, and it is now populated below with kind 'sram-feed-tier'. A reader who wants the capacity gets it from a field whose name says what it is; a reader who wants vram_gb gets null, which is the truth. memory_bandwidth_gbps: 21000 -> null. 21 PB/s is on-wafer SRAM bandwidth. The WSE-3 has no memory bus and no off-package memory, so there is no DRAM-tier bandwidth to report. This is not 'the vendor did not publish it' - the vendor published a large number, just not one of this quantity. Keeping it here would let a roofline be computed that is meaningless for a dataflow part. memory_bus_bit: already null and stays null, for the structural reason the notes above state rather than the 'not published' reason. tdp_w 27000: unchanged, and still the CS-3 SYSTEM's 27 kW max from the datasheet, not a wafer figure. It is a system figure in a per-part field, which is a milder version of the same problem and is left in place because the datasheet figure is the only published power number and the notes name it as a system figure. WHY null AND NOT A NEW FIELD: the schema has no way to say 'the number exists and it is not this quantity', so the honest states are null-with-reason or a wrong number. This pass chose null-with-reason. A reader who wants the SRAM bandwidth must go to onchip_cache and to the CS-3 datasheet source, and the source records still carry the 21 PB/s figure verbatim so nothing is lost from the corpus. WHAT A ROOFLINE READER SHOULD DO INSTEAD, because 'this part has no roofline' is a finding and not a dead end: the binding constraint on a WSE is that the model must fit in on-wafer SRAM, plus static compile-time scheduling and the absence of dynamic batching. Bandwidth is not the limit for a model that fits, because there is no off-package memory to stream weights from. That is the opposite of the GPU situation, where bandwidth is the limit precisely because there IS an off-package tier. data/flops/alt-cerebras-limitations and data/flops/alt-cerebras-wafer-sram already carry this reasoning, and this record's new nulls are what make a numeric reader land on the same conclusion instead of on 21000 GB/s. === SOURCE DECLARED 2026-10-08 (w8-reverify). === flops[].source_id cited 'vb-cerebras-cs3-datasheet-125pf-sparse' on 1 row(s), and that source file exists and was read — but it was never added to this record's sources[], so the figure carried per-row provenance that the record-level source list did not back. AGENTS.md rule 2 requires every cited source to be resolvable, and it was, but the reverse held too: a source a record actually cites is a source the record must declare. Added now. No number changed. validate.py has never caught this class of defect — it checks that sources[] entries resolve, never that a cited source_id appears in sources[] — which is why 121/121 tests passed over it. === vram_gb NULL RE-VERIFIED 2026-10-08 (w8-reverify) — THE 2026-10-05 CORRECTION HOLDS ON THIS RECORD. === The re-verification brief named this the highest-risk change in wave 7, and the test it names is the right one: if the vendor publishes external memory for the part, null was wrong. It was checked against every Cerebras first-party page this repo can reach, and the conclusion is that the null stands. THE CS-3 DATASHEET, read live 2026-10-08, has one memory row and it is on-chip: 'Processor Memory | 44 GB total, 21 PB/s SRAM bandwidth', and the processor row repeats '21 PB/s on-chip SRAM bandwidth (44 GB on-chip SRAM)'. There is no DRAM row, no HBM row and no external-memory row. The CS-3 system page likewise lists '44GB On-Chip SRAM' and '21PB/s Memory Bandwidth' and nothing external. WHAT CEREBRAS DOES PUBLISH THAT LOOKS LIKE EXTERNAL MEMORY, AND WHY IT IS NOT ONE: the CS-3 announcement blog publishes MemoryX attachment capacities - 24 TB, 36 TB, 120 TB and 1,200 TB - and says 'The CS-3 can be configured with up to 1,200 terabytes of external memory'. Those are separate MemoryX UNITS for CLUSTER configurations, with no per-part bandwidth published anywhere, and the smallest of them is 545x this wafer's own 44 GB. Nothing in them licenses a per-device vram_gb value. Both the absence and the near-miss are recorded in source w8r-cerebras-cs3-blog-memoryx-external-memory, added to this record, so a future pass does not have to re-establish either. SO THE RECORD'S OWN REASONING STANDS, WITH ONE WORD SHARPENED: the 2026-10-05 audit wrote that this is a part with 'no DRAM tier', which is true of the WSE-3 as a part but sits next to a vendor that sells external memory for the systems it is installed in. The load-bearing distinction is between 'the vendor publishes no per-part DRAM figure' (true) and 'the vendor publishes no external-memory figures' (false). The null was reached for the right reason; the prose next to it should not invite the wider reading. The 44 GB is and stays in onchip_cache.sram_mb with kind 'sram-feed-tier', and the 21 PB/s stays preserved verbatim in the two source records. NO FIELD CHANGED VALUE in this re-verification - only the sources list and this note.

Notes

CORRECTED 2026-10-04: The previous version of this record contained a FABRICATED sparse/dense pair (125 PFLOPS dense + 250 PFLOPS sparse) that did not reflect any Cerebras publication. The 125 PFLOPS figure is the CS-3 wafer; the 250 PFLOPS figure belongs to the WSE-3 Turbo in CS-4, a DIFFERENT and faster wafer. They are NOT a sparse/dense pair of the same part. This record now covers only the CS-3 system. A separate record for CS-4 / WSE-3 Turbo should be created if needed. WHAT CEREBRAS DOES NOT DO WELL: (1) 44 GB SRAM capacity is tiny for modern LLMs — a 70B model in BF16 needs ~140 GB, so even the CS-3 cannot hold a single 70B model on-chip. (2) No dynamic batching — the wafer-scale architecture requires static compilation and scheduling. (3) Limited ecosystem — Cerebras SDK only, no PyTorch/JAX native support, no vLLM. (4) No MLPerf submission — Cerebras has never submitted to MLPerf, so there is no independent benchmark. (5) The 125 PFLOPS figure has no published precision label or sparsity statement, making it impossible to compare against GPU TFLOPS numbers. (6) TDP is 25 kW for the CS-3 system, which is enormous. (7) The wafer-scale approach means the entire wafer is a single device — if any part fails, the whole wafer is affected. WHAT CEREBRAS IS GOOD AT: (1) On-wafer SRAM bandwidth of 21 PB/s is orders of magnitude above HBM, eliminating the memory wall for models that fit in 44 GB. (2) Deterministic execution with no cache misses or variance. (3) The CS-4 system (3 wafers, 250 PFLOPS per wafer, 43.2 PB/s) is the fastest LLM inference system available for models that fit on-chip, with published results like 750 tok/s on GPT-5.6 Sol Ultrafast. (4) The wafer-scale approach eliminates inter-chip communication bottlenecks for models that fit on a single wafer.

POWER: tdp_w corrected 25000 -> 27000. The CS-3 datasheet states 27 kW max system power for the 16-OU system. Also note vram_gb (44 GB) and memory_bandwidth_gbps (21000) hold ON-WAFER SRAM figures, not a DRAM tier: the WSE has no memory bus and no off-package memory. Those are false friends against every GPU field of the same name and must not be read as VRAM or as a memory-bandwidth roofline input. MEMORY_BUS_WIDTH: STILL NULL, AND HERE IS PRECISELY WHAT WAS LOOKED FOR (2026-10-04). The vendor publishes NO memory-bus width for this part, and none was back-computed from the published bandwidth. NO FIGURE WAS BACK-COMPUTED FROM BANDWIDTH.[REPAIRED 2026-10-08 by w8-reverify. This sentence previously read '(2026-10-04). %s NO FIGURE WAS BACK-COMPUTED FROM BANDWIDTH' — a literal printf placeholder left in the JSON by the wave-3 pass (commit 9968769) across 34 accelerator records, and invisible to every check in this repo because validate.py never reads notes. The substitution text is the same boilerplate the 7 uncorrupted records in this family already carry, and it is only valid here because memory_bus_bit is null on every one of the 34, so "the vendor does not publish it" is the true state. A record with a non-null memory_bus_bit must NOT receive this filler.] That is the standing rule for this field in this repo and it was applied to every null here rather than to a subset: a width derived by dividing a published aggregate bandwidth by an assumed or half-assumed per-pin rate is two inferences stacked on one number, and if either inference is wrong the record looks populated while being wrong. The correct state of this field is 'the vendor does not publish it', and the note above names the specific document that was read and found not to carry it, so a future pass does not have to re-establish that from scratch.

MEMORY_BUS_WIDTH: STILL NULL, AND HERE IS PRECISELY WHAT WAS LOOKED FOR (2026-10-04). STRUCTURALLY NOT APPLICABLE, which is different from 'not found'. The WSE-3 has no DRAM tier at all - memory is the on-wafer SRAM, whose 'width' is a wafer-scale fabric rather than a bus. There is no memory bus to report a width for. Left null deliberately. NO FIGURE WAS BACK-COMPUTED FROM BANDWIDTH. That is the standing rule for this field in this repo and it was applied to every null here rather than to a subset: a width derived by dividing a published aggregate bandwidth by an assumed or half-assumed per-pin rate is two inferences stacked on one number, and if either is wrong the record looks populated while being wrong. The correct state of this field is 'the vendor does not publish it', and the text above names the specific document read and found not to carry it, so a future pass does not have to re-establish that from scratch.

=== FIELDS CORRECTED 2026-10-05 (non-GPU classification audit, w7-nonxpu-computers). THE PROSE ABOVE WARNED ABOUT THESE TWO FIELDS AND THE FIELDS STILL HELD THE FALSE-FRIEND VALUES. === A previous pass appended three paragraphs of prose to notes saying these two fields are on-wafer SRAM figures and not VRAM or a bandwidth roofline input, and then LEFT BOTH FIELDS POPULATED with them. Prose does not constrain a numeric field: any query, join or sort over vram_gb still returned 44 for this part, and any roofline computed from memory_bandwidth_gbps still used 21000. The notes were load-bearing for a human reader and invisible to every other reader in the repo, which is the worse of the two failures. vram_gb: 44 -> null. Cerebras's 44 GB is ON-WAFER SRAM, and the WSE-3 has no DRAM tier. There is no honest value for a field defined as 'HBM or VRAM per device/package'. The 44 GB is not lost to the corpus: schemas/accelerator.schema.json gained an onchip_cache object in 2026-10-05 whose sram_mb sub-field exists for exactly this, and it is now populated below with kind 'sram-feed-tier'. A reader who wants the capacity gets it from a field whose name says what it is; a reader who wants vram_gb gets null, which is the truth. memory_bandwidth_gbps: 21000 -> null. 21 PB/s is on-wafer SRAM bandwidth. The WSE-3 has no memory bus and no off-package memory, so there is no DRAM-tier bandwidth to report. This is not 'the vendor did not publish it' - the vendor published a large number, just not one of this quantity. Keeping it here would let a roofline be computed that is meaningless for a dataflow part. memory_bus_bit: already null and stays null, for the structural reason the notes above state rather than the 'not published' reason. tdp_w 27000: unchanged, and still the CS-3 SYSTEM's 27 kW max from the datasheet, not a wafer figure. It is a system figure in a per-part field, which is a milder version of the same problem and is left in place because the datasheet figure is the only published power number and the notes name it as a system figure. WHY null AND NOT A NEW FIELD: the schema has no way to say 'the number exists and it is not this quantity', so the honest states are null-with-reason or a wrong number. This pass chose null-with-reason. A reader who wants the SRAM bandwidth must go to onchip_cache and to the CS-3 datasheet source, and the source records still carry the 21 PB/s figure verbatim so nothing is lost from the corpus. WHAT A ROOFLINE READER SHOULD DO INSTEAD, because 'this part has no roofline' is a finding and not a dead end: the binding constraint on a WSE is that the model must fit in on-wafer SRAM, plus static compile-time scheduling and the absence of dynamic batching. Bandwidth is not the limit for a model that fits, because there is no off-package memory to stream weights from. That is the opposite of the GPU situation, where bandwidth is the limit precisely because there IS an off-package tier. data/flops/alt-cerebras-limitations and data/flops/alt-cerebras-wafer-sram already carry this reasoning, and this record's new nulls are what make a numeric reader land on the same conclusion instead of on 21000 GB/s.

=== SOURCE DECLARED 2026-10-08 (w8-reverify). === flops[].source_id cited 'vb-cerebras-cs3-datasheet-125pf-sparse' on 1 row(s), and that source file exists and was read — but it was never added to this record's sources[], so the figure carried per-row provenance that the record-level source list did not back. AGENTS.md rule 2 requires every cited source to be resolvable, and it was, but the reverse held too: a source a record actually cites is a source the record must declare. Added now. No number changed. validate.py has never caught this class of defect — it checks that sources[] entries resolve, never that a cited source_id appears in sources[] — which is why 121/121 tests passed over it.

=== vram_gb NULL RE-VERIFIED 2026-10-08 (w8-reverify) — THE 2026-10-05 CORRECTION HOLDS ON THIS RECORD. === The re-verification brief named this the highest-risk change in wave 7, and the test it names is the right one: if the vendor publishes external memory for the part, null was wrong. It was checked against every Cerebras first-party page this repo can reach, and the conclusion is that the null stands. THE CS-3 DATASHEET, read live 2026-10-08, has one memory row and it is on-chip: 'Processor Memory | 44 GB total, 21 PB/s SRAM bandwidth', and the processor row repeats '21 PB/s on-chip SRAM bandwidth (44 GB on-chip SRAM)'. There is no DRAM row, no HBM row and no external-memory row. The CS-3 system page likewise lists '44GB On-Chip SRAM' and '21PB/s Memory Bandwidth' and nothing external. WHAT CEREBRAS DOES PUBLISH THAT LOOKS LIKE EXTERNAL MEMORY, AND WHY IT IS NOT ONE: the CS-3 announcement blog publishes MemoryX attachment capacities - 24 TB, 36 TB, 120 TB and 1,200 TB - and says 'The CS-3 can be configured with up to 1,200 terabytes of external memory'. Those are separate MemoryX UNITS for CLUSTER configurations, with no per-part bandwidth published anywhere, and the smallest of them is 545x this wafer's own 44 GB. Nothing in them licenses a per-device vram_gb value. Both the absence and the near-miss are recorded in source w8r-cerebras-cs3-blog-memoryx-external-memory, added to this record, so a future pass does not have to re-establish either. SO THE RECORD'S OWN REASONING STANDS, WITH ONE WORD SHARPENED: the 2026-10-05 audit wrote that this is a part with 'no DRAM tier', which is true of the WSE-3 as a part but sits next to a vendor that sells external memory for the systems it is installed in. The load-bearing distinction is between 'the vendor publishes no per-part DRAM figure' (true) and 'the vendor publishes no external-memory figures' (false). The null was reached for the right reason; the prose next to it should not invite the wider reading. The 44 GB is and stays in onchip_cache.sram_mb with kind 'sram-feed-tier', and the 21 PB/s stays preserved verbatim in the two source records. NO FIELD CHANGED VALUE in this re-verification - only the sources list and this note.

Cross-references

Interconnect (1)