inference/research
verifiedhigh · 0.85updated 2026-10-05

Cerebras WSE-3: wafer-scale SRAM bandwidth advantage for inference

compute bound

Fields

id
flops/alt-cerebras-wafer-sram
Class
custom_kernel
Bound by
compute
Arithmetic intensity
The WSE-3 packs 44 GB of on-chip SRAM at 21 PB/s bandwidth (CS-3) or 43.2 PB/s (CS-4 Turbo). This is ~1000x the bandwidth of GPU HBM. Weights stay on-chip — there is no weight streaming from HBM. Tokens flow uninterrupted through model layers pipelined across wafers. The bandwidth advantage is most pronounced for decode, where GPU inference is bottlenecked by memory bandwidth. Cerebras claims up to 30x faster inference than GPUs for frontier models.
Scales with
model_size (must fit in 44 GB SRAM per wafer, or be pipelined across wafers)sram_bandwidth (21-43.2 PB/s, ~1000x GPU HBM)num_wafers (CS-4 has 3 wafers for larger models)token_generation_rate (up to 750 tok/s for GPT-5.6 Sol Ultrafast)
Hardware parameters
parametermemory_bandwidtheffectThe bandwidth advantage is most pronounced for decode, where GPU inference is bottlenecked by memory bandwidth.basisunverifiedvalues_forcerebras-wse-3
Workarounds
Pipeline model layers across multiple wafers for models larger than 44 GBUse the wafer-scale SRAM to eliminate HBM bottleneck entirelyLeverage 2 microsecond wafer-to-wafer latency for multi-wafer modelsUse disaggregated inference (prefill on one system, decode on another)
notes
The bandwidth advantage is the entire architectural point. On GPUs, inference is bottlenecked by memory bandwidth — model weights must be repeatedly transferred from HBM to compute. Cerebras eliminates this by putting all weights in SRAM. The 21 PB/s bandwidth means the memory bottleneck is effectively removed. The tradeoff is capacity: 44 GB is small compared to GPU HBM (80-192 GB), so very large models need multi-wafer pipelining. The CS-4's 3 wafers and 2 microsecond wafer-to-wafer latency make this practical. Measured: GPT-5.6 Sol Ultrafast at 750 output tokens/sec. HARDWARE-PARAMETER ROWS ADDED 2026-10-05 (schema authority pass, agent w5-schema). `affected_by_hardware` names PRODUCTS (accelerator record ids); these rows name the hardware QUANTITIES that move this class's cost, which had nowhere to live until flop.hardware_parameters was added in this commit. Every effect below is lifted verbatim from this record's own existing prose and no figure here is new - `magnitude` reproduces numbers the record already quoted. `basis` is 'unverified' where the effect was asserted with nothing concrete behind it, rather than being dressed up. The rows were pulled out of free-text notes; they are not a change to what this record claims.

Notes

The bandwidth advantage is the entire architectural point. On GPUs, inference is bottlenecked by memory bandwidth — model weights must be repeatedly transferred from HBM to compute. Cerebras eliminates this by putting all weights in SRAM. The 21 PB/s bandwidth means the memory bottleneck is effectively removed. The tradeoff is capacity: 44 GB is small compared to GPU HBM (80-192 GB), so very large models need multi-wafer pipelining. The CS-4's 3 wafers and 2 microsecond wafer-to-wafer latency make this practical. Measured: GPT-5.6 Sol Ultrafast at 750 output tokens/sec.

HARDWARE-PARAMETER ROWS ADDED 2026-10-05 (schema authority pass, agent w5-schema). `affected_by_hardware` names PRODUCTS (accelerator record ids); these rows name the hardware QUANTITIES that move this class's cost, which had nowhere to live until flop.hardware_parameters was added in this commit. Every effect below is lifted verbatim from this record's own existing prose and no figure here is new - `magnitude` reproduces numbers the record already quoted. `basis` is 'unverified' where the effect was asserted with nothing concrete behind it, rather than being dressed up. The rows were pulled out of free-text notes; they are not a change to what this record claims.

Cross-references

Hardware-dependent (1)