Architecture-conforming engines remove the decode GEMM cost model entirely: Groq, Cerebras, TPU

THE MODEL DEFAULT IS WRONG FOR THESE ENGINES IN A WAY THAT IS NOT A CONSTANT ERROR. The standard decode cost model (fct-batch1-step-cost-formula) is T = max(2*P_t/BW, 2*B*P_t/PEAK): the memory-bound term 2*P_t/BW comes from STREAMING THE WEIGHTS once per step, and the compute-bound term from doing B rows of work against them. Both terms presuppose a memory hierarchy where weights live off-chip and must be fetched. An architecture-conforming engine removes that premise. Groq: SRAM-only, no off-chip HBM tier, so there IS no weight-streaming path - a model larger than 230 MB must be tiled across chips and the tapeout is fixed at compile time, and the compiler statically schedules ALL data movement, which is why static shapes are mandatory (no dynamic batching, no variable-length sequences without bucketing, no data-dependent control flow). Cerebras: 44 GB of on-chip SRAM per wafer at 21 PB/s (CS-3) or 43.2 PB/s (CS-4 Turbo), weights stay on-chip, no HBM streaming, tokens flow uninterrupted through layers pipelined across wafers. TPU: an on-chip Cmem SRAM feed tier added specifically so that lower batch sizes are viable, and Google's own reason for adding 8-bit mode support and faster MXU weight-loading. THE DOMINANT FLOP CLASS CHANGES ACCORDINGLY. Where the default profile says decode is bound_by memory (a weight stream), these engines make the dominant term the LAUNCH/SCHEDULING and INTERCONNECT terms and the SRAM feed, not an HBM read. Concretely: Groq's constraint class is recorded as bound_by launch_overhead because a fully statically-scheduled datapath has no dynamic launch cost at all but a fixed compile-time schedule whose granularity is the bucket; Cerebras is recorded bound_by compute only because 21 PB/s of on-chip bandwidth removes the memory term by construction. THE CAPACITY CONSEQUENCE A PLANNER MUST NOT MISS: on these engines batch size is not a free runtime knob. Groq's compiler requires static shapes, so variable concurrency requires BUCKETING - a fixed set of precompiled batch shapes - which means the decode step cost is a step function of batch, not the smooth max() curve the standard model predicts.

Design consequence, and the class of gap this record fills: every other flop record in the repo describes how an engine executes the MODEL DEFAULT arithmetic faster or slower. These three engines change which terms exist. The 2*P_t/BW weight-stream term - the term that makes decode memory-bound at batch 1 on a GPU, the term the entire serving literature is organised around - is ABSENT by construction when weights are resident in on-chip SRAM. That is why the standard 'decode is memory-bound' intuition fails on Groq and Cerebras, and why a batch-1 tok/s figure from one of them is not comparable to a GPU figure even after normalising for precision and model size. THE HONEST NEGATIVES, which are the substance of this record: (1) Cerebras publishes NO per-chip throughput figure for standard benchmarks - no MLPerf, no third-party throughput comparison; the 750 tok/s figure in this repo is for one specific model (GPT-5.6 Sol Ultrafast) and is a vendor claim, so NO per-chip tok/s is asserted here. (2) Groq publishes NO per-chip compute figure at all on its own site; the 188 TFLOPS FP16 / 750 TOPS INT8 figures in the accelerator record are press-attributed and flagged basis:'unverified' in the 2026-10-04 backfill, so they are NOT used as inputs to anything here. (3) The wafer-scale architecture is optimised for single-stream low-latency decode, NOT high-throughput batched inference, so it is the wrong engine for the workload the default cost model describes. (4) No vLLM, no PyTorch native support on Cerebras; SDK only, and gated behind registration. WHAT A READER SHOULD TAKE AWAY: for these engines the question 'what is the roofline crossover batch B*' has no answer in the standard form, and the substitute question is 'what are the precompiled batch buckets and how much padded flops does each waste' - which is a compile-time question, not a runtime one. Engine records: [[engines/alt-groq-groqflow]], [[engines/alt-cerebras-csl-sdk]], [[engines/alt-tpu-inference-vllm-plugin]], [[engines/alt-jetstream]]. Existing flop records for the same engines: [[alt-groq-compiler-constraints]], [[alt-groq-deterministic-architecture]], [[alt-cerebras-wafer-sram]], [[alt-cerebras-limitations]], [[alt-tpu-xla-static-shapes]], [[alt-tpu-bucketing-padding]], [[alt-tpu-mfu-performance-rules]]. The default decode model being departed from: [[fct-batch1-step-cost-formula]]. HARDWARE-PARAMETER ROWS ADDED 2026-10-05 (schema authority pass, agent w5-schema). `affected_by_hardware` names PRODUCTS (accelerator record ids); these rows name the hardware QUANTITIES that move this class's cost, which had nowhere to live until flop.hardware_parameters was added in this commit. Every effect below is lifted verbatim from this record's own existing prose and no figure here is new - `magnitude` reproduces numbers the record already quoted. `basis` is 'unverified' where the effect was asserted with nothing concrete behind it, rather than being dressed up. The rows were pulled out of free-text notes; they are not a change to what this record claims.