KV-cache tiering (LMCache): deciding, outside the kernel, how much KV the attention kernel must read from HBM

Not an arithmetic mechanism at all, which is why it matters as a kernel-level decision input. A KV tier between the GPU and host/remote storage changes how many bytes the decode attention kernel pulls from HBM, but it does not change the attention FLOPs at all. The strongest version of the effect is prefix reuse: if a shared prefix KV is resident in GPU memory because a cache tier kept it there, every request reading that prefix does zero HBM traffic for it. So the practical AI of decode attention is set by cache hit rate, not only by the kernel's tiling.

Recorded because it is the largest single lever on decode-attention bytes that lives entirely outside the attention kernel, and because it interacts with kernels rather than being orthogonal to them: a paged KV layout (see the paged-attention record) is what makes selective block eviction and prefix sharing possible at all, and a quantized KV format changes what can be transferred cheaply. LMCache itself is a cache/movement layer, not a GPU kernel library, and no performance number is taken from it here. HARDWARE-PARAMETER ROWS ADDED 2026-10-05 (schema authority pass, agent w5-schema). `affected_by_hardware` names PRODUCTS (accelerator record ids); these rows name the hardware QUANTITIES that move this class's cost, which had nowhere to live until flop.hardware_parameters was added in this commit. Every effect below is lifted verbatim from this record's own existing prose and no figure here is new - `magnitude` reproduces numbers the record already quoted. `basis` is 'unverified' where the effect was asserted with nothing concrete behind it, rather than being dressed up. The rows were pulled out of free-text notes; they are not a change to what this record claims.