You Only Cache Once: Decoder-Decoder Architectures for Language Models

YOCO: a self-decoder whose global KV cache is computed once and a cross-decoder that consumes it by cross-attention, so the growing per-token cache is stored once rather than per layer. Claims 'orders of magnitude' improvement in inference memory, prefill latency and throughput, and an early-exit prefill.