Zipformer: A faster and better encoder for automatic speech recognition

Conformer encoders were accurate but too slow and too memory-hungry for on-device and high-concurrency streaming ASR.

Four changes over Conformer: (1) U-Net-like encoder where middle blocks operate at LOWER frame rate, so most compute runs on short sequences; (2) block reorganisation with many more modules per block but fewer blocks, and REUSE OF ATTENTION WEIGHTS between modules in a block; (3) BiasNorm, a LayerNorm variant that retains some length/positional information; (4) SwooshR/SwooshL activations. Ships with ScaledAdam, an Adam variant that scales updates by each tensor's current scale. Code is in the k2/icefall toolchain.

Reports faster convergence and better accuracy than Adam for ScaledAdam, and Zipformer outperforming other SOTA ASR encoders on LibriSpeech, AISHELL-1 and WenetSpeech. IMPORTANT: the paper does NOT state a wall-clock speedup against Conformer with named hardware, so speedup_reported is left null rather than paraphrasing 'faster' - the title's claim is not a number.

TITLE CORRECTION worth carrying: the widely cited title 'Zipformer: A Zipper Architecture for Efficient Conformer-Based Speech Recognition' does not match the arXiv record. The arXiv entry 2310.11230 is titled 'Zipformer: A faster and better encoder for automatic speech recognition' and the venue is ICLR 2024 (conference paper, per the arXiv comment). Venue ICLR 2024 and the arXiv id were both verified. Zipformer is the default streaming encoder in the k2/icefall and WeNet stacks and is the encoder behind several production streaming ASR offerings; the frame-rate-reduction U-Net plus intra-block attention reuse is why it is cheaper than Conformer at equal WER. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'main'. This is STRUCTURAL rather than researched: iclr has no Findings volume and no workshop track in which a paper could appear, so 'main' is the only possible honest value and no per-paper evidence was needed. The field is annotated here so that 'findings' and 'workshop' remain MEANINGFUL - a filter for findings=true must return only the papers that actually are findings, which requires every other paper to be explicitly NOT findings.