Conformer: Convolution-augmented Transformer for Speech Recognition

Transformers model global context but miss the local patterns of audio; CNNs do the opposite.

Macaron-style block: feed-forward half, multi-head self-attention half, and a CONVOLUTION MODULE sandwiched between them, with the convolution applied to the residual branch only. Depthwise separable convolution captures local structure at a fraction of the attention cost, so the sequence length can be lowered by strided downsampling between blocks.

2.1%/4.3% WER on Librispeech test-clean/test-other without a language model, 1.9%/3.9% with an external LM; a 10M-parameter model still reaches 2.7%/6.3%. Quality-only: the paper reports no wall-clock latency or hardware throughput, so there is no measured serving speedup to record.

Verified in the ISCA archive at interspeech_2020/gulati20_interspeech.html. Conformer is the encoder inside essentially every non-streaming ASR stack built since 2020 (NVIDIA NeMo, ESPnet, WeNet, sherpa-onnx) and is the direct ancestor of Zipformer. The stride pattern is the inference-relevant part: frame rate halves at each downsampling stage, so a 30s utterance produces a few hundred encoder frames rather than 3000, and every downstream layer is correspondingly cheaper. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'main'. This is STRUCTURAL rather than researched: interspeech has no Findings volume and no workshop track in which a paper could appear, so 'main' is the only possible honest value and no per-paper evidence was needed. The field is annotated here so that 'findings' and 'workshop' remain MEANINGFUL - a filter for findings=true must return only the papers that actually are findings, which requires every other paper to be explicitly NOT findings.