wav2vec: Unsupervised Pre-training for Speech Recognition

Character-based acoustic models need large amounts of transcribed speech, so they do not generalise to rare domains or languages.

Raw audio waveform is passed through a 7-layer strided convolutional network; representations at the top layer are quantised into a codebook and a masked span is predicted by contrastive loss against the true quantised target plus negatives sampled from other time steps.

Reduces WER of a strong character log-mel filterbank baseline by up to 36% when only a few hours of transcribed data is available; reaches 2.43% WER on WSJ nov92 while using two orders of magnitude less labelled training data than the prior best character system (Deep Speech 2). Quality-only claim, no hardware stated.

Interspeech 2019 (ISCA archive; crossref container 'Interspeech 2019', DOI 10.21437/Interspeech.2019-2837). The waveform-in/waveform-out encoder is the origin of the raw-audio (rather than log-Mel) front end that production speech stacks still use, because it front-ends the whole compute cost before any transformer runs. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'main'. This is STRUCTURAL rather than researched: interspeech has no Findings volume and no workshop track in which a paper could appear, so 'main' is the only possible honest value and no per-paper evidence was needed. The field is annotated here so that 'findings' and 'workshop' remain MEANINGFUL - a filter for findings=true must return only the papers that actually are findings, which requires every other paper to be explicitly NOT findings.