wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations

Representation learning from raw audio needed a contrastive objective that scales without labelled transcripts.

Same CNN feature encoder as wav2vec, but masks spans in the LATENT space (not the raw signal) and solves a contrastive task against a learned product-quantised codebook, discarding the quantiser at fine-tune time so the inference graph is encoder-plus-LM only.

1.8/3.3 WER on Librispeech clean/other with all labelled data; 4.8/8.2 WER with only ten minutes of labelled data and 53k hours of unlabelled pre-training. With one hour of labels it beats prior SOTA trained on 100 hours. Quality baseline is Librispeech, hardware not stated.

Venue NeurIPS 2020 (openalex records the 'Neural Information Processing Systems' container for this title). Whisper and most current ASR do NOT use wav2vec 2.0 pretraining, so it is upstream of research lineages rather than of shipped models; the strong version of the claim is that latent masking and codebook contrast remain the default self-supervised recipe. Latent masking is why the encoder cost is essentially fixed by window length regardless of content. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'main'. This is STRUCTURAL rather than researched: neurips has no Findings volume and no workshop track in which a paper could appear, so 'main' is the only possible honest value and no per-paper evidence was needed. The field is annotated here so that 'findings' and 'workshop' remain MEANINGFUL - a filter for findings=true must return only the papers that actually are findings, which requires every other paper to be explicitly NOT findings.