HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units

Speech has no lexicon and no fixed segmentation, so masked prediction needs a target definition before the loss can be written.

Two-stage: pretrain a wav2vec 2.0-style model, then cluster its latent features with k-means offline (100 clusters, two refinement iterations), then train a BERT-style masked prediction of the cluster IDs over masked spans only. The prediction loss on masked regions forces a combined acoustic and language model.

Matches or beats wav2vec 2.0 on Librispeech (960h) and Libri-light (60,000h) at 10min/1h/10h/100h/960h fine-tune budgets, with the 1B-parameter model improving further. A WER comparison only; the paper states no latency, FLOP or hardware figures, so this record carries no speedup claim.

Venue is IEEE/ACM TASLP vol. 38 issue 1 (Jan 2021) pp. 42-52, confirmed via crossref; the schema's venue enum has no TASLP entry, so 'other' is the honest encoding rather than 'icassp'. HuBERT encoders are shipped in several production ASR stacks as a frozen feature extractor, which is where the inference benefit lands: the encoder runs once per utterance and its output is cached across a beam search. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'main'. Peer-reviewed venue confirmed as IEEE/ACM TASLP 2021 (same). This is an additive track annotation; `venue` is unchanged at 'other' and no status change is implied by the track alone. VENUE RECLASSIFIED 2026-10-10 (crawler/w11-venue-enum): venue 'other' -> 'taslp', venue_kind 'journal'. Confirmed by record notes and agreeing publisher/index deposit in data/sources/.