Unsupervised Dense Information Retrieval with Contrastive Learning (Contriever)

Dense retrievers needed labelled query-passage pairs and did not transfer to domains with no training data.

Unsupervised SimCSE-style contrastive pretraining of a BERT bi-encoder on large unlabelled text chunks, then optional fine-tuning on small in-domain data. Uses the same in-batch-negatives dual-encoder design as DPR but with the supervision removed.

On BEIR, the unsupervised model beats BM25 on 11 of 15 datasets at Recall@100, and pretraining then fine-tuning improves BEIR further on both few-thousand-example in-domain data and full MS MARCO. The '11 of 15 datasets' phrasing is the honest form of the claim - it does not dominate, and MTEB's own finding that no single method dominates across tasks is consistent.

VENUE CORRECTION, verified rather than assumed: this is Transactions on Machine Learning Research (Semantic Scholar venue 'Trans. Mach. Learn. Res.'), NOT NeurIPS 2022 - I checked the NeurIPS 2022 proceedings index for the title and it is absent. It is commonly miscited as a NeurIPS paper. The schema has no TMLR entry, so 'other' is the honest encoding. Relevance to serving: it established that a useful dense retriever can be trained with NO labels, which is what makes domain-specific embedding indexes affordable to build at all. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'main'. Peer-reviewed venue confirmed as TMLR (same). This is an additive track annotation; `venue` is unchanged at 'other' and no status change is implied by the track alone. VENUE RECLASSIFIED 2026-10-10 (crawler/w11-venue-enum): venue 'other' -> 'tmlr', venue_kind 'journal'. Confirmed by record notes and agreeing publisher/index deposit in data/sources/.