SimCSE: Simple Contrastive Learning of Sentence Embeddings

Supervised sentence-embedding methods needed a separately trained model per dataset, so embeddings did not transfer.

Unsupervised variant: drop the same sentence into the encoder twice and predict one from the other under a contrastive objective, using ONLY STANDARD DROPOUT as the noise source. Supervised variant: use NLI entailment pairs as positives and contradiction pairs as hard negatives. Also shows the objective regularises the pretrained embedding space away from anisotropy.

Unsupervised and supervised BERT-base models average 76.3% and 81.6% Spearman correlation on standard STS tasks, improvements of 4.2% and 2.2% over the previous best results. This is a quality delta on STS benchmarks, not a cost or latency measurement.

Venue EMNLP 2021 (crossref ACL proceedings; arXiv comment 'Accepted to EMNLP 2021'). Inference cost is identical to SBERT with the same base model - the contribution is entirely in the training objective, which is worth stating plainly because SimCSE is sometimes misread as an architecture change. It is a drop-in quality upgrade to an already-deployed encoder at zero serving cost. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass): venue_track set to 'main' from the ACL Anthology bib index (anthology.bib, 131650 entries) by normalized title match. Evidence: anthology id 2021.emnlp-main.552, booktitle 'Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing'. The Anthology id is the authoritative signal here and it is unambiguous, so this is an additive clarification of a track, not a change of venue - `venue` is untouched and no status change is implied. The reason this matters for inference readers: a Findings paper and a main-track paper at the same venue are not the same kind of evidence, and until this field existed the repo could not say which was which.