Towards General Text Embeddings with Multi-stage Contrastive Learning (GTE)

Every embedding task was served by its own specialised model, so a single general-purpose encoder was unavailable.

Multi-stage contrastive training over a diverse mixture of datasets from many sources, with a large-scale unsupervised pre-training stage followed by supervised fine-tuning; one model covers classification, clustering, STS, reranking and retrieval, and treats CODE as plain text rather than a separate task.

gte_text_base at 110M parameters is reported to outperform the OpenAI embedding API and to surpass 10x larger embedding models on MTEB, and to beat previous code retrievers of similar size without per-language fine-tuning. Vendor-reported quality against a black-box API with no cost comparison - note that the API cost is exactly what the Embedder's Dilemma paper later measures.

VENUE: Semantic Scholar records this title's venue as arXiv.org only; no peer-reviewed venue found, so recorded as a preprint. GTE became the default embedder inside several serving products and is the base model for later multilingual variants (mGTE, recorded in the source file pap-mm-giga-embeddings' sibling set). The general-task framing matters operationally: one encoder replaces a task-per-model fleet, which shrinks the total resident weight footprint of an embedding service. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'unknown'. `venue` is 'arxiv-preprint', which means no peer-reviewed venue has been established for this work as of the verification date, so the track genuinely is not known. 'unknown' is used here rather than leaving the field null or guessing 'main': the schema separates 'unknown' from 'main' precisely so that an unprereviewed preprint is not silently counted as main-track work. This annotation is what lets a downstream filter distinguish 'peer reviewed, main track' from 'not peer reviewed' without inferring it from the venue string.