ColBERT / PLAID late-interaction retrieval
cuda, cpu
Fields
id
engines/niche-colbertRepo
stanford-futuredata/ColBERT
Languages
PythonC++CUDA
License
MIT
First release
2020
Design goal
researchthroughput-firstretrieval
Scheduling
none
Backends
cudacpu
Formats
safetensorspytorch-checkpoint
Notable features
A different inference shape, not a different engine: passages are encoded into matrices of token-level embeddings rather than single vectors, and query-document scoring is a MaxSim over those matrices. The retrieval-quality gain comes from that fine-grained late interaction (SIGIR'20)PLAID (CIKM'22, arXiv:2205.09707) is the systems contribution: it is what makes late interaction tractable by using approximate candidate generation so only a fraction of token-level scores are ever computedColBERTv2 (NAACL'22, arXiv:2112.01488) added lightweight late interaction and doclength-aware scoring, and the released ColBERTv2.0 checkpoint is trained on MS MARCO Passage RankingShips the LoTTE benchmark alongside the checkpoint, so the quality claim is checkable rather than assertedIndexer/Searcher API: index your corpus once into matrices, then serve MaxSim queriesIndexUpdater and additional HuggingFace model support merged in January 2023, still flagged beta by the README3944 stars, MIT, v0.2.22 released 2025-08-11, last push 2025-10-14, 290 open issues
Hardware caveats
STORAGE, NOT MEMORY, IS THE BINDING CONSTRAINT. An index is one embedding matrix row per token of every passage, so index size scales with corpus token count. This is the practical reason late interaction is expensive and the reason PLAID existsFAISS IS A HARD DEPENDENCY and the README's warning is the operational fact: 'pip install colbert-ai[torch,faiss-gpu]' usually works but 'if you face issues conda is always more reliable for faiss and torch'. This is the single most common ColBERT install failureA separate conda_env_cpu.yml exists for CPU-only environments, and the README warns that testing CPU execution on a machine that has GPUs may need extra care - i.e. the CPU path is genuinely secondaryPython 3.7+ and PyTorch 1.9+ floor per the READMERequires a downloaded checkpoint, not a safetensors file from the Hub: the ColBERTv2.0 checkpoint is a tar.gz from Stanford's own downloads server (downloads.cs.stanford.edu). Offline or air-gapped installs need manual stagingNot a serving engine: no HTTP API, no batching story, no continuous batching, no KV cache, scheduling is 'none'. It is a library plus an index formatNot a drop-in for a vector database. ColBERT indexes are not comparable to single-vector indexes and cannot be queried with pgvector or a stock ANN serviceCUDA and CPU only in the README; no ROCm, Metal or Vulkan pathLast push 2025-10-14, so the code is maintained but at a much lower cadence than the engines here. Research software cadence, not production cadence
Best for
Retrieval quality above what single-vector embeddings deliver, when recall on hard queries matters more than index size or query latencyBuilding or reading a late-interaction index, and understanding why MaxSim scoring costs what it costsStudying PLAID as the template for making an expensive scoring function affordable - approximate candidate generation as a systems leverUsing LoTTE + the released ColBERTv2 checkpoint as a checkable baseline for any new retrieval method
Avoid for
Memory-constrained or cost-sensitive index storage, which is the fundamental trade late interaction makesOperational simplicity - the FAISS/torch dependency conflict and the manual checkpoint download are real frictionDropping into an existing vector database workflow; it is a different index format with a different lifecycleProduction serving of retrieval itself without adding a service layer around it
notes
Included because late-interaction retrieval is a distinct inference workload with distinct hardware needs - the scoring is a per-token matrix reduction, not a generation loop - and it is routinely confused with the generation engines. PLAID arXiv:2205.09707 (2022-05-19), ColBERT arXiv:2004.12832 (2020-04-27) and ColBERTv2 arXiv:2112.01488 (2021-12-02) all verified via the arXiv API. Related: [[niche-text-embeddings-inference]] and [[niche-infinity]] serve the single-vector embedding case that ColBERT is an alternative to; Infinity can serve ColBERT-family models but ColBERT here is the reference implementation of the late-interaction idea. ||| MAINTENANCE (dated fact, GitHub API, 2026-10-04): last commit 2025-10-14 (11.7 months before this record), latest release v0.2.22 published 2025-08-11, 43 contributors (distinct committers including anonymous), open_issues_count 105 (issues PLUS pull requests - see [[ev-github-open-issues-count-includes-pull-requests]]), not archived, 3944 stars. See [[ev-maintenance-status-is-four-dated-facts]] for how to read this fact set.
Notes
Included because late-interaction retrieval is a distinct inference workload with distinct hardware needs - the scoring is a per-token matrix reduction, not a generation loop - and it is routinely confused with the generation engines. PLAID arXiv:2205.09707 (2022-05-19), ColBERT arXiv:2004.12832 (2020-04-27) and ColBERTv2 arXiv:2112.01488 (2021-12-02) all verified via the arXiv API. Related: [[niche-text-embeddings-inference]] and [[niche-infinity]] serve the single-vector embedding case that ColBERT is an alternative to; Infinity can serve ColBERT-family models but ColBERT here is the reference implementation of the late-interaction idea. ||| MAINTENANCE (dated fact, GitHub API, 2026-10-04): last commit 2025-10-14 (11.7 months before this record), latest release v0.2.22 published 2025-08-11, 43 contributors (distinct committers including anonymous), open_issues_count 105 (issues PLUS pull requests - see [[ev-github-open-issues-count-includes-pull-requests]]), not archived, 3944 stars. See [[ev-maintenance-status-is-four-dated-facts]] for how to read this fact set.
Cross-references
This record references no other records — it is a leaf. It is cited by 0.