BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models

End-to-end vision-language pretraining was prohibitively expensive because both towers had to be trained jointly.

Freezes a pretrained image encoder AND a frozen LLM, and bridges them with a lightweight QUERYING TRANSFORMER (Q-Former). Two stages: stage 1 bootstraps vision-language representation learning from the frozen image encoder (image-text contrastive, image-text matching, image-grounded text generation); stage 2 bootstraps vision-to-language generation from the frozen LM. Only the Q-Former is trained.

Outperforms Flamingo-80B by 8.7% on zero-shot VQAv2 while using 54x fewer trainable parameters. This is a TRAINING-efficiency claim (fewer trainable parameters), not an inference throughput measurement.

Venue ICML 2023, confirmed at proceedings.mlr.press v202/li23q. INFERENCE-ENABLING CONTRIBUTION: Q-Former is a learned bottleneck that emits 32 visual query tokens by default, regardless of image resolution. In a serving system the vision tower cost then splits cleanly into (a) one encoder pass at full resolution and (b) a tiny 32-token LM prefill. This is the production-friendly form of vision-token reduction - you reduce the LM's contribution to prefill to a constant and the cost floor becomes the encoder itself. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'main'. This is STRUCTURAL rather than researched: icml has no Findings volume and no workshop track in which a paper could appear, so 'main' is the only possible honest value and no per-paper evidence was needed. The field is annotated here so that 'findings' and 'workshop' remain MEANINGFUL - a filter for findings=true must return only the papers that actually are findings, which requires every other paper to be explicitly NOT findings.