Giga-Embeddings: Mixture-of-Experts Encoders for High-Throughput Text Embeddings

High-quality embedders were too slow and expensive to serve at high throughput, forcing quality-versus-cost compromises.

A sparse Mixture-of-Experts TEXT ENCODER - a 10B-parameter MoE with ~1.8B active parameters per token - so per-query cost tracks active parameters, not total. The family also ships a dense 3B encoder and a 480M distilled encoder trained with a dimension-agnostic objective aligning teacher and student similarity distributions.

In a vLLM benchmark with 1024-token inputs the 10B MoE processes 114.5k tokens/s: 25% higher throughput than the dense 3B model and 1.56-2.65x the throughput of the evaluated external systems. The 480M model scores 70.98 on Russian MTEB, beating FRIDA with 42% fewer parameters. vLLM, input length and baseline are all named - one of the few embedding papers that reports tokens/s with a serving engine attached.

Venue ACL 2026 (arXiv comment). RARE AND USEFUL because it reports throughput IN vLLM rather than as a synthetic microbenchmark, which means the numbers already include whatever batching behaviour the engine imposes. The MoE-for-encoders idea is the embedding analogue of serving a sparse LLM: total weights stay large for quality, active parameters per token stay small for cost, and vLLM's MoE path is what makes that work rather than hand-rolled routing. The distilled 480M member is the practical one for most deployments - a quality/cost Pareto point rather than a frontier record. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'main'. ACL has no Findings volume at this venue year in the record's evidence - the venue rests on the arXiv comment field reading 'ACL 2026', and the title is NOT present in the ACL Anthology bib index (fetched 2026-10-04), which is expected for a 2026 venue whose proceedings are still being posted. ACL's main track is therefore the only value the available evidence supports; flagged here rather than asserted as confirmed.