RouteLLM: Learning to Route LLMs with Preference Data

Calling one strong model for every query is wasteful when most queries are easy, so routing between a strong and a weak model trades a little accuracy for a large cost cut.

Fit a small BERT-based classifier on preference data — pairwise win/loss judgements between a strong and a weak model's answers on the same prompt — rather than on the prompt text alone, so the router learns where the strong model actually adds value instead of where prompts look hard. Ships a cost-threshold knob to move the accuracy/cost operating point.

Per the arXiv abstract: significantly reduces costs — by over 2 times in certain cases — without compromising response quality, on widely recognised benchmarks. The savings are on API price for choosing a weaker model, NOT on measured GPU throughput. The paper additionally reports that the routers transfer: they keep working when the strong/weak pair is swapped at test time. Note the v1 abstract does not state a win-rate threshold; the 'over 95% of GPT-4's win rate' figure appears in the paper body, so quote the abstract's 'over 2x cost reduction' unless you have checked the specific table.

Venue verified on OpenReview: content.venue = 'ICLR 2025 Poster', with full reviews attached. Title wrinkle: the arXiv version says 'Learning to Route LLMs WITH Preference Data'; the OpenReview title says 'FROM Preference Data'. Adoption in-production is defensible — lm-sys released the router and it is a real deployment tool for API-cost reduction — but it is a TOOL people opt into, not something embedded in vLLM or SGLang. Set expectations accordingly. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'main'. This is STRUCTURAL rather than researched: iclr has no Findings volume and no workshop track in which a paper could appear, so 'main' is the only possible honest value and no per-paper evidence was needed. The field is annotated here so that 'findings' and 'workshop' remain MEANINGFUL - a filter for findings=true must return only the papers that actually are findings, which requires every other paper to be explicitly NOT findings.