FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance

API pricing across strong models differs by orders of magnitude, so naively calling an expensive model for every query is economically indefensible at scale.

A cost-and-accuracy study rather than a system: it measures cost and quality across a ladder of models and then shows three strategy families — LLM cascade (fall back to a cheaper model when the strong one is unsure), LLM router (pick the model up front) and LLM approximation (reuse a cached/smaller answer) — each with its own Pareto curve.

Per the paper's framing: up to two orders of magnitude cost reduction while matching or improving accuracy over the strongest single model, across published workloads. These are API-PRICE savings against a baseline of always calling the most expensive model — not throughput, latency or GPU utilisation gains. The paper is a study of three strategy families (cascade, router, approximation), each with its own Pareto curve, so a single '2x' is only meaningful once the strategy and budget are named.

Venue verified on OpenReview: content.venue = 'Accepted by TMLR'. Recorded venue as 'other' because the schema's enum has no TMLR value; the TMLR acceptance is stated here so a reader does not have to guess. Adoption research-only, and this is a case where the citation count badly overstates the adoption: it is widely cited as the origin of LLM cascades, but no engine ships a FrugalGPT cascade. Its taxonomy is the durable contribution. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'main'. Peer-reviewed venue confirmed as TMLR (same). This is an additive track annotation; `venue` is unchanged at 'other' and no status change is implied by the track alone. VENUE RECLASSIFIED 2026-10-10 (crawler/w11-venue-enum): venue 'other' -> 'tmlr', venue_kind 'journal'. Confirmed by record notes and agreeing publisher/index deposit in data/sources/.