MiniLLM: On-Policy Distillation of Large Language Models

Distilling a WHITE-BOX LLM into a smaller LLM is under-explored - most KD work either distills classifiers or only imitates a black-box API - and forward KL lets the student waste capacity imitating low-probability regions of the teacher.

Two changes. (1) REVERSE Kullback-Leibler divergence instead of forward KL: forward KL is mass-covering and makes the student spread probability over the teacher's low-probability tails, which is exactly the wrong inductive bias for sequence generation. Reverse KL is mode-seeking and yields a sharper, more precise student. (2) An ON-POLICY optimization derivation that makes this objective trainable - the student samples its own outputs and is scored against the teacher on those, removing the train/inference distribution mismatch.

Paper's own claim (abstract): students generate MORE PRECISE RESPONSES with higher overall quality, LOWER EXPOSURE BIAS, BETTER CALIBRATION, and higher long-text generation performance than the baselines, in the instruction-following setting. No numeric accuracy delta or speedup ratio appears in the abstract. 'Better calibration' is the underrated claim here: a well-calibrated student does not need to be retrained before it can be quantized or served under distribution shift.

arXiv ID 2306.08543 verified via /abs/; the arXiv comment field reads 'Published as a conference paper in ICLR 2024' while Semantic Scholar returns an EMPTY venue field - the arXiv comment is the evidence used here, and both agree on the paper. Year = venue year 2024 (v1 2023-06-14). Distinct from GKD (2306.13649), which generalizes the on-policy idea to arbitrary student-teacher losses and RLHF integration. arXiv ID and title verified by fetching https://arxiv.org/abs/2306.08543 and reading the citation_title meta tag on 2026-10-03. Venue and year from the Semantic Scholar graph API batch endpoint (paper/arXiv:2306.08543) unless the record's notes say otherwise. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'main'. This is STRUCTURAL rather than researched: iclr has no Findings volume and no workshop track in which a paper could appear, so 'main' is the only possible honest value and no per-paper evidence was needed. The field is annotated here so that 'findings' and 'workshop' remain MEANINGFUL - a filter for findings=true must return only the papers that actually are findings, which requires every other paper to be explicitly NOT findings.