WizardLM: Empowering Large Pre-Trained Language Models to Follow Complex Instructions

Instruction-following data is expensive to hand-write, and humans struggle to produce the high-complexity instructions that actually teach a model.

Evol-Instruct: iteratively REWRITE a seed instruction into progressively more complex ones (add constraints, deepen reasoning, concretize), producing a difficulty-graded instruction corpus without human authors. The generated instructions are mixed and used to fine-tune LLaMA. The method is a DATA-generation method, not an architecture change - which is why it composes with whatever base model you have.

Paper's own claim (abstract): human evaluation on a complexity-balanced testbed and Vicuna's testset shows Evol-Instruct instructions are SUPERIOR TO HUMAN-CREATED ONES; on the high-complexity subset WizardLM outputs are PREFERRED TO OPENAI CHATGPT; and in GPT-4 automatic evaluation WizardLM achieves 'more than 90% capacity of ChatGPT on 17 out of 29 skills'. The 17/29 is the honest headline - on 12 of 29 skills it did NOT reach 90% of ChatGPT. COMPRESSION RATIO: not applicable. QUALITY COST: a self-judged score against ChatGPT, with both the judge (GPT-4) and the baseline proprietary.

arXiv ID 2304.12244 verified via /abs/; the page's journal-ref field reads 'The Twelfth International Conference on Learning Representations (ICLR 2024)' - a rare case where the arXiv page itself carries the venue, and Semantic Scholar agrees. Year = venue year 2024 (v1 2023-04-24). arXiv ID and title verified by fetching https://arxiv.org/abs/2304.12244 and reading the citation_title meta tag on 2026-10-03. Venue and year from the Semantic Scholar graph API batch endpoint (paper/arXiv:2304.12244) unless the record's notes say otherwise. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'main'. This is STRUCTURAL rather than researched: iclr has no Findings volume and no workshop track in which a paper could appear, so 'main' is the only possible honest value and no per-paper evidence was needed. The field is annotated here so that 'findings' and 'workshop' remain MEANINGFUL - a filter for findings=true must return only the papers that actually are findings, which requires every other paper to be explicitly NOT findings.