Phi-4 Technical Report

Most capable LLMs are trained on organic data; previous Phi models distilled GPT-4 and plateaued at the teacher.

Data-quality-centred recipe with synthetic data woven through the WHOLE pipeline rather than only at the post-training step: synthetic pre-training data, synthetic reasoning traces/problems with known answers, synthetic data for post-training, plus data curriculum design and a modified post-training scheme - all on essentially the phi-3 architecture (minimal architectural change). The explicit claim is that the techniques go BEYOND distillation: the student surpasses its own teacher.

Paper's own claim (abstract): phi-4 is a 14B model that, 'while previous models in the Phi family largely distill the capabilities of a teacher model (specifically GPT-4), phi-4 SUBSTANTIALLY SURPASSES ITS TEACHER MODEL on STEM-focused QA capabilities', achieving strong performance relative to its size 'especially on reasoning-focused benchmarks', with minimal architecture change from phi-3. No compression ratio or throughput figure in the abstract; the contribution is capability-per-parameter, which is an inference-cost argument by implication rather than a measured one.

arXiv ID 2412.08905 verified via /abs/; venue: Semantic Scholar returns arXiv.org only and the arXiv comment field is empty - arxiv-preprint, year = v1 year 2024. Author list abbreviated (very long list on the arXiv page). adoption=in-production: phi-4 weights are publicly released and deployed in Microsoft Copilot and on Azure AI Foundry. arXiv ID and title verified by fetching https://arxiv.org/abs/2412.08905 and reading the citation_title meta tag on 2026-10-03. Venue and year from the Semantic Scholar graph API batch endpoint (paper/arXiv:2412.08905) unless the record's notes say otherwise. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'unknown'. `venue` is 'arxiv-preprint', which means no peer-reviewed venue has been established for this work as of the verification date, so the track genuinely is not known. 'unknown' is used here rather than leaving the field null or guessing 'main': the schema separates 'unknown' from 'main' precisely so that an unprereviewed preprint is not silently counted as main-track work. This annotation is what lets a downstream filter distinguish 'peer reviewed, main track' from 'not peer reviewed' without inferring it from the venue string.