ShortGPT: Layers in Large Language Models are More Redundant Than You Expect

Deep LMs contain many near-duplicate layers and some layers that barely affect function, but pruning them requires complex machinery.

Defines BLOCK INFLUENCE (BI), a metric for a layer's significance in the LLM, computed from similarity between a layer's input and output hidden representations (cosine similarity: BI = 1 - max cos-sim means the layer changes the representation most). Pruning is then plain LAYER REMOVAL - delete the lowest-BI layers outright - rather than structured pruning with masks or low-rank approximations.

Claims to significantly outperform prior state-of-the-art MODEL PRUNING methods, and is ORTHOGONAL to quantization-like methods, enabling further combined reduction in parameters and computation. NO numeric speedup multiplier is stated in the abstract, so speedup_reported deliberately records the qualitative claim and the orthogonality property rather than inventing a number. Pruning fraction and its effect on quality are in the paper body, not the abstract, and are not recorded here.

ADOPTION: research-only - layer removal is trivially implementable and hence widely TRIED, but no production engine ships a ShortGPT-derived depth configuration for any mainline checkpoint, because the layers to drop are checkpoint- and task-specific. INCLUDED because it is the cheapest possible depth-pruning result and therefore the control case against LayerSkip (pap-eff-layerskip) and Mixture-of-Depths (pap-eff-mixture-of-depths): it shows how much of the 'LLMs have redundant depth' claim is obtainable with NO training and NO early-exit machinery at all. VENUE NOTE: no venue verifiable via Semantic Scholar and no venue in the arXiv comment, so venue = arxiv-preprint, year = ArXiv v1 (Mar 2024). CAVEAT: pruning-by-removal is not lossless - quality degradation is traded against layer count - so it should never be cited as a lossless inference speedup, unlike every speculative-decoding record in this set. DANGLING REF RESOLVED 2026-10-04 by reading the paper (arXiv:2403.03853v1, Appendix H 'Hardware Environment'). The hardware table reads "CPU 2x Intel(R) Xeon(R) Gold 6430 CPU @ 2.1GHz / GPU 8x NVIDIA A100-80GB Tensor Core GPU". Source: https://arxiv.org/pdf/2403.03853 . 80 GB stated, 8-GPU node. AFFILIATIONS FILLED 2026-10-04 (paper-verification pass): from the Semantic Scholar graph API (paper/arXiv:2403.03853), which is the only machine-readable source that carries author affiliations — the arXiv Atom feed returns author names only, and no ORCID is exposed by either API. One known upstream typo corrected: "Tel Aviv Univesity" -> "Tel Aviv University". S2 affiliation coverage is partial (120 of 249 corpus records carry any), so these lists may be incomplete rather than exhaustive. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass): venue_track set to 'findings' from the ACL Anthology bib index (anthology.bib, 131650 entries) by normalized title match. Evidence: anthology id 2025.findings-acl.1035, booktitle 'Findings of the Association for Computational Linguistics: ACL 2025'. The Anthology id is the authoritative signal here and it is unambiguous, so this is an additive clarification of a track, not a change of venue - `venue` is untouched and no status change is implied. The reason this matters for inference readers: a Findings paper and a main-track paper at the same venue are not the same kind of evidence, and until this field existed the repo could not say which was which.