Accelerating Sparse Deep Neural Networks

Sparsity research mostly targets high sparsity levels that no shipping matrix-math unit supports, so it produces compression without speedup.

Documents and designs the NVIDIA Ampere Sparse Tensor Core: a 2:4 semi-structured sparsity pattern (exactly 2 zeros in every group of 4 consecutive weights along the K dimension) so the hardware can drop half of every MAC while keeping a dense, regular memory layout - the metadata and pruning overhead is paid at training time instead of at inference time. Ships a training workflow (magnitude pruning plus fine-tuning to recover the accuracy lost to the constraint) that produces networks that satisfy the 2:4 requirement while matching unpruned accuracy.

Paper's own claim (abstract): the 2:4 (50%) sparsity pattern 'leads to TWICE THE MATH THROUGHPUT of dense matrix units' on Sparse Tensor Cores. That is a hardware claim, not a model claim, and it is the number that made 2:4 the default structured-sparsity pattern. COMPRESSION RATIO: 2x (50% of weights stored, zero-valued). QUALITY COST: the training workflow maintains accuracy on 'a wide range of common tasks and model architectures' - pruning to 2:4 is not free, the cost is paid in fine-tuning, which is why this became the pattern of choice for inference-side sparsity.

arXiv ID 2104.08378 verified by Semantic Scholar /paper/search/match (title exact) and cross-checked against the repo's flop-flashattention-era sources; NOTE the v1 date (2021-04-19) is NOT independently verified against the /abs/ page because arXiv was rate-limiting during this pass - flagged rather than guessed. Venue: Semantic Scholar returns arXiv.org only - arxiv-preprint. adoption=in-production is justified structurally: 2:4 is a HARDWARE constraint of Ampere-and-later Tensor Cores, so every deployment that uses it is running this design. arXiv ID and title verified by fetching https://arxiv.org/abs/2104.08378 and reading the citation_title meta tag on 2026-10-03. Venue and year from the Semantic Scholar graph API batch endpoint (paper/arXiv:2104.08378) unless the record's notes say otherwise. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'unknown'. `venue` is 'arxiv-preprint', which means no peer-reviewed venue has been established for this work as of the verification date, so the track genuinely is not known. 'unknown' is used here rather than leaving the field null or guessing 'main': the schema separates 'unknown' from 'main' precisely so that an unprereviewed preprint is not silently counted as main-track work. This annotation is what lets a downstream filter distinguish 'peer reviewed, main track' from 'not peer reviewed' without inferring it from the venue string.