27 — Paper reading map: the 282 paper records by topic and adoption
The repo holds a paper record for every method that shapes an inference decision, but
until now almost none were cited from docs/. This map makes them navigable: one
section per category, papers ordered by adoption (what a reader can actually run
today first), then by year. The adoption field is the record's own claim — see each
record's adoption evidence before treating it as settled.
How to read it
| adoption | papers | meaning |
|---|---|---|
in-production |
76 | the method ships in a production system named in the record |
in-upstream-engine |
67 | merged into an open engine (vLLM, SGLang, llama.cpp, TRT-LLM, ...) |
research-only |
136 | paper results only; buys nothing without custom kernels or code |
abandoned |
5 | superseded or unmaintained |
unknown |
1 | not established from sources |
Where to start. For serving, the canonical spine is Orca's iteration-level scheduling (Orca: A Distributed Serving System for Transformer-Based Generative Models), PagedAttention (Efficient Memory Management for Large Language Model Serving with PagedAttention) and Sarathi-Serve's chunked prefill (Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve). For memory, MQA/GQA/MLA (Fast Transformer Decoding: One Write-Head is All You Need, GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints, DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model). For quantization, LLM.int8(), GPTQ, AWQ and the MX formats (LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale, GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers, AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration, Microscaling Data Formats for Deep Learning). For decoding, the two speculative-decoding originals (Fast Inference from Transformers via Speculative Decoding, Accelerating Large Language Model Decoding with Speculative Sampling).
What the map shows that no single record does. Pruning (7) and long-context
(7) have zero papers past research-only; scheduling has one
(Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve). Treat a speedup claim from those categories as a paper
result until an engine record says otherwise.
Topics
quantization (37)
4 in-production, 4 in-upstream-engine, 29 research-only. Synthesis: 23-quantization-selection.md, 04-quant-selection.md.
serving-systems (31)
7 in-production, 2 in-upstream-engine, 22 research-only. Synthesis: 12-serving-workflows.md, 21-deployment-topology.md.
attention (29)
4 in-production, 11 in-upstream-engine, 10 research-only, 4 abandoned. Synthesis: 02-flop-map.md, 13-model-compatibility.md.
kv-cache (19)
5 in-upstream-engine, 14 research-only. Synthesis: 09-model-memory-envelope.md.
speech (18)
8 in-production, 6 in-upstream-engine, 4 research-only.
multimodal (18)
13 in-production, 3 in-upstream-engine, 2 research-only.
speculative-decoding (18)
3 in-production, 7 in-upstream-engine, 8 research-only. Synthesis: 12-serving-workflows.md.
moe (18)
4 in-production, 5 in-upstream-engine, 8 research-only. Synthesis: 13-model-compatibility.md, 17-deployment-shape.md.
other (12)
6 in-production, 3 in-upstream-engine, 3 research-only.
embedding (11)
10 in-production, 1 in-upstream-engine.
| paper | year | adoption |
|---|---|---|
| Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks | 2019 | in-production |
| SimCSE: Simple Contrastive Learning of Sentence Embeddings | 2021 | in-production |
| Text Embeddings by Weakly-Supervised Contrastive Pre-training (E5) | 2022 | in-production |
| Matryoshka Representation Learning | 2022 | in-production |
| Towards General Text Embeddings with Multi-stage Contrastive Learning (GTE) | 2023 | in-production |
| MTEB: Massive Text Embedding Benchmark | 2023 | in-production |
| Nomic Embed: Training a Reproducible Long Context Text Embedder | 2024 | in-production |
| M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings | 2024 | in-production |
| The Embedder's Dilemma: LLMs Are Better, but at What Cost? | 2026 | in-production |
| Giga-Embeddings: Mixture-of-Experts Encoders for High-Throughput Text Embeddings | 2026 | in-production |
| LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders | 2024 | in-upstream-engine |
ssm (11)
7 in-upstream-engine, 4 research-only. Synthesis: 13-model-compatibility.md.
| paper | year | adoption |
|---|---|---|
| RWKV: Reinventing RNNs for the Transformer Era | 2023 | in-upstream-engine |
| Jamba-1.5: Hybrid Transformer-Mamba Models at Scale | 2024 | in-upstream-engine |
| Jamba: A Hybrid Transformer-Mamba Language Model | 2024 | in-upstream-engine |
| Mamba: Linear-Time Sequence Modeling with Selective State Spaces | 2024 | in-upstream-engine |
| Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality | 2024 | in-upstream-engine |
| MambaVision: A Hybrid Mamba-Transformer Vision Backbone | 2025 | in-upstream-engine |
| Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models | 2025 | in-upstream-engine |
| Efficiently Modeling Long Sequences with Structured State Spaces | 2022 | research-only |
| Retentive Network: A Successor to Transformer for Large Language Models | 2023 | research-only |
| RecurrentGemma: Moving Past Transformers for Efficient Open Language Models | 2024 | research-only |
| Zamba: A Compact 7B SSM Hybrid Model | 2024 | research-only |
retrieval (8)
4 in-production, 2 in-upstream-engine, 2 research-only.
| paper | year | adoption |
|---|---|---|
| ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT | 2020 | in-production |
| Dense Passage Retrieval for Open-Domain Question Answering | 2020 | in-production |
| SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking | 2021 | in-production |
| ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction | 2022 | in-production |
| Unsupervised Dense Information Retrieval with Contrastive Learning (Contriever) | 2021 | in-upstream-engine |
| PLAID: An Efficient Engine for Late Interaction Retrieval | 2022 | in-upstream-engine |
| Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generation | 2025 | research-only |
| TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked Text | 2025 | research-only |
distillation (8)
4 in-production, 4 research-only.
| paper | year | adoption |
|---|---|---|
| Orca: Progressive Learning from Complex Explanation Traces of GPT-4 | 2023 | in-production |
| The Llama 3 Herd of Models | 2024 | in-production |
| Phi-4 Technical Report | 2024 | in-production |
| DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning | 2025 | in-production |
| Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes | 2023 | research-only |
| On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes | 2024 | research-only |
| MiniLLM: On-Policy Distillation of Large Language Models | 2024 | research-only |
| WizardLM: Empowering Large Pre-Trained Language Models to Follow Complex Instructions | 2024 | research-only |
long-context (7)
7 research-only. Synthesis: 09-model-memory-envelope.md.
| paper | year | adoption |
|---|---|---|
| Learning to Compress Prompts with Gist Tokens | 2023 | research-only |
| Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention | 2024 | research-only |
| MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention | 2024 | research-only |
| LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning | 2024 | research-only |
| Efficient Streaming Language Models with Attention Sinks | 2024 | research-only |
| DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads | 2025 | research-only |
| Titans: Learning to Memorize at Test Time | 2025 | research-only |
scheduling (7)
1 in-upstream-engine, 5 research-only, 1 unknown. Synthesis: 12-serving-workflows.md, 19-production-operations.md.
| paper | year | adoption |
|---|---|---|
| Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve | 2024 | in-upstream-engine |
| Fast Distributed Inference Serving for Large Language Models | 2023 | research-only |
| Llumnix: Dynamic Scheduling for Large Language Model Serving | 2024 | research-only |
| Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloads (TetriInfer) | 2024 | research-only |
| Preble: Efficient Distributed Prompt Scheduling for LLM Serving | 2025 | research-only |
| Power Aware Dynamic Reallocation For Inference (RAPID) | 2026 | research-only |
| PackServe: SLO-Aware Request Scheduling for Agentic LLM Serving at Scale | 2026 | unknown |
pruning (7)
7 research-only.
| paper | year | adoption |
|---|---|---|
| LLM-Pruner: On the Structural Pruning of Large Language Models | 2023 | research-only |
| SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot | 2023 | research-only |
| The LLM Surgeon | 2024 | research-only |
| Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning | 2024 | research-only |
| ShortGPT: Layers in Large Language Models are More Redundant Than You Expect | 2024 | research-only |
| SliceGPT: Compress Large Language Models by Deleting Rows and Columns | 2024 | research-only |
| A Simple and Effective Pruning Approach for Large Language Models | 2024 | research-only |
inference-time-compute (6)
4 in-production, 2 research-only.
| paper | year | adoption |
|---|---|---|
| Chain-of-Thought Prompting Elicits Reasoning in Large Language Models | 2022 | in-production |
| Self-Consistency Improves Chain of Thought Reasoning in Language Models | 2023 | in-production |
| Large Language Models Cannot Self-Correct Reasoning Yet | 2024 | in-production |
| Let's Verify Step by Step | 2024 | in-production |
| Mixture-of-Depths: Dynamically allocating compute in transformer-based language models | 2024 | research-only |
| s1: Simple test-time scaling | 2025 | research-only |
position-encoding (6)
6 in-upstream-engine. Synthesis: 13-model-compatibility.md.
| paper | year | adoption |
|---|---|---|
| A Length-Extrapolatable Transformer | 2022 | in-upstream-engine |
| Extending Context Window of Large Language Models via Positional Interpolation | 2023 | in-upstream-engine |
| LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens | 2024 | in-upstream-engine |
| Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution | 2024 | in-upstream-engine |
| YaRN: Efficient Context Window Extension of Large Language Models | 2024 | in-upstream-engine |
| LongRoPE2: Near-Lossless LLM Context Window Scaling | 2025 | in-upstream-engine |
routing (4)
1 in-production, 3 research-only.
| paper | year | adoption |
|---|---|---|
| RouteLLM: Learning to Route LLMs with Preference Data | 2025 | in-production |
| Mixture-of-Experts with Expert Choice Routing | 2022 | research-only |
| FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance | 2023 | research-only |
| Measured Joules, Learned Routes: Learning to Route for Energy-Efficient LLM Serving | 2026 | research-only |
structured-output (3)
3 in-upstream-engine. Synthesis: 03-engine-selection.md.
| paper | year | adoption |
|---|---|---|
| JSONSchemaBench: A Rigorous Benchmark of Structured Outputs for Language Models | 2025 | in-upstream-engine |
| XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models | 2025 | in-upstream-engine |
| XGrammar-2: Dynamic and Efficient Structured Generation Engine for Agentic LLMs | 2026 | in-upstream-engine |
security (3)
2 in-production, 1 research-only. Synthesis: 22-safety-and-security.md.
| paper | year | adoption |
|---|---|---|
| Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations | 2023 | in-production |
| Llama Guard 3-1B-INT4: Compact and Efficient Safeguard for Human-AI Conversations | 2024 | in-production |
| Introducing v0.5 of the AI Safety Benchmark from MLCommons | 2024 | research-only |
cluster (2)
1 in-upstream-engine, 1 abandoned. Synthesis: 21-deployment-topology.md.
| paper | year | adoption |
|---|---|---|
| AIBrix: Towards Scalable, Cost-Effective Large Language Model Inference Infrastructure | 2025 | in-upstream-engine |
| Salus: Fine-Grained GPU Sharing Primitives for Deep Learning Applications | 2019 | abandoned |
agentic (1)
1 in-production.
| paper | year | adoption |
|---|---|---|
| ReAct: Synergizing Reasoning and Acting in Language Models | 2023 | in-production |
sparsity (1)
1 in-production.
| paper | year | adoption |
|---|---|---|
| Accelerating Sparse Deep Neural Networks | 2021 | in-production |
Related documents
- README.md — the reading guide for all docs.
- 25-citation-integrity.md — why a paper record's number is checked against the paper before it is cited.