Speculative Decoding: Fast Inference with Speculative Sampling

Fields

id

sources/src5-speculative-decoding-paper

URL

https://arxiv.org/abs/2302.01318

Publisher

Google

Kind

paper

Published

2023-02-01

Accessed

2026-10-04

notes

Speculative decoding uses a smaller draft model to generate candidate tokens that are verified in parallel by the target model. Key for LLM inference speedup. Used by vLLM, TensorRT-LLM, SGLang. Enables 2-3x speedup.

Notes

Speculative decoding uses a smaller draft model to generate candidate tokens that are verified in parallel by the target model. Key for LLM inference speedup. Used by vLLM, TensorRT-LLM, SGLang. Enables 2-3x speedup.

Cross-references

This record references no other records.