EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty

Fields

id

sources/sd-eagle1-paper

URL

https://arxiv.org/abs/2401.15077

Publisher

arXiv (Li, Wei, Zhang, Zhang; Peking University / Microsoft Research / University of Waterloo)

Kind

paper

Published

2024-01

Accessed

2026-10-04

notes

EAGLE performs autoregression at feature (second-to-top-layer) level, reusing top-layer features from target model. For LLaMA2-Chat 70B: 2.7x-3.5x latency speedup, doubled throughput. Uses tree-structured draft allowing alternative branches on rejection (vs chain-structured standard speculative sampling). Ranked first on Spec-Bench at time of submission.

Notes

EAGLE performs autoregression at feature (second-to-top-layer) level, reusing top-layer features from target model. For LLaMA2-Chat 70B: 2.7x-3.5x latency speedup, doubled throughput. Uses tree-structured draft allowing alternative branches on rejection (vs chain-structured standard speculative sampling). Ranked first on Spec-Bench at time of submission.

Cross-references

This record references no other records.