EAGLE-2 acceptance length and speedup on Vicuna-13B (temperature=0)
4.65 tokens_per_cycle
Fields
id
benchmarks/sd-eagle2-acceptance-vicuna13b-t0Metric
quality
Value
4.65
Unit
tokens_per_cycle
Model
Vicuna-13B
Methodology
temperature=0, 6 tasks (MT-bench, HumanEval, GSM8K, Alpaca, CNN/DM, Natural Questions). Average acceptance length τ = 4.65 tokens per draft-verify cycle. Speedup 4.04x mean across tasks.
Measured by
self
Reproducible
yes
notes
EAGLE-2 with dynamic draft tree, depth 6, 10 nodes expanded per layer, 50 total draft tokens. Baseline is vanilla autoregressive decoding (1.00x). Per-task speedups: MT-bench 4.26x, HumanEval 4.96x, GSM8K 4.22x, Alpaca 4.25x, CNN/DM 3.40x, Natural Questions 3.13x.
Notes
EAGLE-2 with dynamic draft tree, depth 6, 10 nodes expanded per layer, 50 total draft tokens. Baseline is vanilla autoregressive decoding (1.00x). Per-task speedups: MT-bench 4.26x, HumanEval 4.96x, GSM8K 4.22x, Alpaca 4.25x, CNN/DM 3.40x, Natural Questions 3.13x.