EAGLE-2 acceptance length and speedup on Vicuna-13B (temperature=1)
4.26 tokens_per_cycle
Fields
id
benchmarks/sd-eagle2-acceptance-vicuna13b-t1Metric
quality
Value
4.26
Unit
tokens_per_cycle
Model
Vicuna-13B
Methodology
temperature=1, 6 tasks. Average acceptance length τ = 4.26 tokens per draft-verify cycle. Speedup 3.65x mean across tasks.
Measured by
self
Reproducible
yes
notes
At temperature=1 (stochastic sampling), acceptance rates are lower than at temperature=0 but EAGLE-2 still achieves 3.65x speedup. The gap between temperature=0 (4.04x) and temperature=1 (3.65x) shows that stochastic sampling reduces acceptance rate. Baseline is vanilla autoregressive decoding.
Notes
At temperature=1 (stochastic sampling), acceptance rates are lower than at temperature=0 but EAGLE-2 still achieves 3.65x speedup. The gap between temperature=0 (4.04x) and temperature=1 (3.65x) shows that stochastic sampling reduces acceptance rate. Baseline is vanilla autoregressive decoding.