Lookahead decoding acceptance length and speedup on Vicuna-13B (temperature=0)
1.62 tokens_per_cycle
Fields
id
benchmarks/sd-lookahead-acceptance-vicuna13b-t0Metric
quality
Value
1.62
Unit
tokens_per_cycle
Model
Vicuna-13B
Methodology
temperature=0, 6 tasks. Average acceptance length τ = 1.62 tokens per draft-verify cycle. Speedup 1.58x mean across tasks.
Measured by
self
Reproducible
yes
notes
Lookahead decoding uses Jacobi iteration for parallel decoding. No auxiliary models or data stores. Exact (distribution-preserving). Speedup ≈ acceptance length because draft is nearly free. Baseline is vanilla autoregressive decoding.
Notes
Lookahead decoding uses Jacobi iteration for parallel decoding. No auxiliary models or data stores. Exact (distribution-preserving). Speedup ≈ acceptance length because draft is nearly free. Baseline is vanilla autoregressive decoding.