Medusa acceptance length and speedup on Vicuna-13B (temperature=0)
2.44 tokens_per_cycle
Fields
id
benchmarks/sd-medusa-acceptance-vicuna13b-t0Metric
quality
Value
2.44
Unit
tokens_per_cycle
Model
Vicuna-13B
Methodology
temperature=0, 6 tasks. Average acceptance length τ = 2.44 tokens per draft-verify cycle. Speedup 2.07x mean across tasks.
Measured by
self
Reproducible
yes
notes
Medusa-1 (frozen backbone). Uses tree-based attention for parallel verification. Typical acceptance scheme is lossy (does not guarantee distribution preservation). Baseline is vanilla autoregressive decoding.
Notes
Medusa-1 (frozen backbone). Uses tree-based attention for parallel verification. Typical acceptance scheme is lossy (does not guarantee distribution preservation). Baseline is vanilla autoregressive decoding.