Hydra acceptance length and speedup on Vicuna-13B (temperature=0)
3.4 tokens_per_cycle
Fields
id
benchmarks/sd-hydra-acceptance-vicuna13b-t0Metric
quality
Value
3.4
Unit
tokens_per_cycle
Model
Vicuna-13B
Methodology
temperature=0, 6 tasks. Average acceptance length τ = 3.40 tokens per draft-verify cycle. Speedup 2.69x mean across tasks.
Measured by
self
Reproducible
yes
notes
Hydra++ with sequentially-dependent draft heads. Improves over Medusa by up to 1.31x. Baseline is vanilla autoregressive decoding.
Notes
Hydra++ with sequentially-dependent draft heads. Improves over Medusa by up to 1.31x. Baseline is vanilla autoregressive decoding.