PLD (prompt lookup) acceptance length and speedup on Vicuna-13B (temperature=0)
1.69 tokens_per_cycle
Fields
id
benchmarks/sd-pld-acceptance-vicuna13b-t0Metric
quality
Value
1.69
Unit
tokens_per_cycle
Model
Vicuna-13B
Methodology
temperature=0, 6 tasks. Average acceptance length τ = 1.69 tokens per draft-verify cycle. Speedup 1.64x mean across tasks.
Measured by
self
Reproducible
yes
notes
PLD (prompt lookup decoding) uses n-gram retrieval from the prompt as draft. Zero draft model cost. Speedup ≈ acceptance length because draft is nearly free. Highest speedup on summarization (CNN/DM 2.42x) due to high context overlap. Baseline is vanilla autoregressive decoding.
Notes
PLD (prompt lookup decoding) uses n-gram retrieval from the prompt as draft. Zero draft model cost. Speedup ≈ acceptance length because draft is nearly free. Highest speedup on summarization (CNN/DM 2.42x) due to high context overlap. Baseline is vanilla autoregressive decoding.