Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding

Fields

id

sources/sd-ouroboros-paper

URL

https://arxiv.org/abs/2402.13720

Publisher

arXiv (Zhao et al.; Tsinghua University)

Kind

paper

Published

2024-02

Accessed

2026-10-04

notes

Ouroboros generates draft phrases to parallelize drafting and lengthen drafts in a training-free manner. Speedups up to 2.8x over speculative decoding and 3.9x over vanilla decoding. No fine-tuning of draft or target models. Published at EMNLP 2024.

Notes

Ouroboros generates draft phrases to parallelize drafting and lengthen drafts in a training-free manner. Speedups up to 2.8x over speculative decoding and 3.9x over vanilla decoding. No fine-tuning of draft or target models. Published at EMNLP 2024.

Cross-references

This record references no other records.