Hydra: Sequentially-Dependent Draft Heads for Medusa Decoding

Fields

id

sources/sd-hydra-paper

URL

https://arxiv.org/abs/2402.05109

Publisher

arXiv (Ankner, Parthasarathy, Nrusimha, Rinard, Ragan-Kelley, Brandon)

Kind

paper

Published

2024-02

Accessed

2026-10-04

notes

Hydra heads are sequentially-dependent drop-in replacement for Medusa's independent draft heads. Hydra++ improves throughput by up to 1.31x over Medusa and 2.70x over autoregressive decoding. Key insight: standard Medusa heads speculate tokens independently of preceding tokens in the candidate continuation; Hydra conditions each head on the draft sequence.

Notes

Hydra heads are sequentially-dependent drop-in replacement for Medusa's independent draft heads. Hydra++ improves throughput by up to 1.31x over Medusa and 2.70x over autoregressive decoding. Key insight: standard Medusa heads speculate tokens independently of preceding tokens in the candidate continuation; Hydra conditions each head on the draft sequence.

Cross-references

This record references no other records.