Recurrent Drafter for Fast Speculative Decoding in Large Language Models

ReDrafter uses RNN as draft model conditioning on LLM hidden states, with dynamic tree attention over beam search results. Up to 2.8x speedup on Vicuna MT-Bench with PyTorch on H100 GPUs. Also validated on Apple Silicon (MLX/Metal) with up to 2.3x speedup. Trained via knowledge distillation from the LLM.