arXiv 2605.10453: SlimSpec: Low-Rank Draft LM-Head for Accelerated Speculative Decoding

Fields

id

sources/w15-pap-slimspec

URL

https://arxiv.org/abs/2605.10453

Publisher

arXiv

Kind

paper

Published

2026-05-11

Accessed

2026-10-11

notes

Paper by Anton Plaksin, Sergei Krutikov, Sergei Skvortsov, Alexander Samarin (Nebius). Investigates draft LM-head projection bottleneck in auxiliary-head drafters (EAGLE-3). Evaluates low-rank LM head compression across batch 1 and batch 64 regimes on LLaMA 3.1 8B, GPT-OSS 20B, and Qwen3-30B-A3B.

Notes

Paper by Anton Plaksin, Sergei Krutikov, Sergei Skvortsov, Alexander Samarin (Nebius). Investigates draft LM-head projection bottleneck in auxiliary-head drafters (EAGLE-3). Evaluates low-rank LM head compression across batch 1 and batch 64 regimes on LLaMA 3.1 8B, GPT-OSS 20B, and Qwen3-30B-A3B.

Cross-references

This record references no other records.