arXiv:2406.02532 - SpecExec: Massively Parallel Speculative Decoding for Interactive LLM Inference on Consumer Devices

arXiv:2406.02532, v1 2024-06-04, latest v3 2024-11-30. arXiv comment is EMPTY of any venue claim; venue established from the NeurIPS 2024 index instead. Authors Ruslan Svirschevski, Avner May, Zhuoming Chen, Beidi Chen, Zhihao Jia, Max Ryabinin. Note the proceedings title capitalises 'For' ('Massively Parallel Speculative Decoding For Interactive LLM Inference on Consumer Devices') where arXiv lowercases it - a cosmetic difference, same paper.