EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test

EAGLE-3 abandons feature-level prediction for direct token prediction with multi-layer feature fusion via 'training-time test'. Speedup up to 6.5x, ~1.4x over EAGLE-2. In SGLang framework: 1.38x throughput improvement at batch size 64. Draft tree depth increased from 6 to 8. Key insight: EAGLE's feature prediction constraint limits benefit from scaling training data; removing it enables scaling curve never observed in prior works.