EAGLE-4 Hits 863.8 Tokens/s on 27B Models
Speculative decoding delivers speedups without changing model outputs, so teams can adopt it without revalidating answer quality. Inference engineers running 27B-class models on limited hardware can treat EAGLE-4 as a drop-on layer rather than a model swap, and its measured edge over DFlash 2 and DSpark gives a concrete baseline for benchmarking their own stacks. The gains come purely from the drafting side, a direction worth watching in the decoding literature.
EAGLE-4 is a speculative decoding add-on that speeds up inference for models already in production. It reaches 863.8 tokens/s on a 27B model on a single request, 1.28X faster than DFlash 2 and 1.40X faster than DSpark. The method keeps the standard draft-and-verify loop, with a small draft model proposing several tokens and the target model checking them in one forward pass. Improvements focus on the drafting side through an adaptive draft structure and a cheaper draft LM head. Output tokens remain identical to standard decoding while running about 3X faster on the tested setup.
Some of the most useful inference work right now is happening in the speculative decoding component.
It has become one of the most dependable ways to get more tokens per second out of a model you’ve already chosen.
EAGLE-4 decodes 863.8 tokens/s on a 27B model on a single request, 1.28X DFlash 2 and 1.40X DSpark.
EAGLE-4 is a speed-up add-on for running AI chatbots: a tiny helper model guesses the next few words, and the big model checks all those guesses at once instead of writing one word at a time. The answers come out identical, just about 3× faster on the tested setup.
EAGLE-4 keeps the draft-and-verify loop and puts its work into the drafting side, with an adaptive draft structure and a cheaper draft LM head.
The loop:
- A small draft model proposes several tokens ahead - The target model checks all of them in one forward pass - Matching tokens are kept, and the first mismatch is replaced with the target’s own token - Repeat from there
