Draft-OPD paper claims 23% gain over EAGLE-3 drafters
TL;DR
- Draft-OPD reports over 5× lossless acceleration for thinking models across diverse tasks in speculative-decoding benchmarks.
- The method improves acceptance over EAGLE-3 by 23% and over DFlash by 13%, per the paper's own numbers.
- Authors argue SFT-trained drafters plateau because training uses target trajectories while inference judges the drafter's own proposals.
Draft-OPD, a training recipe for speculative-decoding draft models, reports over 5× lossless acceleration on thinking-model tasks and gains of 23% over EAGLE-3 and 13% over DFlash, from a group led by Haodi Lei.
The paper's diagnosis: 'SFT quickly plateaus: the draft model's acceptance length on test data stops improving.' The authors pin this on an 'offline-to-inference mismatch': the drafter is fine-tuned on target-generated trajectories but, at inference, evaluated on blocks proposed under its own policy. Their fix is target-assisted rollout paired with replay 'from the verification-exposed error positions,' which they say lets the drafter 'learn from target feedback on both accepted and rejected proposals.'
The abstract does not publish per-model breakdowns, name the target models used, or disclose training-compute cost. Two researchers on our Who's Who watchlist already have the link in their recent posts.
Shared on Bluesky by 2 AI experts
Originally reported by arxiv.org
Read the original article →Original headline: Draft-OPD: On-Policy Distillation for Speculative Draft Models