这篇论文搞了个叫ASD的投机解码改进，不用重新训练就能提速，实测吞吐最高涨15%，代码也开源了，跑推理的可以看看。
Approximate Speculative Decoding（ASD）是一种免训练的投机解码验证器，用预算化的最长前缀选择替代逐位截断，在保留目标贪心后缀的同时接受部分不匹配token。在Qwen3-14B+DSpark-14B的七项任务上，ASD相比严格验证平均提升7.78%吞吐，区间为3.05%到15.26%。在FP4到FP8兼容设置下，DeepSeek-V4-Flash（284B）搭配DSpark时，GSM8K和MATH-500上的验证方接受率提高约10%到16%。该方法无需新草稿模型或微调，预算为零时精确退化为标准贪心验证，代码已开源。
Approximate Speculative Decoding
Speculative decoding accelerates autoregressive generation by verifying a draft block with a target model in parallel. Under standard greedy verification, decoding stops at the first draft token that differs from the target argmax, discarding the remaining target-scored suffix. Although accepting such a mismatch changes the decoding trajectory, it can make a contiguous suffix reusable when its tokens remain target-greedy under the realized prefix. In this paper, we introduce \textbf{Approximate Speculative Decoding (ASD)}, a training-free verifier that replaces binary first-mismatch truncation with budgeted longest-prefix selection. ASD accepts selected mismatches subject to a local target-logit regret gate, a per-block exception cap, and a persistent request-level regret budget, then reuses the contiguous target-greedy suffix without additional approximate decisions or target-model forward passes. ASD requires neither a new draft model nor fine-tuning, and exactly reduces to standard greedy verification when the budget is zero. Experiments show that ASD improves fixed-workload throughput by $3.05\%$--$15.26\%$ over matched strict verification and averages a $7.78\%$ gain across seven Qwen3-14B + DSpark-14B tasks. On DeepSeek-V4-Flash (284B) with DSpark it also raises verifier-side acceptance by roughly $10\%$--$16\%$ on GSM8K and MATH-500 in an FP4-to-FP8 compatibility setting. The source code is publicly available at: https://github.com/Kissmetothemoon/ASD