树状推测解码适配 DeepSeek-V4，吞吐最高提升 18.5%
Adapting Tree-Structured Speculative Decoding to DeepSeek-V4 for Efficient Inference
DeepSeek-V4 用上树状推测解码，吞吐最高快 18.5%，论文还讲了什么负载收益最大。
自回归解码中目标模型反复执行是 LLM 推理延迟的主要来源，这篇 arXiv 论文把树状推测解码适配到 DeepSeek-V4-Flash 推理管线。DeepSeek-V4 的 CSA/HCA 在线压缩注意力会让不同分支从共享前缀压缩出不同状态、破坏跨分支一致性，作者用分支感知因果验证、临时状态隔离和接受路径状态刷新解决该问题。在 GSM8K、MBPP、ShareGPT 三个数据集、预算 D=5 到 D=8、批大小 1 到 64 的设置下，树状推测的接受长度在所有配置中都高于线性推测，D=8 时约为 2.83–3.41 对 2.39–2.84。吞吐量在几乎所有配置下提升，最高约 18.5%，且相对增益随预算增大、在小到中批大小的低可预测负载上最明显，超过一定预算后吞吐进入平台期。
Adapting Tree-Structured Speculative Decoding to DeepSeek-V4 for Efficient Inference
Repeated execution of the target model during autoregressive decoding is a major source of LLM inference latency. Unlike linear speculation, which follows a single candidate chain, tree-structured speculation retains multiple branches from shared prefixes; under the same budget, this broader coverage can improve acceptance and efficiency. Adapting it to DeepSeek-V4 is nontrivial: its CSA/HCA online compressed attention concentrates the difficulty on the target-verify side, where branches diverging from a shared prefix compress into different states, breaking cross-branch state consistency. We integrate tree-structured speculative decoding into the DeepSeek-V4-Flash pipeline via branch-aware causal verification, temporary state isolation, and accepted-path state refresh, keeping verification and compressed-state updates consistent across branches. Across budgets D=5 to D=8, batch sizes 1 to 64, and three datasets (GSM8K, MBPP, ShareGPT), tree speculation achieves a higher accepted length than the matched linear configurations in all settings (e.g., at D=8 about 2.83--3.41 versus 2.39--2.84) and improves throughput in nearly all configurations---marginal only at the smallest budget---by up to about 18.5%. More importantly, the gains follow stable, transferable regularities: the relative gain grows with the budget and is most pronounced for less predictable workloads at small-to-medium batch sizes, while beyond a certain budget throughput plateaus and decouples from the still-rising accepted length. These results show that retaining multiple candidate paths under the same budget can effectively improve DeepSeek-V4 decoding efficiency, and offer experience for adapting speculative decoding to future models with compressed, sparse, or structured context representations.
