这篇论文提出了嵌套蒙特卡洛方法在离散扩散语言模型推理时间控制中的应用，对于想要了解如何优化文本生成模型的人来说是个好资源。
本研究针对离散扩散语言模型的文本生成推理时间控制，旨在引导采样向序列级奖励发展，无需重新训练。通过嵌套蒙特卡洛方法解决现有粒子方法如best-of-n采样和bootstrap SMC的过度优化和权重退化问题。NSMC和FA-NSMC在毒性和流畅性引导任务上优于best-of-n和bootstrap SMC。
Discrete Diffusion Inference-Time Control with Nested Sequential Monte Carlo
We study inference-time control for text generation in discrete diffusion language models, where the goal is to steer sampling toward sequence-level rewards without retraining. Prior work in this domain has focused on particle-based methods such as best-of-$n$ sampling and bootstrap sequential Monte Carlo, which may suffer from overoptimism and weight degeneracy, respectively. We address these limitations using \emph{nested} sequential Monte Carlo methods. We formulate nested SMC (NSMC) and fully-adapted nested SMC (FA-NSMC) for Feynman--Kac steering, identifying and correcting errors in prior formulations that lead to biased final estimates. We evaluate these methods on toxicity and fluency steering tasks, showing that NSMC and FA-NSMC consistently outperform best-of-$n$ and bootstrap SMC.