扩散语言模型新测试时扩展方法：频率感知重掩码提升推理准确率
Test-Time Scaling for Diffusion Language Models with Frequency-Aware Remasking
扩散语言模型 decoded 时信噪声大，这篇给出了能直接用的解法。相比瞬时置信度或熵这类局部信号，它看 token 在去噪过程中跳变多频繁，专治难题。做 LLaDA 或 Dream 推理优化的开发者可以照着复现。
论文针对扩散语言模型（DLM）的测试时扩展问题提出两阶段策略：先用快速基础采样器解决简单问题，再对剩余难题施加频率感知重掩码。核心机制是追踪每个 token 在去噪步骤中预测变化的频率，优先提交频繁变化的 token，促使样本探索不同推理路径而非坍缩到模型偏好的轨迹上。
实验基于 LLaDA-8B-Instruct 和 Dream-7B-Instruct 两个模型：MATH-500 上绝对准确率分别提升最多 3.73 和 7.33 个百分点，HumanEval 上绝对覆盖率分别提升最多 11.39 和 10.98 个百分点。对于基础确定性感知策略在 100 次采样后仍未解出的题目，该方法能恢复其中 3.75% 到 18.75% 的解。
Test-Time Scaling for Diffusion Language Models with Frequency-Aware Remasking
Diffusion language models (DLMs) generate text through iterative denoising, where decisions about which tokens to commit or remask strongly affect output quality. Existing certainty-aware strategies rely on instantaneous confidence or entropy, but these local signals can be misleading on difficult reasoning problems. We formulate DLM test-time scaling as adaptive allocation of both inference compute and decoding strategies. Our two-stage approach uses a fast base sampler to solve easy questions, then applies frequency-aware remasking to the remaining hard questions. This strategy tracks how often each token's prediction changes across denoising steps and preferentially commits frequently changing tokens, encouraging samples to explore alternative reasoning paths rather than collapse onto the model's preferred trajectory. Experiments with LLaDA-8B-Instruct and Dream-7B-Instruct show consistent improvements over a strong adaptive allocation baseline, with absolute accuracy gains of up to 3.73 percentage points (LLaDA) and 7.33 percentage points (Dream) on MATH-500, and absolute coverage gains of up to 11.39 percentage points (LLaDA) and 10.98 percentage points (Dream) on HumanEval. On questions unsolved by the base certainty-aware strategy after 100 samples, frequency-aware remasking recovers solutions for 3.75% to 18.75% of examples.
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include 扩散语言模型, LLaDA, Dream, test-time scaling.
Source attribution: curated from arXiv cs.AI (@Bowen Zuo, Yue Yu, Dongruo Zhou, Yinglun Zhu).
