扩散语言模型采样器会放大性别偏见，Fork-First Sampling 提出修复
Who Gets Unmasked? Sampler-Induced Bias in Diffusion Language Models
LLaDA 这类扩散语言模型自带的采样规则会让文本更偏向男性职业，这篇论文讲清了原因，还给了个一次前向传播就能修的采样方法。
扩散语言模型（dLLM）通过逐步去掩码生成文本，采样器除了决定写什么词，还决定何时提交，后一个决策会引入模型本身的偏见之外的新偏差。研究发现包括 LLaDA 默认规则在内的常见提交策略，倾向于优先提交概率最高的位置，这种“偷看”会偏向多数选项。在 LLaDA 和 Dream 上的实验显示，偷看式采样器放大了职业性别刻板印象，并使观点分布趋于单一，置信度排序会把性别均衡的职业更多分配给男性，并行解码越快偏移越明显。论文提出 Fork-First Sampling，在写入任何内容前先决定分歧点，只需额外一次前向传播即可贴近目标分布、提升流畅度，并避免基于提示词的修复方法会生成“特意提及主角性别”的文本。
Who Gets Unmasked? Sampler-Induced Bias in Diffusion Language Models
Diffusion language models (dLLMs) generate by gradually unmasking a canvas, so their samplers decide not only which token to write but also when to commit it. We show that this second decision changes who appears in generated text, a sampler-induced bias on top of the model's own. Common commit rules, including LLaDA's default, commit the positions whose sampled tokens are most probable; this peeking favors majority options. To separate the sampler from the model, we read the model's belief about a fork, such as a protagonist's gender, in one forward pass, and show that this belief matches exact sampling from the model. Measured against it, peeking samplers on LLaDA and Dream amplify occupational gender stereotypes and collapse opinion distributions, and confidence-based ordering shifts gender-balanced occupations toward men, more so under faster parallel decoding. We propose Fork-First Sampling, which decides the fork before anything else is written: it follows the belief or any target distribution, improves fluency, and avoids the stories that remark on the protagonist's gender, which prompt-based fixes produce, at the cost of one forward pass. Fairness evaluations of dLLMs should report the commit rule.
