论文精选

ALIGNBEAM:跨词汇表推理时安全对齐转移方法

ALIGNBEAM : Inference-Time Alignment Transfer via Cross-Vocabulary Logit Mixing

精选理由

做模型安全对齐的团队终于有了跨家族迁移方案——ALIGNBEAM 无需训练即可在推理时转移安全能力,适合需要部署不同系列模型但担心安全退化的开发者直接尝试。

AI 摘要

ALIGNBEAM 是一种无需训练的推理时安全对齐方法,解决了领域微调导致大语言模型安全性下降的问题。现有方法要求安全锚点模型和目标模型共享词汇表,但 ALIGNBEAM 通过逐 token 翻译锚点 logits 到目标模型词汇表,并利用小型 LLM 法官选择最安全的候选续写,突破了这一限制。该方法不改变任何模型权重,可在部署时调整安全-效用权衡。在跨词汇表和同词汇表评估中,ALIGNBEAM 显著提升了对抗性基准的拒绝率,同时保持任务准确性和推理开销在实用范围内。结果表明,安全对齐可以在推理时在不同模型家族之间转移,无需修改任何模型权重。

原文 · arXiv cs.LG

ALIGNBEAM : Inference-Time Alignment Transfer via Cross-Vocabulary Logit Mixing

Domain fine-tuning degrades the safety of large language models: fine-tuned specialists readily comply with harmful prompts framed in domain language. Existing inference-time defenses that mix logits from a safe anchor model require both models to share a vocabulary, which rules them out for the cross-family specialists where safety is most degraded. We present ALIGNBEAM, a training-free method that lifts this restriction by translating anchor logits into the target model's vocabulary token-by-token at each decoding step; a small LLM judge then selects the safest among K candidate continuations. No weights are changed, and the safety-utility trade-off can be tuned at deployment without retraining. Across both cross-vocabulary and same-vocabulary evaluation pairs, ALIGNBEAM substantially raises refusal on adversarial benchmarks while keeping task accuracy and inference overhead within practical bounds. The results show that safety alignment can be transferred between model families at inference time, without touching either model's weights.