多轮对话中的危害放大是LLM安全部署的盲区，做AI安全或内容审核的团队可以关注HarmAmp基准和TrajSafe方案，直接用于评估和加固自己的模型。
大型语言模型在作为助手的同时，也可能被恶意用户利用，通过多轮交互放大危害，包括让新手生成专业有害内容，以及规模化执行有害操作。现有研究常忽略多轮对话中的危害累积。为此，研究者提出了HarmAmp基准，涵盖12类风险场景，并设计了TrajSafe主动监控器，通过探测用户真实意图和引导模型安全完成来干预有害轨迹。实验表明，TrajSafe能显著降低多轮交互中的危害，同时保持低过度拒绝率和模型通用能力。这项工作为缓解LLM交互中的细微安全风险提供了新范式。
Investigating and Alleviating Harm Amplification in LLM Interactions
Large language models (LLMs) can serve as helpful assistants, yet they can equally function as harm amplifiers that enable malicious users to achieve harmful outcomes beyond their capabilities through extended interactions. This risk manifests along two axes, i.e., democratizing domain expertise that allows novices to produce specialized harmful content, and scaling harmful operations at volumes that manual effort cannot match. Existing works, however, often overlook how LLMs compound harm across multi-turn conversations. We introduce HarmAmp, a new benchmark for multi-turn harm amplification scenarios spanning twelve risk categories. Each scenario is grounded in real-world threats and satisfies rigorous criteria, i.e., substantive amplification, operational specificity, and multi-turn necessity. We further propose TrajSafe, a proactive monitor that anticipates harmful trajectories and intervenes through actions such as probing users' genuine intents and steering the models towards safer completion. Our extensive experiments demonstrate that TrajSafe significantly reduces the harmfulness incurred in multi-turn interactions while preserving a low over-refusal rate and the target model's general capabilities. Our work offers a promising paradigm to alleviate the nuanced safety risks in LLM interactions.