DT-Guard:面向无推理LLM安全护栏的意图驱动推理主动训练

DT-Guard: Intent-Driven Reasoning-Active Training for Reasoning-Free LLM Safety Guardrail

精选理由

想用轻量模型做好内容安全?DT-Guard靠训练时推理、运行时只打标签,4B就干掉8B的护栏,速度快还准。

AI 摘要

DT-Guard提出一种推理主动训练、推理自由推理的范式,在训练时使用推理监督,推理时只输出结构化安全标签。它采用意图-类别-安全的三步决策过程,并构建带意图标签、风险类别、安全标签和推理轨迹的数据集。通过Rollout-Guided Progressive Hard-Case Optimization (RG-PHO)提升硬样本鲁棒性。在提示侧和响应侧安全基准上,DT-Guard分别达到0.886和0.870的平均F1分数,4B参数版本取得0.878的双侧平均F1,超越多个8B基线。

原文 · arXiv cs.AI

DT-Guard: Intent-Driven Reasoning-Active Training for Reasoning-Free LLM Safety Guardrail

Large language models deployed in open-world applications require safety guardrails that are both robust to complex risks and efficient enough for low-latency runtime moderation. Existing guardrails face a practical trade-off between lightweight classification-based models, which are efficient but often struggle with concealed intent, ambiguous semantics, and borderline safety decisions, and reasoning-based guards, which improve judgment quality but introduce additional token generation and inference latency. We present DT-Guard, a content safety guardrail model based on a Reasoning-Active Training, Reasoning-Free Inference paradigm. The key idea is to use reasoning supervision during training while emitting only structured safety labels at inference time. DT-Guard formulates safety judgment as a progressive decision process, Intent - Category - Safety, and constructs an intent-driven dataset with intent labels, risk categories, safety labels, and structured reasoning trajectories. To further improve hard-case robustness, we propose Rollout-Guided Progressive Hard-Case Optimization (RG-PHO), which uses multi-rollout consistency to identify stably mastered, persistently failed, and preference-unstable samples, and applies targeted supervised and preference optimization accordingly. At inference time, DT-Guard directly generates structured labels without explicit reasoning traces, preserving deployment efficiency. Experiments on prompt-side and response-side safety benchmarks show that DT-Guard achieves average F1 scores of 0.886 and 0.870, respectively. With only a 4B backbone, it reaches a dual-side average F1 of 0.878, outperforming strong 8B guardrail baselines. These results demonstrate that reasoning supervision can be effectively internalized into low-latency safety discrimination.