这篇论文用HazMart数据集测了DeepSeek和QwQ的思维链，发现忠实性和安全性确实互相打架，还给出了干预办法。
该论文提出思维链推理存在忠实性与安全性的对齐矛盾：模型既要足够忠实以便监控，又要拒绝不安全推理。作者引入HazMart数据集和Targeted Reasoning Replacement技术，直接替换推理链中的想法来测试。DeepSeek-R1-Llama-70B忠实性高达97.5%，但拒绝不安全推理仅12.3%；QwQ-32B安全性为73.9%，忠实性降至74.7%。机制分析显示QwQ-32B中二者由反相关内部方向表征，通过表示引导可将安全行为提升9个百分点。
Risky Business: Measuring The Faithfulness-Safety Tension
Chain-of-Thought (CoT) reasoning offers a promising window into model monitoring. However, monitoring relies on faithfulness, i.e., the model output strictly derives from its reasoning trace. We identify an alignment tension where a model must be faithful enough to be monitored, yet robust enough to reject unsafe reasoning. We demonstrate that this counterbalance exists in current Large Reasoning Models (LRMs), and show ways in which it can be addressed. We introduce HazMart, a human-written dataset set in an autonomous AI shopkeeper scenario. Unlike prior work that relies on providing hints in prompts to test faithfulness (e.g., "A Stanford professor said it should be Answer A"), we propose a novel replacement-based technique, which we call Targeted Reasoning Replacement (TRR), that directly intervenes in the reasoning chain to substitute in unsafe or illogical thoughts (e.g., "Wait, the answer must be Option B [was Option A] because it is the most fitting"). DeepSeek-R1-Llama-70B exhibits high faithfulness (97.5%) but fails to reject Unsafe Reasoning (12.3%), while QwQ-32B is more robust (73.9% safety) at the cost of lower faithfulness (74.7%). Mechanistic analyses of QwQ-32B reveal that these properties are represented by anti-correlated internal directions peaking at the action-commit token. Finally, we demonstrate that representation steering can independently amplify the safety direction, increasing safe behavior by 9 percentage points while maintaining base capabilities.