CAV：用于网络威胁调查的冲突感知证据验证方法
When AI Investigators Cannot Trust Their Tools: Conflict-Aware Evidence Verification for Cyber Threat Investigation
搞安全运营的可以看看：多个检测工具互相打架时，智能体会被带偏到准确率 0.13，CAV 用固定验证规则拉回 0.88。
论文研究 LLM 智能体在安全告警分诊中依赖多个检测工具时的信任问题。在修正版 CIC-IDS2017 数据集上，四个检测器中有 1 个被污染就会把朴素方法的准确率压到 0.50-0.72，3 个被污染时所有未验证的调查智能体准确率不超过 0.13。作者提出 Conflict-Aware Evidence Verification（CAV），结合等权重投票、证据冲突门控和独立最近邻信任检查。在 3B 到 14B 参数的开源模型上，CAV-LLM 在 3 个检测器被污染时达到 0.84-0.88，且仅在 0.4% 的正常流量上调用 LLM。
When AI Investigators Cannot Trust Their Tools: Conflict-Aware Evidence Verification for Cyber Threat Investigation
LLM agents increasingly triage security alerts by combining the outputs of several detection tools, implicitly treating those tools as trustworthy. Unlike prior work on conflicts between a model's knowledge and its context, we study conflicts among the tools themselves: what happens when this machine-generated evidence is conflicting or wrong. On the corrected CIC-IDS2017 dataset, four heterogeneous detectors emit labels and confidences that we corrupt under a threat model varying the number of corrupted tools, the persistence of the corruption, and the confidence attached to wrong outputs. Across open-weight LLMs from 3B to 14B parameters, unverified investigators are fragile: one corrupted detector out of four lowers naive accuracy to .50-.72, three lower every unverified investigator to .13 or less, and honest disagreement alone makes them abstain on up to 63% of flows. We propose Conflict-Aware Evidence Verification (CAV), built from established components: an equal-weight vote that ignores reported confidences, an evidence-conflict gate, and an independent nearest-neighbour trust check that tests the leading candidates. With its prompt frozen before testing, CAV-LLM improves on the naive investigator in every corrupted condition ($p < 0.001$), reaching .84-.88 with three corrupted detectors against at most .08, while calling the LLM on 0.4% of clean traffic. A fixed verification rule is as accurate or better unless most tools are corrupted. When we make the independent channel itself noisy, CAV inherits its errors: it stays ahead when two or more tools are corrupted, but adding the channel as an extra voter is better when at most one is. Verification thus moves trust from the tools to the check, which must be measured and protected in turn.
