推理增强语言模型用于网络安全检测分类

Cybersecurity Detection Classification with Reasoning-enabled Language Models

精选理由

这篇论文把推理模型用在安全告警分类上,82.6%准确率,比直接LLM打标签的恶意/良性召回提升明显,还解决了置信度问题。

AI 摘要

安全运营中心(SOC)面临告警疲劳,检测量超过人工处理能力。研究团队通过自动提示优化、自训练和可验证奖励强化学习,在真实人工标注的Windows端点检测上训练出链式推理(CoT)分类器。CoT会降低标签token概率,因此另训练校准器读取完整推理轨迹并估计判定正确概率。系统达到82.6%测试准确率,在高置信度操作点下,良性召回率比直接标注的LLM分类器提升43.0%,恶意召回率提升18.3%。未训练的置信度评判会使高置信召回归零,而微调的30B模型显著优于前沿通用模型。

原文 · arXiv cs.LG

Cybersecurity Detection Classification with Reasoning-enabled Language Models

A major issue in Security Operations Centers (SOCs) is alert fatigue, as the number of detections reported is more than staff can triage in a given day. Prior work prompts or fine-tunes large language models (LLMs) to emit a triage label directly, but does not train them to reason about whether a detection is a genuine threat. We train a chain-of-thought (CoT) reasoning-enabled triage classifier on real, human-labeled Windows endpoint detections by combining automated prompt optimization, self-training, and reinforcement learning with verifiable rewards. We find that CoT reasoning also degrades the label-token probabilities that automated triage relies on, so we separately train a calibrator that reads the full reasoning trace and estimates the probability that the verdict is correct. Our system reaches 82.6% test accuracy and, at the high-confidence operating point that governs automated triage, improves benign recall by 43.0% and malicious recall by 18.3% over a direct-label LLM classifier. We further show that the trained calibrator is necessary - an untrained confidence judge collapses high-confidence recall to zero - and that a finetuned 30B model significantly outperforms frontier general-purpose models, motivating targeted training over scale.

推理增强语言模型用于网络安全检测分类 · AI 热点