评估开放权重LLM生成自动驾驶漏洞结构化威胁信息

Evaluating Open-Weight LLMs for Generating Structured Threat Information for Autonomous Vehicle Vulnerabilities

精选理由

这篇论文测了11个开源模型做漏洞结构化的效果,Gemma-4-31B和Codestral-22B组队挺有意思,做安全情报的可以看看F1分数。

AI 摘要

该论文构建了CAV-STIXGen数据集,将自动驾驶车辆漏洞描述映射到STIX域对象、关系对象、CWE及MITRE ATT&CK技术。评估了11个开放权重LLM(4B至120B参数),单模型在SDO、SRO、CWE映射上F1分别达0.94、0.63、0.99,而完整ATT&CK映射仍有挑战。多智能体方案中Gemma-4-31B与Codestral-22B在SDO和SRO上分别获0.91和0.43的F1。分析了CWE与ATT&CK共现模式,展示了AI辅助漏洞转STIX的自动化威胁情报能力。

原文 · arXiv cs.AI

Evaluating Open-Weight LLMs for Generating Structured Threat Information for Autonomous Vehicle Vulnerabilities

Connected and Autonomous Vehicles (CAVs) rely on interconnected software and hardware components, including sensors, Electronic Control Units, in-vehicle infotainment systems, and telematics units, where vulnerabilities can compromise assets, users, and vehicle operations. These vulnerabilities are commonly documented as plain text in the Common Vulnerabilities and Exposures (CVE) database; however, security practitioners require structured information about affected assets, types of weaknesses, and attack behaviors to effectively mitigate the risks from these vulnerabilities. To this end, we evaluate open-weight Large Language Models (LLMs) for generating Structured Threat Information Expression (STIX), a well-known structured format for representing threat information, for CAV-related CVEs. We construct a dataset called CAV-STIXGen that maps CAV vulnerability descriptions to STIX domain objects (SDO), STIX relationship objects (SRO), Common Weakness Enumeration (CWE), and MITRE ATT&CK techniques mappings. Using this dataset, we evaluated 11 open-weight LLMs (4B to 120B parameters) across various prompting strategies and temperatures. Single-model configurations achieve F1 scores of 0.94 for SDO, 0.63 for SRO, and 0.99 for CWE mapping, while complete MITRE ATT&CK mapping remains challenging. In a multi-agent setup, Gemma-4-31B and Codestral-22B achieve F1 scores of 0.91 for SDOs and 0.43 for SROs, respectively. Lastly, we analyze CWE and MITRE ATT&CK co-occurrences to identify recurring threat patterns in the CAV domain, demonstrating how AI-assisted vulnerability-to-STIX translation can automate threat intelligence and prioritize defense in transportation security.