多类与多标签BERT在CVE-to-CWE映射中的错误结构对比

Multi-Class vs. Multi-Label BERT for CVE-to-CWE Mapping: How Taxonomy Structure Shapes the Errors

精选理由

这篇论文用三种BERT模型对比了多类和多标签的漏洞分类效果,发现错误更多来自CWE层次结构而非模型选择,对安全领域做文本分类的人很有参考价值。

AI 摘要

该论文将CVE到CWE映射视为文本分类任务,比较了多类(每CVE预测一个CWE)和多标签(允许多个CWE)两种公式。在三个编码器(BERT Base、SecureBERT、CySecBERT)和三个嵌套标签空间(83、47、25类)上评估,多类训练在所有设置中macro-F1更高,但差距从21个百分点缩小至2个百分点。后处理阈值优化在25类设置中弥补了差距。混淆分析显示,主要错误模式遵循CWE层次结构,且三个编码器间高度相关(Pearson r>0.92),表明错误结构主要由分类体系设计而非编码器选择驱动。层次放松评估将macro-F1从约81%提升至约90%,表明严格指标低估了分支级分类质量。CySecBERT在多数设置中取得最强结果。

原文 · arXiv cs.LG

Multi-Class vs. Multi-Label BERT for CVE-to-CWE Mapping: How Taxonomy Structure Shapes the Errors

Assigning Common Weakness Enumeration (CWE) categories to Common Vulnerabilities and Exposures (CVE) records remains an important but largely manual step in vulnerability analysis. We study this task as a text classification problem and compare two modelling choices: a \emph{multi-class} formulation that predicts a single CWE per CVE and a \emph{multi-label} formulation that allows multiple assignments. Three transformer encoders (BERT Base, SecureBERT, and CySecBERT) are evaluated on three nested label spaces (83, 47, and 25 classes). Multi-class training achieves higher macro-F1 across all settings, although the gap to multi-label narrows from 21 to 2 percentage points as the label space shrinks. Post-hoc threshold optimisation on the multi-label side closes this gap on the 25-class setting. Confusion analysis shows that the dominant misclassification patterns follow the CWE hierarchy and are shared across all three encoders (Pearson $r > 0.92$), which suggests that the error structure is driven more by taxonomy design than by encoder choice. A hierarchy-relaxed evaluation that forgives within-family confusions raises macro-F1 from ${\sim}$81\% to ${\sim}$90\%, indicating that strict metrics understate branch-level classifier quality. CySecBERT achieves the strongest results overall, with statistically significant gains concentrated in the multi-label setting.