VICBench 搞了个 100 个真实漏洞提交的基准，覆盖 Python/Java/C++，平均改动比旧数据集大不少，测漏洞检测用它更靠谱。
VICBench 面向代码漏洞检测，包含 100 个经人工专家与智能体工作流双重验证的漏洞引入提交，对应 100 个 CVE，覆盖 88 个项目的 Python、Java、C++ 代码。该基准涵盖 48 种 CWE 类型，修复补丁平均 38.6 行，漏洞引入提交平均 252.5 行，复杂度高于此前数据集。评测中，现有最先进的 V-SZZ 和 LLM4SZZ 算法 F1 值仅为 33.3%-40.1%，表明自动化检测仍需大量人工介入。VICBench 能更可靠地评估漏洞检测方法的真实表现。
VICBench: A Multi-Language Benchmark for Code Vulnerability Detection
Evaluating security vulnerability detection tools requires benchmark datasets with vulnerability-inducing commits (VICs) - the commits that first introduce vulnerabilities into codebases. VICs are essential for determining the full range of vulnerable software versions. Existing vulnerability datasets suffer from limited programming language coverage, restricted patch complexity, and narrow project scope. Through our dual annotation by human experts and an agentic workflow, we create a benchmark - VICBench - of 100 verified VICs for 100 CVEs across 88 projects in Python, Java, and C++, covering 48 CWE types. VICBench features complex real-world vulnerability fixes averaging 38.6 lines and corresponding VICs of 252.5 lines - significantly larger than prior work. Our evaluation shows that state-of-the-art algorithms V-SZZ and LLM4SZZ achieve only 33.3%-40.1% F1, confirming that using existing approaches still entails significant manual effort. VICBench enables robust evaluation of vulnerability detection approaches.