katIA 结合静态分析与 LLM 提升代码变更影响分析
Hybrid Source Code Change Impact Analysis
做代码工具或研究 CIA 的开发者可以看看这篇。它把静态分析和 Gemini 2.5 Flash 结合，还公开了 1,931 条人工标注数据，多智能体版本 F1 到 33.3%，比之前最优方法高 2 个点，代价是每次要跑 11 秒左右。
论文提出代码变更影响分析（CIA）工具 katIA，将静态影响分析与 LLM 推理结合，并人工标注了来自 22 个开源 Java 项目、263 个 commit 的 1,931 个变更代码实体作为评测数据集。katIA 有两个变体：katIA-SP 用单个提示词优化静态影响集，katIA-MA 使用三个专用智能体协作。在自建数据集上，使用 Gemini 2.5 Flash 的 katIA-MA 达到 FQN 匹配 F1 29.1%、CE 匹配 F1 33.3%，比 katIA-SP 的 22.4% 和 27.0% 更高，CE 匹配成绩超过现有最优方法 2.0 个百分点。
代价是耗时增加：中位生成时间从 0.570 秒升至 11.214 秒，中位 token 用量从 1,235 升至 6,462。作者对比三个 LLM 后发现 Gemini 2.5 Flash 在 katIA-SP 下表现最稳定。论文同时指出以往 CIA 研究的数据集与实验设置存在偏差，导致报告结果不可靠，新数据集为此提供了更严格的评测基准。
Hybrid Source Code Change Impact Analysis
Change impact analysis (CIA) helps developers identify code entities impacted by their initial changes. While several approaches for CIA were proposed, they still struggle with either low precision or low recall, thereby increasing manual effort for developers to verify the results. Furthermore, limitations in the used datasets and experimental setups can bias the results reported by many CIA approaches. In this paper, we first address the dataset limitations by manually labeling 1,931 changed code entities stemming from 263 commits of 22 open-source Java projects. Next, we introduce two variants of katIA, our code change impact analysis approach that combines static impact analysis with the reasoning capabilities of large language models (LLMs). The first variant uses a single prompt (katIA-SP) for refining an initial static impact set while the second variant uses three specialized agents (katIA-MA). The evaluation with our curated dataset shows that katIA-SP with Gemini 2.5 Flash achieved an F1-score of 22.4% when matching impacted entities by their fully qualified names (FQN) and 27.0% when matching them by their class and entity names (CE). The latter shows an improvement of 2.0 percentage points over the state-of-the-art. This score is further improved by katIA-MA which, using Gemini 2.5 Flash, yields an FQN-matching F1-score of 29.1% and a CE-matching F1-score of 33.3%. While these improvements come at the cost of increased median generation time from 0.570 s to 11.214 s and median token usage from 1,235 to 6,462 tokens, the improved performance justifies the additional cost. The comparison of three LLMs using katIA-SP finds that Gemini 2.5 Flash is the most consistent model. These results indicate that combining static impact analysis with LLM-based refinement, particularly with the multi-agent variant, can improve code change impact analysis.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include katIA, Gemini 2.5 Flash, 静态分析, 代码变更影响分析.
Source attribution: curated from arXiv cs.AI (@Katharina Stengg, Christian Macho, Martin Pinzger).