LLM-Enhanced Commit Message Generation via Issue Information: An Exploratory Study

精选理由

研究ISAC框架,了解如何利用问题信息提升LLM生成提交信息的效果,与现有方法相比有显著提升。

AI 摘要

提出ISAC框架,结合代码差异和问题信息生成提交信息,ApacheCM-Issue数据集支持评估,GPT-5.5和DeepSeek-V4-Flash模型测试,问题信息提升模型性能,结构化问题摘要改善感知完整性,但可能牺牲上下文细节。

原文 · arXiv: DeepSeek

Commit messages help developers understand code changes, support collaboration, and improve long-term maintenance. However, the use of issue information alone as the external context for LLM-based CMG has not been systematically studied. We propose an ISsue-Augmented framework for Commit message generation (ISAC) by combining code diffs with issue information as LLM input. To support the evaluation, we construct ApacheCM-Issue, a commit-issue aligned dataset built upon ApacheCM by linking commits with issues from GitHub and Apache Jira. Using samples from Scala, Java, and C++ projects, we evaluate four input configurations using two representative LLMs, GPT-5.5 and DeepSeek-V4-Flash in different reasoning configurations. The results show that incorporating issue information consistently improves LLM-based CMG across all evaluated model configurations and metrics, with the largest gains observed for CIDEr. Incorporating a similar historical commit further improves automatic metric scores, while replacing full issue information with a structured issue summary decreases them. ISAC also outperforms the four reproduced state-of-the-art (SOTA) CMG baselines across all five automatic metrics on the experimental dataset. The human evaluation further shows that structured issue summaries may improve perceived completeness, although replacing the original issue information can sacrifice contextual details and lead to worse results on automatic metrics.

  • Suhail08-20 21:25原文
  • 深度求索 DeepSeek08-21 09:17原文
  • 歸藏(guizang.ai)08-23 09:23原文
  • AI Will08-24 02:34原文