Papers

用 Min-K% 方法追踪编程作业中代码的来源:区分学生与 AI 生成

Analyzing Code Provenance in LLMs for Programming Education

Why we picked it

老师担心作业是 AI 代写的,这篇论文给了个检测思路:把 Min-K% 扩展到代码溯源,还对比了 GPT-5、DeepSeek 等多个模型的检测差异。

这篇论文研究如何在编程教育中判断学生代码是本人写的还是 AI 生成的。作者扩展了 Min-K% 概率方法,加入合成非成员代码生成,用于对 Python 代码做成员推断和来源追踪。测试覆盖 GPT-2、DistilGPT-2、DialoGPT 等免费模型,以及 GPT-5、Gemini 2.5、Mistral、DeepSeek 和 LLaMA 3.3 等商业模型。可行性先用 Stack Overflow 数据验证,再用于高校学生作业,结果显示免费模型比商业模型更容易检测出代码记忆痕迹。

Original · arXiv: DeepSeek

Analyzing Code Provenance in LLMs for Programming Education

Large language models (LLMs) play an increasing role in programming education, challenging traditional assessment methods as they can generate fully functional code instantly. This disrupts the reliability of student submissions as indicators of learning and effort. To address this issue, code provenance is essential for tracing the origins of code and distinguishing student-authored, AI-generated, and externally sourced components. We extend the Min-K% probability approach by incorporating synthetic non-member code generation and apply it to Python code for membership inference and provenance tracking. Evaluations cover free LLMs such as GPT-2, DistilGPT-2, and DialoGPT, as well as state-of-the-art commercial models including GPT-5, Gemini 2.5, Mistral, DeepSeek, and LLaMA 3.3. Before evaluating our approach on higher-education student assignments, we validate its feasibility using Stack Overflow data. The results show that free models enable more sensitive code memorization detection than commercial models. By systematically comparing these LLMs, we assess trade-offs between detection accuracy and privacy protection, taking an initial step toward effective assessment of AI-assisted programming education through combined provenance detection and process visibility.