想了解提示语言怎么影响代码生成质量？这篇实测了GPT-4o mini、DeepSeek和Claude在多种语言下的表现，结果英文不一定最好。
该研究分析了GPT-4o mini、DeepSeek和Claude在460个编程任务（Python和Java各230个）上的代码生成质量，将英文提示翻译并人工校订为中文、印地语、西班牙语和意大利语。通过测试通过率、代码度量标准和静态分析工具评估，发现英文提示并非总能产生最佳功能正确性或代码质量，提示语言的影响取决于编程语言和LLM，且生成的代码常在注释和字符串中混合英语与提示语言。这是首个用于研究代码生成中语言偏差的精选多语言基准。
Large Language Models for Code Generation from Multilingual Prompts: A Curated Benchmark and a Study on Code Quality
Large Language Models (LLMs) perform differently on identical programming tasks when prompted in different natural languages, a phenomenon known as language bias. While this behavior has been widely studied for general text generation, its impact on code generation quality and programming conventions remains largely unexplored. We investigate how the language used to describe programming tasks affects the source code generated by GPT-4o mini, DeepSeek, and Claude. Our study comprises 460 coding tasks spanning Python (230) and Java (230). We translate and manually curate the original English prompts into Chinese, Hindi, Spanish, and Italian while preserving their technical meaning. We evaluate the generated code using multiple dimensions, including functional correctness through test pass rates, structural quality using established code metrics, issues detected by static analysis tools, and lexical characteristics such as the language used in identifiers and comments. Our results show that (i) English prompts do not consistently produce the best functional correctness or code quality, (ii) the impact of prompt language depends on both the programming language and the LLM, and (iii) generated code frequently mixes English with the prompt language in comments and string literals. These findings provide the first curated multilingual benchmark for studying language bias in code generation and offer insights for developing more robust multilingual code generation systems.