XREPOTEST提供了针对大型语言模型单元测试生成的挑战性基准，揭示了不同模型在不同上下文下的表现差异，值得研究。
XREPOTEST是一个针对大型语言模型的多语言仓库级单元测试生成基准测试，涵盖Rust、Go、Julia、PHP和Ruby五种语言。它使用容器化执行框架和多种上下文增强策略，包括文件级、基于LSP和基于检索的上下文。实验结果表明，独立测试和仓库级测试之间存在显著差距，并揭示了更丰富的上下文和测试可靠性之间的权衡。
XREPOTEST: Benchmarking Multilingual Repository-Level Unit Test Generation for Large Language Models
Large language models (LLMs) have shown promise for automated unit test generation, but existing evaluations largely rely on standalone settings and a narrow set of programming languages, overestimating real-world readiness. We introduce XREPOTEST, a multilingual repository-level benchmark for unit test generation spanning five underexplored languages: Rust, Go, Julia, PHP, and Ruby. XREPOTEST evaluates tests under realistic repository constraints using a containerized execution framework and multiple context augmentation strategies, including file-level, LSP-based, and retrieval-based context. Beyond standard metrics such as test pass rate and coverage, we propose Invocation Rate (IR) to assess whether generated tests meaningfully exercise the intended functionality. Experiments with 14 state-of-the-art LLMs, including Claude 4.5, GPT-5.2, DeepSeek V4-Pro, and Qwen families, reveal a substantial gap between standalone and repository-level performance, as well as trade-offs between richer context and test reliability. Overall, XREPOTEST provides a challenging and informative benchmark to advance scalable and robust unit test generation in realistic software environments. The dataset and code are publicly available at: https://github.com/solis-team/XRepoTest