这个基准直击当前 AI 智能体在研究场景中的短板——不是执行能力不够，而是缺乏研究者的细腻判断。做 AI 评估或智能体开发的团队值得关注，它揭示了提升 AI 研究素养的新方向。
随着基础模型和智能体框架的进步，AI 在研究任务中展现出强大能力，但仍无法完全替代人类研究人员。为此，研究者提出了 AARR（Act As a Real Researcher）基准系列，首个基准 AARRI-Bench 专注于评估智能体在细粒度研究场景中的专业性、严谨性和推理能力。实验显示，最佳配置（Mini-SWE-Agent 搭配 Claude Opus 4.7）仅达到 68.3% 的成功率，常忽略人类研究者能轻易察觉的细微关键细节。结果表明，开发类人研究 AI 需要更深入地探索研究行为，而非仅依赖复杂框架。数据已开源。
Act As a Real Researcher: A Suite of Benchmarks Evaluating Frontier LLMs and Agentic Harnesses in Research Lifecycle
As foundation models advance and agent scaffolding becomes increasingly sophisticated, agents have demonstrated remarkable proficiency in complex, long-horizon coding tasks and even autonomous experiment execution. Despite their evolution from research assistants into autonomous research agents, these systems still exhibit significant limitations in field sensitivity, research ethics, and nuanced scientific judgment. Consequently, frontier agents remain unable to fully replace human researchers. To bridge this gap, we conceptualize the AARR (Act As a Real Researcher) benchmark series. Unlike existing benchmarks that primarily assess macro-level execution capabilities, AARR focuses on whether agents can emulate the professionalism, thoroughness, and nuanced reasoning that characterize human researchers in granular research scenarios. In this work, we propose AARRI-Bench (Act As a Real Research Intern), the first benchmark in this series. We conduct extensive experiments across frontier models and agentic systems, revealing that even the best-performing configuration (Mini-SWE-Agent with Claude Opus 4.7) achieves only 68.3\% success rate, frequently overlooking subtle yet critical details that are obvious to real human researchers. Our results indicate that developing researcher-like AI requires further exploration of research behavior, rather than merely complex scaffolding. Our data is released at https://github.com/AARR-bench/AARRI-bench.