想知道AI做研究到底行不行？这篇论文用两个真实案例告诉你：干活还行，但搞科研还差得远，五个硬伤很实在。
该研究提出“影子评估”方法，让AI agent尝试回答两篇未发表的NeurIPS 2026论文的核心研究问题。agent在6天内消耗数千美元算力完成了全部工程任务，但未取得实质性进展，两篇论文均被原作者明确拒绝。研究归纳了五个失败模式：对发表标准判断不佳、研究设计缺陷缺乏创造性应对、无效回溯、资源意识差、指令漂移。另一模型和脚手架的稳健性检验重现了相同失败。
Can AI agents conduct open-ended AI research? Early evidence from two case studies
Forecasts of explosive AI progress hinge on AI agents automating AI research. But evidence on whether agents can carry out open-ended AI research is thin. Current evaluations either test agents on narrow, verifiable tasks, which excludes open-ended research, or submit AI-generated papers to blind peer review, which is overstretched, stochastic, and suffers from poor review quality. We introduce a third way to measure progress towards AI R\&D automation. An agent takes on the central, open-ended research question of a high-quality unpublished paper, and the paper's original authors grade its output. We call these shadow evaluations. We ran shadow evaluations on two unpublished NeurIPS 2026 submissions, giving frontier agents six days and thousands of dollars of compute. The agents completed all of the engineering without human help, yet could not make substantial progress towards answering the research questions. As a result, both papers were unambiguously rejected by the authors. We identify five recurring failure modes: poor judgment about the bar for publishable research, uncreative responses to shortcomings in the research design, ineffective backtracking from dead ends, poor resource awareness, and instruction drift. A robustness check with a second model and scaffold reproduced these failures. We release the expert reviews, survey responses, agent repositories, and logs. Our results provide early evidence that today's agents can do the engineering of AI research, but struggle with critical parts of the research lifecycle.