这篇论文把自主研究比作模糊测试,点出反馈架构才是瓶颈,不是生成。想搞懂 AI 科研智能体该往哪走,值得一看。
arXiv 论文提出自主研究智能体的 generate-and-rank 范式忽略了稀疏反馈问题。研究者认为其控制循环与灰盒模糊测试器一致,需在每次实验暴露廉价密集的认知进步信号。该信号应指导下一步干预,使智能体搜索而非重复采样,最终验证仍需保护性证据避免自适应复用。论文提出测试候选信号预测验证进步、反馈导向搜索比重复采样单位成本更多验证发现、保护性验证减少假发现。
Agentic Auto-Research is Fuzz Testing
Autonomous research agents can generate experiments faster than researchers can validate them. Researchers have responded by scaling the proposer and ranking more samples with a learned judge or human reviewers. We argue that this *generate-and-rank* paradigm misses the problem of sparse feedback. Within a declared research problem, an agent follows the control loop of a greybox fuzzer: it proposes a candidate, executes it, observes feedback, and chooses what to try next. A fuzzer rarely finds a bug, but coverage makes partial progress observable on every execution. Fuzzers then use that signal to mutate inputs and allocate effort, rather than only to rank completed runs. Auto-research needs the same two capabilities. First, each experiment should expose a cheap, dense signal of epistemic progress before final scientific validation is available. Second, that signal should determine the next intervention so that the agent searches rather than repeatedly samples. Because the optimized progress signal is guidance rather than a verdict, final validation must still decide what counts as a discovery using evidence protected from adaptive reuse. We propose controlled tests of whether candidate signals predict validated progress, whether feedback-directed search yields more validated discoveries per unit cost than repeated sampling, and whether protected validation reduces false discoveries. Feedback architecture, not only generation, is a central bottleneck in auto-research.