PRIME:用反事实路由改进 LLM 科学创意的修订决策层
PRIME: Predictive Routing of Interventions by Modeling Effects for Scientific Ideation
这篇论文给现有的 AI 科研创意系统加了个“改哪、怎么改”的决策层,靠反事实推演选修改方案,消融数据挺扎实,做科研智能体的可以看看。
arXiv 论文提出 PRIME,一个可插拔的反馈到修订决策层,用于改进现有 LLM 科学创意系统。它在结构性归因、反事实路由和干预执行三个阶段工作,核心是解决作者命名的“诊断-干预错位”问题:弱点出现的位置不等于应修改的范围。验证基于 112 条人工标注的创意-反馈对,横跨四个基准、两个创意生成器和固定与开放问题设定。消融实验显示去掉反事实路由后质量从 3.463 降到 1.960,路由遗憾从 0.426 升至 1.929。
PRIME: Predictive Routing of Interventions by Modeling Effects for Scientific Ideation
Feedback-driven iteration is widely used to improve LLM-generated scientific ideas, but existing systems either leave revision decisions implicit in generation, align revisions with feedback-localized weaknesses, or restrict action selection to system-specific operations. This is problematic because problems, hypotheses, and solutions are interdependent: the location where a weakness manifests does not itself determine the revision scope that will produce a better successor. We term this phenomenon diagnosis-intervention mismatch. We introduce Predictive Routing of Interventions by Modeling Effects (PRIME), a plug-and-play feedback-to-revision decision layer for existing scientific ideators. PRIME operates in three stages: (1) Structural Attribution interprets feedback over the problem-hypothesis-solution structure without fixing a revision target; (2) Counterfactual Routing constructs alternative scope-conditioned interventions and compares their projected successor utility and risks through lightweight rollouts; and (3) Intervention Execution compiles the selected intervention into an ordered generator-facing plan with semantic actions, preservation constraints, and dependency requirements for successor generation. We validate diagnosis intervention mismatch on 112 human-annotated idea-feedback pairs. Across four benchmarks, two ideators, and fixed- and open-problem settings, PRIME improves successor quality and routing reliability over strong baselines; removing counterfactual routing reduces quality from 3.463 to 1.960 and increases route regret from 0.426 to 1.929, confirming cross-scope effect modeling as the primary source of its gains.