这篇论文拿IdeaGene-Bench测了14个LLM，最强也只有27.3%准确率，而且加谱系信息反而搞乱排名。想看看AI离真正理解科学进化还有多远？读它。
该论文提出IdeaGene-Bench (IG-Bench)基准，包含1,961条黄金谱系轨迹、1,085个Idea Genome对象和920个pairwise GenomeDiff记录，覆盖10个科学领域。IG-Exam含42个任务类型、1,029个实例，用于封闭式谱系推理；IG-Arena使用种群进化得分(PES)评估生成质量。在14个LLM科学家的测试中，最强系统仅达到27.3%的精确准确率，且结构化谱系上下文会打乱模型排名。该研究揭示了当前AI在科学思想继承与变异推理上的组合瓶颈。
Ideas Have Genomes: Benchmarking Scientific Lineage Reasoning and Lineage-Grounded Idea Generation
Scientific ideas rarely start from a blank page. They inherit mechanisms, repair known limitations, and recombine pieces of earlier work, much like biological genomes. Current benchmarks still say little about whether AI systems can follow this inheritance structure. We present IdeaGene-Bench (IG-Bench), a benchmark for scientific lineage reasoning and lineage-grounded idea generation. IG-Bench is organized around the IdeaGene framework: each paper or proposal is represented as a set of minimal, typed, evidence-grounded Idea Genome objects, and a GenomeDiff aligns these objects to record inheritance, mutation, loss, external import, and novel insertion under six operational evolutionary dynamics. The benchmark contains 1,961 golden lineage traces, 1,085 curated Idea Genome objects, and 920 pairwise GenomeDiff records across 10 scientific domains. It supports two evaluations. IG-Exam (42 task types, 1,029 instances) tests closed-form lineage reasoning across Idea Genome abstraction, inheritance tracing, evolutionary reasoning, and lineage verification. IG-Arena evaluates generation with a lineage-conditioned Population-Evolution Score(PES), asking whether a proposal can be inserted as a coherent descendant of a given lineage population: it should inherit the right Idea Genome objects, vary meaningfully from nearby work, and offer selection value for future research. Experiments on 14 LLM-based scientists expose a compositional bottleneck. The strongest system reaches only 27.3% exact accuracy on lineage reasoning, and structured lineage context reshuffles system rankings rather than helping every participant uniformly.