做LLM评估和科学推理研究的团队终于有了一个能独立控制提取与推理难度的基准，可以精准诊断模型短板。想了解自家模型在科学推理上到底弱在哪，建议直接看这篇。
SciR 是一个新的科学推理基准，专门评估大语言模型在科学场景下的演绎、归纳和因果推理能力。它通过从形式化对象（如演绎树、归纳规则假设、因果图）生成任务，确保答案可验证，再渲染成多文档科学文本。该基准独立控制两个难度轴：信息提取难度和推理本身难度，从而揭示模型在不同维度上的表现差异。测试六个模型后发现，两个难度轴都会降低模型性能，且效果叠加，即使是神经符号管道也受渲染影响。推理模型如DeepSeek-R1主要在推理轴上优于非推理指令模型。
SciR: A Controllable Benchmark for Scientific Reasoning in LLMs
Three paradigmatic forms of inference recur across scientific reasoning: deduction, induction, and causal abduction. Reliably evaluating LLMs on these in scientific settings is currently out of reach: scientific benchmarks built on human annotations are costly and lack mechanistic ground truth, while synthetic logical-reasoning benchmarks do not resemble real scientific documents. We introduce SciR, a benchmark that combines multi-paradigm reasoning with controllable scientific rendering, anchored on three paradigmatic scientific problems. Tasks are generated from formal objects (deduction tree, inductive rule hypothesis, causal graph) to guarantee verifiable answers, then rendered into multi-document scientific discourse via per-track domain-tuned genres. The construction lets us independently vary two difficulty axes: how hard it is to extract the key information needed for inference, and how hard the principled inference itself is. We test six models. Both axes hurt every model, and their effects compound. The rendering even hurts neurosymbolic pipelines, which hand inference to a verified solver. The two axes yield a per-model extraction-vs-inference profile: for instance, reasoning models like deepseek-r1 mostly surpass non-reasoning instruct models on the inference axis. To our knowledge, SciR is the first multi-paradigm scientific-reasoning benchmark with parametric control on both extraction and inference difficulty.