GIM 用多认知域整合任务戳穿了现有基准的饱和困境，做 LLM 评估的团队可以直接用它来检测模型真实推理能力，比 GPQA 和 ARC-AGI 更贴近实际应用场景。
GIM（Grounded Integration Measure）是一个包含 820 道原创问题的新基准，旨在通过要求模型协调多种认知操作（如约束满足、状态跟踪、认知警觉、受众校准）来评估 LLM，而非单纯增加知识难度或抽象推理。该基准使用公共-私有问题拆分以检测数据污染，并基于超过 20 万次模型响应校准了 IRT 模型，提供更稳健的能力估计。研究对 22 个模型和 47 种测试配置进行了全面评估，发现思考预算和量化等家族内配置选择与模型选择同等重要。GIM 解决了现有基准在记忆与能力、推理与实践脱节上的问题，为 LLM 评估提供了新视角。
GIM: Evaluating models via tasks that integrate multiple cognitive domains
As LLM benchmarks saturate, the evaluation community has pursued two strategies to increase difficulty: escalating knowledge demands (GPQA, HLE) or removing knowledge entirely in favor of abstract reasoning (ARC-AGI). The first conflates memorization with capability; the second divorces reasoning from the practical contexts in which it matters. We take a different approach. The Grounded Integration Measure (GIM) is a benchmark of 820 original problems (615 public, 205 private) where difficulty comes from integration; individual problems require coordinating multiple cognitive operations (constraint satisfaction, state tracking, epistemic vigilance, audience calibration) over broadly accessible knowledge, so that reasoning stays grounded in realistic tasks without being gated on specialized expertise. Each problem is an original expert-authored composition, majority with rubric-decomposed scoring (median 6 independently judged criteria). A balanced public--private split provides built-in contamination diagnostic. We calibrate a continuous response 2-parameter logistic (2PL) IRT model over >200k prompt-response pairs across 28 models, producing robust ability estimates that correctly order test-configurations even when raw accuracy is distorted by errors or missing data, addressing a common challenge in benchmark reporting. Using this framework, we present a comprehensive leaderboard spanning 22 models and 47 test-configurations (unique model, thinking-level pairs), and conduct what is to our knowledge the most extensive published study of how test-time compute trades off against model capability on a fixed benchmark: 11 models swept across 35 test-configurations. We observe that within-family configuration choices, such as thinking budget and quantization, matter as much as model selection. We release the evaluation framework, calibrated IRT parameters, and all public problems.