想测试数学推理模型是真会思考还是靠蒙?这个挑战赛提供了新基准和资源,帮你判断模型的鲁棒性。
AIMO可解释性挑战赛旨在通过模型内部机制区分前沿数学语言模型的鲁棒推理与虚假推理。比赛基于AI数学奥林匹克(AIMO)问题及Fields Model Initiative资源,提供新发布的奥林匹克级数学推理题及其符号表示。参赛者可访问前沿推理模型及其对抗鲁棒性评估,开发识别鲁棒推理的方法。比赛将创建开放的鲁棒性基准和基线系统,推动数学推理与可解释性研究。
AIMO Interpretability Challenge
We propose the AIMO Interpretability Challenge, a competition on distinguishing robust from spurious reasoning in frontier mathematical language models based on the models' internal mechanisms. The challenge is motivated by a central limitation of standard reasoning benchmarks: strong final-answer accuracy does not reveal whether a model relies on stable reasoning mechanisms or exploits brittle reasoning shortcuts. Building on AI Mathematical Olympiad (AIMO) problems and submissions, together with resources from the Fields Model Initiative, the competition will provide (1) newly-published olympiad-level math reasoning problems and their symbolic representations, allowing generation of novel functional variants, (2) access to frontier reasoning models, and (3) assessments of models' adversarial robustness on these problems. Participants will use these resources, along with our computing infrastructure support, to develop methods for identifying which models solve problems robustly. Our competition will also create a new, open robustness benchmark and baseline systems, aiming to provide a lasting foundation for standard benchmarking in mathematical reasoning and interpretability. Scientifically, the competition connects interpretability and generalization research around a central question in AI research: can we determine if, and to what extent, the decision-making of frontier AI models is generalizable and thus, reliable?