ActiveVision新基准:GPT-5.5和Claude Fable 5主动观察得分不足11%

An Exam for Active Observers

精选理由

想看最强的AI在实时观察任务上有多弱吗?ActiveVision让GPT-5.5和Claude Fable 5原形毕露,人类碾压它们。

AI 摘要

多模态大模型(MLLM)在主动观察任务上能力缺失。新基准ActiveVision包含17个任务(3个类别),要求模型反复视觉感知而非单次描述。GPT-5.5在最高推理档次上仅解决10.6%的项,11个任务得零分;Claude Fable 5仅3.5%,而三名人类被试平均96.1%。即使让模型编写并运行自己的视觉代码,大部分差距依然存在,因为代码在真实图像上不可靠,且捕捉自身失败需要模型缺乏的主动感知。结果表明当前MLLM缺乏稳健的主动视觉观察能力。

原文 · arXiv cs.AI

An Exam for Active Observers

Human vision is a closed loop: gaze is continuously redirected by intermediate hypotheses rather than a single snapshot. Decades of psychophysics and cognitive science have argued that this active observation is essential for a wide range of tasks. Whether today's multimodal large language models (MLLMs) exercise active observation is an empirical question that current vision-language benchmarks do not answer. We introduce ActiveVision, a benchmark that makes active observation measurable for MLLMs, comprising 17 tasks across 3 categories. Tasks are designed to force repeated visual perception rather than a single static description. Frontier MLLMs collapse on ActiveVision: the highest-scoring model we evaluate, GPT-5.5 at the highest exposed reasoning-effort tier, solves only 10.6% of items and scores zero on 11 of the 17 tasks, and even Claude Fable 5, despite topping most reasoning and coding leaderboards, solves just 3.5%, far behind three human participants who average 96.1%. Furthermore, much of the gap persists even when models write and run their own vision code: such code is unreliable on realistic imagery, and catching its failures itself requires the active perception the models lack. Together, these results indicate that current MLLMs lack robust active visual observation, motivating architectures and training objectives that close the perception-reasoning loop.