这篇论文用49道题测了Gemini、GPT-4等6个模型对科学图表的理解力，发现Gemini最强但人类平均还是更高，开源模型差距明显，想了解模型真实视觉推理能力可以一读。
本文对6个多模态大模型（MLLM）进行了科学可视化素养评估测试，该测试包含49道题目、18个科学可视化图表，覆盖8种可视化技术和11种任务类型。模型成绩与485名人类参与者数据对比，Gemini整体表现最强，在评估子集上超过人类平均水平，而开源模型低于人类基线。模型在科学插画、搜索和空间理解任务上表现较好，但在基于纹理和整合的可视化以及定量估计任务上表现不佳。误差分析显示模型在细粒度定量估计、流向判读和编码含义解读上存在系统性失败。代码和模型输出已开源。
Benchmarking Multimodal Large Language Models for Scientific Visualization Literacy
Multimodal large language models (MLLMs) are increasingly used to interpret visualizations, yet current evaluations remain largely chart-centric and provide limited evidence of understanding of scientific visualization (SciVis). We benchmark six MLLMs on the scientific visualization literacy assessment test, a standardized SciVis literacy assessment comprising 49 items based on 18 scientific visualizations and illustrations, spanning 8 techniques and 11 task types. We evaluate three closed-source and three open-source models under a closed-world protocol and compare their performance using data from 485 human participants. Results show that current MLLMs do not exhibit uniform SciVis literacy. Gemini is the strongest model overall, exceeding the human mean across the evaluated subsets, whereas the open-source models remain below the human baseline. Performance is highly uneven across techniques and tasks: models perform best on scientific illustration, search, and spatial understanding, but struggle on texture-based and integration-based visualizations and on quantitative estimation. Error analysis reveals recurring failures in fine-grained quantitative estimation, flow-direction interpretation, and grounded encoding interpretation. These findings position SciVis literacy as a necessary benchmark dimension for evaluating multimodal AI systems. Our code and model outputs are publicly available at https://github.com/patdmp/mllm-scivis-lit-benchmark.