新基准 MM-IssueLoc 专门测模型到底有没有用截图这类视觉证据，而不是全靠文本猜。当前最强系统得分才三十多，说明视觉定位远没解决。
MM-IssueLoc 是评估多模态仓库级问题定位的新基准，包含 652 个 issue-PR 实例、23 种编程语言，标注了 7 种图像类别和 4 个相关性等级。它提供文件级与函数级真值标签，支持纯文本与带图像两种评估模式。在测试中，最强 agent 仅达到 38.96 文件 Acc@5 和 22.45 函数 Acc@10，最强检索器达到 33.86 函数 Acc@10，表明现有系统在多模态定位上仍不可靠。该基准将视觉证据作为显式评估变量，避免与下游补丁生成效果混在一起。
MM-IssueLoc: A Controlled Benchmark for Evaluating Visual Evidence in Multimodal Repository-Level Issue Localization
Real repository issues routinely include visual evidence such as screenshots, error dialogs, rendered UI states, and logs, yet repository-level issue localization is evaluated mostly as a text-only task. Existing multimodal SE benchmarks evaluate end-to-end repair, entangling localization with patch synthesis and obscuring whether visual input helped, hurt, or was ignored. We introduce \textbf{MM-IssueLoc}, a controlled benchmark and evaluation protocol for repository-level localization with visual evidence. MM-IssueLoc contains 652 issue-PR instances across 23 languages, with annotations for 7 image categories and 4 relevance levels. It provides file-level and function-level gold labels, paired text-only and with-image evaluation, and VCE-based diagnostics that convert images into structured textual evidence. We evaluate LLM-based and retrieval-based systems, including MM-IssueLoc-VL-Emb as a controlled multimodal retriever. Results show that existing systems remain far from reliable multimodal repository localization: the strongest agent reaches 38.96 file Acc@5 and 22.45 function Acc@10, while the strongest retriever reaches 33.86 function Acc@10. Cross-benchmark comparisons show that high localization scores on text-dominant SWE benchmarks do not transfer cleanly to multimodal issue localization. MM-IssueLoc turns visual evidence into an explicit evaluation variable, enabling future work to test whether systems improve by using visual evidence for localization, rather than by relying on text-only cues or downstream patch-generation effects.