做多语言智能体或网页浏览任务的团队会立刻意识到差距——韩语场景下最强模型准确率不到一半，说明现有评估严重偏向英语。做韩语 NLP 或本地化产品的开发者可以直接用这个基准测试自己的模型。
K-BrowseComp 是一个专门针对韩语环境的网页浏览智能体基准测试，包含 400 个问题。其中 300 个问题由韩语母语者手工构建和验证，前沿模型如 GPT-5.5、DeepSeek-V4-Pro 和 GLM-5.1 在该子集上仅达到 30.00-45.67% 的准确率，远低于 BrowseComp 的表现。韩国本土大模型表现更差，仅 0.00-10.33%。研究还构建了 100 个合成问题作为压力测试，最强模型仅达 26.00%。该基准填补了韩语智能体评估的空白，揭示了当前模型在非英语环境下的显著短板。
K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts
Frontier model evaluations are shifting from foundational capabilities (e.g., instruction following and reasoning) toward compositional, agentic ones, but Korean agentic benchmarks remain scarce. We introduce K-BrowseComp, a web-browsing agent benchmark grounded in Korean contexts, consisting of 400 problems. The 300-problem K-BrowseComp-Verified subset is manually constructed and validated by native Korean speakers. On this subset, frontier LLMs, including GPT-5.5, DeepSeek-V4-Pro, and GLM-5.1, reach only 30.00--45.67\%, a substantial drop from BrowseComp, while Korean LLMs released through Korea's Proprietary AI Foundation Model program obtain only 0.00--10.33\%. We further construct a 100-problem synthetic split using hard few-shot exemplars and failure-mode-targeted generation to exploit the asymmetry between solving and creating web browsing problems. On the adversarially filtered synthetic diagnostic split, the strongest model reaches only 26.00\%, and we report this split separately as a targeted stress test. We publicly release our data and code.
- IT之家06-02 08:13原文