MKA 基准发布：针对视觉复杂文件的知识工作智能体评测
Artificial Analysis 和 Google 做了个专门测智能体处理复杂文档的基准，任务由行业专家写，还故意保密防止模型背题。首批数据里 Gemini 4 Carbon 分数略输 Claude Fable 5.2，但成本只有对方五分之一，做模型选型的人可以直接看榜单。
Artificial Analysis 与 Google 联合推出 MKA（Multimodal Knowledge Agent）基准，用于评测处理视觉复杂文件的智能体。基准包含 200 个任务，覆盖 10 个行业与 83 个职业，由领域专家撰写，每个任务相当于资深专业人员 3 至 8 小时的工作量。
评测机制上，每个任务设有约 70 项 yes/no 评分项，权重按 75% 事实准确性与 25% 视觉设计计算。所有模型在统一的 Stirrup 框架中运行，使用相同工具、沙箱与对话轮次预算。200 个任务由 Artificial Analysis 私有保管，防止泄漏进训练数据，另有 10 个公开任务放在 Hugging Face 上。
首批结果显示 Gemini 4 Carbon 质量排名第二，比 Claude Fable 5.2 低 3 分，但单任务成本为 6.29 美元，远低于后者的 31.97 美元。对于需要评估模型在真实专业文档场景能力的开发者与企业，这个基准提供了一个抗污染的参考。
Evaluations are the backbone of any frontier progress. Today we're launching MKA (Multimodal Knowledge Agent) with @ArtificialAnlys , a benchmark for agents doing knowledge work on visually complex files. - 200 tasks across 10 verticals and 83 occupations, written by domain experts - Each task takes a seasoned professional 3 to 8 hours - ~70 yes/no rubric checks per task, weighted 75% factual accuracy and 25% visual design - Every model runs in the same harness (Stirrup) with the same tools, sandbox and turn budget - 200 tasks stay private with Artificial Analysis, so they can't leak into training data - 10 public tasks on @huggingface 🤗 Gemini 4 Carbon is #2 on quality, 3 points behind Claude Fable 5.2, at $6.29 per task vs $31.97. Blog: blog.google/innovation-and… D Results: artificialanalysis.ai/evaluations/mka 6📊 9 ⚡
This report is categorized under AITOP's "Models" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to US-China foundation model benchmarks and commercialization comparison and AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include MKA, Artificial Analysis, Gemini 4 Carbon, Claude.
Source attribution: curated from Philipp Schmid (@_philschmid).
