工具加持下 Claude Haiku 5.5 跑赢无工具的 Fable 5.2
把工具拆掉后,Claude Fable 5.2 在 VIALS-AA 上反而输给带工具的 Haiku 5.5,后者每任务只要 $0.07。做科学图像分析的团队可以据此调整选型:小模型加代码沙箱可能比直接上大旗舰更省钱也更准。
Artificial Analysis 在其 VIALS-AA 基准(基于 Handshake AI 的 VIALS 生命科学视觉推理基准的复现)上对比了 agentic 与 one-shot 两种模式:默认模式下模型在沙箱中思考、运行代码并查看图片,one-shot 模式则一次性给出问题和图片、不使用任何工具。
结果显示工具使用让各模型平均得分从约 25% 提升到约 47%,接近翻倍。Claude Fable 5.2(max)带工具以 67% 领跑,每任务成本 $4.20;去掉工具后仅 44%,成本 $0.73。而 Claude Haiku 5.5(max)带工具得 49%、每任务 $0.07,成绩高于无工具的 Fable 5.2,成本约为其十分之一。
对开发者而言,这意味着在科学图像解读类任务上,低成本模型套上 agent 框架可以在成本和成绩上同时压过单次调用的大模型,选型时应优先考虑工具链而非单纯堆模型尺寸。
Take away its tools, and Claude Fable 5.2 scores below Claude Haiku 5.5 with tools on VIALS-AA; across models tool use nearly doubles scores, from ~25% to ~47% on average
VIALS-AA is our implementation of @joinHandshakeAI's VIALS benchmark for visual reasoning in life sciences research. By default, models work as agents: they think, run code and view images in a sandbox. We reran VIALS-AA in a one-shot format: one request with the question and images, and no tools.
Lower cost models in an agent harness can beat single-shot large models on both cost and performance in visual reasoning and interpretation of scientific artifacts.
Claude Fable 5.2 (max) leads VIALS-AA at 67% with tools, at a cost of $4.20 per task; without tools it scores 44% at $0.73 per task. Claude Haiku 5.5 (max) scores 49% with tools, at $0.07 per task: higher than Fable without tools, at about a tenth of the cost.
This report is categorized under AITOP's "Models" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to US-China foundation model benchmarks and commercialization comparison and AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Claude, Claude Haiku 5.5, Claude Fable 5.2, VIALS.
Source attribution: curated from Artificial Analysis (@ArtificialAnlys).