论文多源确认

Epoch AI 发布 FAB 基准:让模型看图找出宜家家具拼装错误

精选理由

Epoch AI 搞了个叫 FAB 的新基准,让模型看宜家说明书加半成品照片找拼装错误,10 个月内最高分从 28% 冲到 80%。

Epoch AI 推出 Furniture Assembly Benchmark(FAB),给模型提供宜家家具说明书和半成品照片,要求找出拼装错误。该基准上线时的最高成绩为 28%,10 个月内已升至 80%。评测重点考察模型对图像与说明文本的联合理解能力,而非单纯的文本推理。

原文 · Epoch AI

Can AI tell if you've built your IKEA furniture wrong? Our new benchmark, the Furniture Assembly Benchmark (FAB), gives models the manual and a photo of a half-completed piece of furniture and asks them to spot the mistake.

The top score has gone from 28% to 80% in just 10 months.

  • Decoder16:00原文
  • Nathan Lambert: Interconnects09-22 13:37原文