Keenable AI 开源 NEEDLE 搜索基准

精选理由

Keenable AI 发布了 NEEDLE 基准，每小时更新查询集，防止搜索模型作弊跳过检索步骤。

AI 摘要

Keenable AI 开源了 NEEDLE 基准测试工具，每小时重建一次查询集。该基准专门解决搜索 API 读取答案键的问题。测试对象包括具有获取工具的搜索智能体。基准测试防止模型直接下载公共数据集跳过检索过程。

Keenable AI 开源 NEEDLE 搜索基准 — 图片来源 · marktechpost

原文 · marktechpost

Keenable AI Open-Sources NEEDLE: A Live Search Benchmark That Rebuilds Its Query Set Every Hour

How do you benchmark a web search API when the thing being tested can read the answer key? A search agent has a fetch tool. If the gold labels sit in a public dataset, the agent can download them mid-evaluation and skip retrieval entirely. A similar problem arises when the answers are already encoded in […] The post Keenable AI Open-Sources NEEDLE: A Live Search Benchmark That Rebuilds Its Query Set Every Hour appeared first on MarkTechPost .

阅读原文