论文精选

Hedge-Bench 1.0:对冲基金分析师真实任务基准,AI 得分不足 16%

Hedge-Bench: Benchmarking Agents on Hard, Realistic Tasks Pertaining to Financial Reasoning

精选理由

金融 AI 终于有了硬核的推理基准——不是算公式或查文档,而是真正考验分析师级别的开放式问题。做量化、金融 NLP 或智能体评估的团队值得关注,可以直接用这个 benchmark 检验自家模型。

AI 摘要

Hedge-Bench 1.0 是一个针对金融推理的 AI 智能体基准测试,包含 102 个来自对冲基金分析师实际工作中的真实任务。与现有依赖模型评判的基准不同,它基于专家推理轨迹进行确定性评分,避免了噪声和循环论证。测试结果显示,前沿模型和智能体在该基准上的得分低于 16%,说明当前 AI 在复杂金融推理上仍有巨大差距。该数据集和评估工具已在 GitHub 开源。

原文 · arXiv cs.AI

Hedge-Bench: Benchmarking Agents on Hard, Realistic Tasks Pertaining to Financial Reasoning

AI agents can increasingly handle the mechanical tasks of financial analysis: retrieving documents, calculating formulas, updating spreadsheets. The harder, more valuable challenge is reasoning through the open-ended questions that define expert Analyst work. Existing benchmarks do not capture this class of problem, and those that attempt to evaluate open-ended reasoning rely on model-judged outputs that introduce noise and circularity. We present Hedge-Bench 1.0: a benchmark of 102 actual, on-the-job tasks grounded in the explicit reasoning traces of professional hedge fund analysts working with relevant information sources. This approach enables deterministic grading against verified expert steps. Frontier models and agents score below 16\% on the benchmark. We publish the dataset and evaluation harness at github.com/Trata-Inc/trata-hedge-bench.