Papers

arXiv 论文:LLM 交易智能体在财报与坏 news 场景中跑不赢简单规则

The Only Edge Was Memorized in Training: LLM Trading Agents Around Earnings and Bad News

Why we picked it

四个厂商的五个模型在 598 次财报和 409 次暴跌上实测,全都没跑赢'每次都买'这种笨策略,GPT 系 59.2% 的方向准确率全靠背训练数据。想做交易智能体的朋友先看这篇,能省掉不少弯路。

arXiv 一篇论文在无前视偏差的基准上测试了 LLM 交易智能体,覆盖 598 次财报公告前与 409 次大盘美股坏 news 暴跌后的交易决策,涉及来自四家厂商的五个模型。所有输入都在决策收盘时截断,并对比了 staged agents、单次调用、简单规则和逻辑回归,每项决策还分别在带与不带股票名称的条件下进行。结果没有任何智能体跑赢简单策略:财报前智能体几乎都在买入,赚到的钱与'每次公告前都买'无异;坏 news 后基于 Gemini 的智能体几乎把每次下跌都当作超跌反弹,但选股表现不比无差别买入更好。唯一的增益来自模型在训练中记住的内容:只给代码和日期时,基于 GPT 的模型对训练截止日期前的公告方向预测准确率为 59.2%,之后降到 53.3%。论文建议智能体开发者必须与简单下注做对照,并在训练截止日期之后做盲测。

Body

这项研究的价值在于方法层面:此前关于 LLM 与财报的研究用的是公告同期或之后发布的文档,关于反转策略的研究按月度组合排序,都无法说明智能体在扣除成本后能否抓住这些交易时点。该基准把新闻按发布时间回放、发现交易新闻本身无 alpha,因此把测试集中在两个可留仓位的时刻——日期确定的财报公告前,以及坏 news 后可能的过度下跌。对想搭交易智能体的人来说,结论很直接:先证明能赢过'闭眼全买'这条基线,再谈智能体架构。

Original · arXiv cs.AI

The Only Edge Was Memorized in Training: LLM Trading Agents Around Earnings and Bad News

Large language model (LLM) trading agents are tested on news, but news reaches stock prices within minutes, and a benchmark that replays headlines at their publication time finds no alpha in trading on them. Money can then remain in two places, before news whose date is known, such as earnings announcements, and after bad news, if the market's fall went too far. However, studies of LLMs and earnings read documents published with or after the announcement, and studies of reversal sort monthly portfolios, so neither tells whether an agent can trade these moments after costs. In this paper, we test LLM agents at both moments on the same look-ahead-free benchmark, before 598 earnings announcements and after 409 bad-news drops of large US stocks. Every input is cut at the decision close, staged agents are compared with single calls, simple rules and a logistic regression, and every decision is made with and without the stock's name. Across five models from four vendors, no agent beat the simple bets. Before earnings, the agents mostly bought and earned what buying before every announcement earned, and after bad news, a Gemini-based agent called almost every fall an overreaction while no agent picked better dips than buying them all. The only gains came from what the models had memorized in training. Given only the ticker and the dates, a GPT-based model called the direction of 59.2% of the announcements before its training cutoff and 53.3% after it. Agent builders should therefore compare an event agent with the simple bet and test it blind after the training cutoff.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include LLM, Gemini, GPT, quantitative trading.

Source attribution: curated from arXiv cs.AI (@Jackson Wang).