AgenticSTS用Slay the Spire 2游戏测试LLM智能体记忆:无记忆胜率3/10,加技能6/10,人类16%。做长时域智能体研究可以参考这个方法论。
AgenticSTS提出了一种有界记忆合同,通过类型化检索为每个决策组装新提示,避免跨决策原始记录累计。在Slay the Spire 2游戏中,前沿LLM在不同配置下最低难度胜率为0%,而人类开发者报告胜率为16%。固定A0消融实验中,无记忆基线获胜3/10局,添加策略技能层后获胜6/10局,但差异统计不显著(Fisher精确p≈0.37)。研究发布了298条完整轨迹、冻结记忆/技能快照、提示记录和分析脚本的可重复测试平台。
AgenticSTS: A Bounded-Memory Testbed for Long-Horizon LLM Agents
Memory for a long-horizon LLM agent is a contract about what each future decision is allowed to see. The simplest contract appends past observations, tool calls, and reflections to every prompt, which makes prior context easy to access but also turns it into a jumbled mixture in which the effect of any single memory component is hard to isolate. We introduce and instrument an alternative bounded contract: every decision is made from a fresh user message assembled by typed retrieval, with no raw cross-decision transcript appended. The prompt thus stays bounded across runs of any length, and any single layer can be ablated in isolation. We instantiate the contract in Slay the Spire 2, a closed-rule stochastic deck-building game whose runs require hundreds of tactical and strategic decisions. A public online benchmark of frontier LLMs on the same game reports zero wins at the lowest difficulty across five configurations, and the developer-reported human win rate at the same difficulty is 16%; the task is hard but not saturated. Within our harness, a fixed-A0 ablation shows the largest observed difference when triggered strategic skills are enabled: the no-store baseline wins 3/10 games and adding the skill layer 6/10. At this sample size the comparison is directional rather than statistically decisive (Fisher exact p\approx0.37); a cross-backbone probe and public accumulating-context baselines are reported as operational comparisons rather than controlled tests of the contract variable itself. We release a reproducible testbed: 298 completed trajectories with condition tags, frozen memory/skill snapshots, prompt records, and analysis scripts -- an agent design and a validated, reusable methodology for studying how explicit memory layers shape long-horizon LLM-agent decisions.