System-One RSI:用引导式练习为交互式智能体建立肌肉记忆
System-One RSI: Building Muscle Memory for Interactive Agents through Guided Practice
推理模型慢、做不了实时控制,这篇用内循环 OODA 决策加外循环智能体复盘的方法解决了。玩《黑神话:悟空》胜率从 0% 练到 66.7%,只需少量真实对局。做游戏智能体或机器人实时控制的开发者可以仔细看实验设置。
arXiv 论文提出 System-One RSI 框架,让智能体通过自主练习掌握亚秒级实时控制,无需专家演示。框架包含两个循环:高频在线内循环是 OODA 周期,由一个 27B 决策模型将视觉观察和任务问题转为结构化判断,再用决策树转化成带时间戳的动作,全程不调用推理模型以避免延迟;离线外循环由一组推理智能体组成,负责修订决策模型提示词和决策树、抓取交互记录、从记录中生成监督信号并训练更新参数。
实验在三个环境中验证:Snake 平均分从 1.30 升至 7.35;《绝区零》评测场景中位完成时间缩短 8.4%;《黑神话:悟空》战斗场景中,闪避策略平均存活时间是静止基线的 11 倍,进攻策略胜率从 0% 提升至 66.7%。整个流程只需少量真实对局,不需要专家演示、环境改造、沙盒隔离或并行仿真。
该方法的借鉴点是:推理模型负责离线复盘和改进,决策模型负责在线低延迟执行,两者分工解决了推理延迟与实时控制的矛盾。对需要做游戏智能体、机器人控制等实时交互任务的开发者,这是一条不依赖人工标注数据的训练路线。
System-One RSI: Building Muscle Memory for Interactive Agents through Guided Practice
Reasoning models provide strong multimodal task understanding but struggle with sub-second control because deliberation increases decision latency. Fast decision models can meet the response window, yet prompting and few-shot examples do not readily provide the fine-grained experience required by a new task. Inspired by coached motor learning, this paper presents System-One Recursive Self-Improvement (System-One RSI), which couples two loops. The high-frequency online inner loop is an Observe, Orient, Decide, Act (OODA) cycle: a decision model maps each visual observation and task question to structured judgments, and a decision tree converts them into timed actions without invoking a reasoning model. The offline outer loop is a recursive improvement cycle carried out by a team of reasoning agents. The team revises the decision-model prompt and decision tree, captures interaction records, derives supervision from those records, and trains or deploys updated model parameters. The improved system then returns to the environment and generates the experience for the next round, consolidating learned distinctions as external muscle memory. The framework operates with a small number of real episodes, without expert demonstrations, environment modification, sandboxing, or parallel simulation. Experiments in Snake, Zenless Zone Zero, and Black Myth: Wukong use a 27B decision model. Mean Snake score increases from 1.30 to 7.35. In an evaluated scenario of Zenless Zone Zero, median completion time decreases by 8.4%. In an evaluated combat scenario of Black Myth: Wukong, the evasive policy achieves 11 times the mean survival time of a stationary reference, while the offensive policy win rate rises from 0% to 66.7% across the evaluated configurations.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include System-One RSI, reasoning models, agents, decision trees.
Source attribution: curated from arXiv cs.AI (@Qingxu Fu, Xuanfa Jin, Ziqian Chen, Zhaoyang Liu, Bolin Ding).