训练停止后的记忆：ChatGPT 误把“考虑去土耳其”记成真事
记忆专场金句：ChatGPT 记错旅行，上下文学习赢过专用记忆，改推理就让任务完成率从22%提到60%。
在记忆与持续学习专场，演讲者举例称 ChatGPT 认为 Shlok Khemani 在 2025 年去过土耳其，但他从未去，模型只是读过他犹豫要不要去的对话，无法识别记录冲突。
在首个真实持续学习基准上，普通上下文学习击败了专用记忆系统。将策略自蒸馏扩展到 100 次工具调用后，教师模型的频繁纠正会让模型退化成一个劲说“maybe”。
另一个实验里，教师不改动工具调用 token，只重塑调用前的推理，就把智能体任务完成率从 22% 提到 60%。演讲者还提醒，智能体不擅长发现异常，最好直接让它调查你已经发现的异常。
Memory talks on what happens after training stops: - ChatGPT thinks Shlok Khemani went to Turkey in 2025. He never did. It read a conversation where he was deciding whether to go, and it still cannot tell that its own record conflicts - On the first real continual learning benchmark, plain in context learning beat the dedicated memory systems - Scale on policy self distillation to a hundred tool calls and the teacher course corrects so often the model collapses into saying "maybe" - A teacher moved an agent from finishing 22% of its tasks to 60% without ever changing the tool call tokens. It only reshaped the reasoning leading up to the call - Agents are bad at finding anomalies. Ask them to investigate the anomalies you already found Watch the Memory & Continual Learning track: youtube.com/watch?v=iqloyW…
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Key entities and technical topics under continuous tracking include ChatGPT, 持续学习, 上下文学习, 智能体.
Source attribution: curated from AI Engineer (@aiDotEngineer).
