论文实锤AI只会照搬历史
一篇arXiv论文（2601.22436）发现，当前LLM智能体系统存储过去任务时包含原始步骤历史或总结规则。研究者通过将正确提示替换为随机垃圾文本来测试记忆使用情况：当步骤历史被破坏时，AI表现显著下降；但当总结规则被破坏时，AI性能无变化。这表明AI并未真正应用抽象规则，而是依赖复制精确历史动作。
“If an AI cannot apply an abstract lesson to a new situation, it is not truly reasoning or learning”...
“If an AI cannot apply an abstract lesson to a new situation, it is not truly reasoning or learning”, new study with further evidence backing up what I have been saying for 25 years. cc @dwarkesh_sp Rohan Paul @rohanpaul_ai Researchers found our current approach to making AI smarter over time has a giant blind spot. AI is not actually understanding or applying high-level abstract lessons at all. Developers spend massive amounts of time building systems that condense past AI mistakes into neat little rules for the future. This paper proves that the AI essentially throws those rules in the trash and only looks at raw historical logs. Modern LLM systems try to get better over time by storing past tasks as either raw step-by-step histories or condensed summary rules. The study tested if these agents actually use their stored memories by secretly swapping the correct tips with random garbage text. - When the step-by-step histories were messed up, the AI failed hard, proving it heavily relies on copying exact past actions. - But when researchers completely corrupted the condensed summary rules, the AI kept acting normally and showed zero performance drop. If an AI cannot apply an abstract lesson to a new situation, it is not truly reasoning or learning. This raises the question if the entire AI industry need to rethink how memory works because right now these agents are just mimicking instead of understanding. ---- arxiv. org/abs/2601.22436 "LLM Agents Are Not Always Faithful Self-Evolvers" 🔗 View Quoted Tweet 💬 10 🔄 12 ❤️ 38 👀 2032 📊 13 ⚡