Skyfall AI搞了个叫Morpheus的模拟环境，永不重置，专门测LLM能不能持续学习，结果发现它们根本不行。
Skyfall AI推出Morpheus，首个不重置的持久企业模拟平台，用于评估模型在真实动态环境中的持续学习能力。现有RL基准如Atari、OpenAI Gym、MuJoCo、Procgen均为游戏世界，每几分钟重置一次。团队测试了前沿LLM在Morpheus上的表现，结论是LLM不是持续学习者。
congrats on the launch! As the task horizons on agents gets longer, we need more work evaluating a...
congrats on the launch! As the task horizons on agents gets longer, we need more work evaluating and training models to be better in long-running, open-ended, evolving real-world environments. Skyfall AI @skyfallai Today we present Morpheus, a persistent enterprise simulation platform designed to make Continual Learning a reality. Morpheus is the world’s first real world Reinforcement Learning environment. Every Reinforcement Learning environment operates in the game world. Benchmarks like Atari, OpenAI Gym, MuJoCo, and Procgen are all small, game-like worlds that reset every few minutes. But the real world never resets. A business keeps running and evolving everyday. We tested how frontier LLMs would perform in realistic and dynamic business environments 🧬on Morpheus. The main conclusion was that LLMs are not continual learners. 🧵Here’s how we did it and what we learned: Your browser does not support the video tag. 🔗 View on Twitter 🔗 View Quoted Tweet 💬 5 🔄 0 ❤️ 2 👀 316 📊 4 ⚡