斯坦福新方法让长推理模型提速3倍，内存不再随思考长度增长。
斯坦福大学研究人员提出Prefix Sliding方法，通过丢弃中间token实现推理过程内存固定。该方法在无需训练的情况下，使现有模型运行速度提升3倍，同时保持全注意力性能。该方法支持超过10万token的RL rollout，解决了长推理过程中的内存消耗问题。
Banger paper from Stanford on efficient test-time scaling. If you run agents that think for a long ...
Banger paper from Stanford on efficient test-time scaling. If you run agents that think for a long time, this one is worth your time. (bookmark it) Long reasoning keeps the entire trace in memory through full attention. This means that the hardest problems, the ones that need the most thinking, are also the ones that cost the most to run. The authors measured what the middle of a reasoning trace is actually worth. Intermediate tokens steadily lose importance as the model keeps going. Their new approach, Prefix Sliding, drops those tokens. It keeps the prefix, which holds the instructions and the available tools, plus a window of the last few thousand tokens. Everything in between gets discarded during generation. Total memory stays capped no matter how long the model reasons. Without any training, this runs existing models 3x faster while matching full-attention performance, and it enables RL rollouts past 100,000 tokens. Paper: arxiv.org/abs/2608.26070 Chat with Paper: academy.dair.ai/papers/prefix-… 💬 7 🔄 3 ❤️ 29 👀 2506 📊 12 ⚡