论文提出循环架构新训练方法,提升模型推理能力
Interesting paper to improve recurrent reasoning. Looped models are great because you get more reas...
朋友,有个新论文,用循环架构和新的训练方法,让模型推理能力更强了,比之前的循环模型效果更好。
这篇论文提出了一种循环架构的新训练方法,通过局部去噪目标让模型在推理时更新隐藏状态。该方法在五个推理基准测试中超越了之前的循环模型。训练时,噪声水平逐步降低并共享同一噪声样本,将每个更新步骤联系起来。
Interesting paper to improve recurrent reasoning. Looped models are great because you get more reas...
Interesting paper to improve recurrent reasoning. Looped models are great because you get more reasoning out of a model without adding parameters. So this work proposes a looped architecture with a new training method. The authors report wins over prior looped models on five of six reasoning benchmarks. More details from the paper: Looped models reason by updating a hidden state again and again at inference time. The hard part is training. Gradients usually flow through only the last one or two updates, so the early updates never learn to set up the later ones. Looped flows train the recurrence with local denoising objectives, the way flow models are trained. Noise levels decrease step by step and share the same noise sample, which ties each update to the next. At inference the model follows a probability flow. A finer time grid spends more compute, and different starting noise can produce different valid answers on tasks with more than one solution. Paper: arxiv.org/abs/2609.11801 Chat with Paper: academy.dair.ai/papers/thinkin… 💬 4 🔄 0 ❤️ 5 👀 930 📊 5 ⚡