PapersSelected

Fixed-Point Training Speeds Up Looped Language Models

Towards Looped Models Done Right, Part II: Rethinking at Fixed Points

Why we picked it

The key insight is that near-fixed-point recurrent states make much of the compute spent on full backpropagation and full KV caches redundant, cutting costs across training, decoding and RL at once. Practitioners building efficient inference stacks or RL pipelines for recurrent models can apply KV sharing and distilled prefill directly. Compared with Huginn's fixed prior, the learned prior plus orthogonal injection improves perplexity at every scale tested up to 1.6B parameters.

A research paper analyses how looped language models behave as recurrent states approach fixed points, where the path to a state matters less.

Body

This property enables truncated backpropagation in training, terminal key-value (KV) sharing for decoding with almost no accuracy loss, a distilled student that prefills up to 1.

79x faster, and RL updates 2x faster than backpropagating through replayed trajectories.The authors introduce a learned depth prior with an entropy term and orthogonal input injection to improve fixed-point formation.

From 100M to 1.6B parameters, these changes lower perplexity at every scale relative to Huginn's prior and existing injection schemes.

At 1.6B, the learned prior with a 3x smaller KV cache matches the downstream average of fixed-depth training with the full cache.

Original · arXiv cs.LG

Towards Looped Models Done Right, Part II: Rethinking at Fixed Points

Every recurrence of a looped language model adds cost in training, decoding, prefill, and reinforcement learning (RL). The closer recurrent states get to fixed points, the less the path to them matters. This enables truncated backpropagation in training; terminal key-value (KV) sharing for decoding with almost no loss in accuracy; a distilled student that prefills up to 1.79x faster; and RL updates that compute gradients from saved rollout states, 2x faster than backpropagating through the replayed trajectory. We therefore improve the two components of training that shape these fixed points: the depth prior and input injection. Fixed-depth training breaks KV sharing, and Huginn's broad depth prior supports sharing but dilutes supervision at the target depth more than sharing requires; we learn the prior from prediction feedback, with an entropy term that keeps it broad. Existing injection schemes let the state's component along the input amplify or cancel the injection; we remove this component with orthogonal injection. From 100M to 1.6B parameters, the learned prior and orthogonal injection lower perplexity at every scale relative to Huginn's prior and existing injection schemes, respectively. At 1.6B, the learned prior with a 3x smaller KV cache matches the downstream average of fixed-depth training with the full cache.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Huginn, looped model, KV cache, recurrent neural networks.

Source attribution: curated from arXiv cs.LG (@Benhao Huang, Chufan Shi, Junlin Chen, Shicheng Wen, Zhengzhong Liu, Eric Xing, Xuezhe Ma).