Fixed-Point Training Speeds Up Looped Language Models
The key insight is that near-fixed-point recurrent states make much of the compute spent on full backpropagation and full KV caches redundant, cutting costs across training, decoding and RL at once. Practitioners building efficient inference stacks or RL pipelines for recurrent models can apply KV sharing and distilled prefill directly. Compared with Huginn's fixed prior, the learned prior plus orthogonal injection improves perplexity at every scale tested up to 1.6B parameters.