做序列建模或基础模型预训练的团队，这篇论文帮你厘清了 xLSTM 相比 Mamba 等架构的实际优势，看完可以直接指导模型选型。
该论文系统比较了三种主流子二次架构（xLSTM、Mamba-2、Gated DeltaNet）在复杂任务上的表现，包括代码模型预训练、大模型蒸馏和时序基础模型预训练。结果显示，xLSTM 在所有任务中综合性能最优，其优势源于更灵活稳定的门控机制带来的状态追踪与记忆累积能力。研究通过统一公式分析和合成任务验证，揭示了 xLSTM 在长程依赖建模上的核心机制。
On Subquadratic Architectures: From Applications to Principles
Transformers dominate modern sequence modeling, but their quadratic attention incurs substantial computational cost. Subquadratic architectures offer a scalable alternative. However, it remains unclear which designs yield the most effective sequence models. We compare three leading approaches: xLSTM, Mamba-2, and Gated DeltaNet. We evaluate these models on tasks with complex dependencies: (1) code-model pre-training, (2) distillation of code models from large language models, and (3) pre-training of time-series foundation models. Across these settings, xLSTM delivers the strongest overall performance. To explain xLSTM's advantage, we present a unified formulation and analyze the underlying architectural mechanisms, focusing on state tracking and memory dynamics. Our results show that xLSTM enables more flexible and stable memory correction via its gating scheme. We corroborate these findings on controlled synthetic length-generalization tasks. Overall, our findings indicate that xLSTM's gains on complex tasks stem from robust state tracking and accumulation.