做长序列建模或量子机器学习的研究者值得关注——QLAM 用量子叠加解决了 SSM 记忆能力不足的问题,在标准基准上已跑赢 Transformer,建议点开看实现细节。
长序列建模中,Transformer 的二次复杂度限制了其扩展性,而状态空间模型(SSM)虽线性高效,但记忆机制偏简单,难以捕捉复杂全局交互。研究者提出量子长注意力记忆(QLAM),将隐藏状态表示为量子态,通过参数化量子电路实现非经典全局更新,既保留 SSM 的循环线性结构,又利用量子叠加丰富记忆表示。QLAM 隐式捕获全局依赖,并通过查询相关测量提取任务信息。在 sMNIST、sFashion-MNIST 和 sCIFAR-10 等序列图像分类任务上,QLAM 一致优于循环基线和 Transformer 模型。这项工作首次将量子系统的叠加特性引入状态序列建模,为长上下文 AI 提供了新思路。
QLAM: A Quantum Long-Attention Memory Approach to Long-Sequence Token Modeling
Modeling long-range dependencies in sequential data remains a central challenge in machine learning. Transformers address this challenge through attention mechanisms, but their quadratic complexity with respect to sequence length limits scalability to long contexts. State-space models (SSMs) provide an efficient alternative with linear-time computation by evolving a latent state through recurrent updates, but their memory is typically formed via additive or linear transitions, which can limit their ability to capture complex global interactions across tokens. In this work, we introduce one of the first studies to leverage the superposition property of quantum systems to enhance state-based sequence modeling. In particular, we propose Quantum Long-Attention Memory (QLAM), a hybrid quantum-classical memory mechanism that can be viewed as a quantum extension of state-space models. Instead of maintaining a classical latent state updated through additive dynamics, QLAM represents the hidden state as a quantum state whose amplitudes encode a superposition of historical information. The state evolves through parameterized quantum circuits conditioned on the input, enabling a non-classical, globally update mechanism. In this way, QLAM preserves the recurrent and linear-time structure of SSMs while fundamentally enriching the memory representation through quantum superposition. Unlike attention mechanisms that explicitly compute pairwise interactions, QLAM implicitly captures global dependencies through the evolution of the quantum state, and retrieves task-relevant information via query-dependent measurements. We evaluate QLAM on sequential variants of standard image classification benchmarks, including sMNIST, sFashion-MNIST, and sCIFAR-10, where images are flattened into token sequences. Across all tasks, QLAM consistently improves over recurrent baselines and transformer-based models.