解耦是机器学习长期难题，这篇用全息简化表示把离散符号结构引入神经网络，做无监督学习的团队值得关注——它既保持了可微分性，又比连续表示更抗噪，理论分析也扎实。
该论文提出了一种基于全息简化表示（HRR）的无监督解耦学习方法，将解耦视为符号结构而非连续表示。通过HRR的“解绑”操作，模型能分离数据中的变化因子，并在潜在遍历和解耦指标上达到与基线相当的性能。理论分析证明解绑操作能产生近似独立的符号-值对，并给出了每个槽位的容量界限。与标准自编码器不同，该方法的潜在单元是向量求和而非标量维度，且对噪声更鲁棒。这项工作为神经解耦提供了新的符号化视角和理论支撑。
Disentanglement with Holographic Reduced Representations
Disentanglement, the separation of factors of variation in data using neural networks, remains a long-standing challenge in machine learning. Prior work has addressed this problem with variational autoencoders and generative adversarial networks that incorporate ideas from variational inference and information-theoretic constraints. In contrast to methods that rely on continuous representations, we propose a design that treats disentangled representations as symbolic structures, motivated by the compositional relationships among the concepts that make up samples from a distribution. However, learning discrete symbolic structures with neural networks while maintaining differentiability is difficult and often requires complex architectures. To address this, we introduce an unsupervised learning algorithm that uses holographic reduced representations (HRR) for neural disentanglement. We show that the HRR unbinding operation provides an inductive bias for separating factors and yields competitive results against baselines, as measured by latent traversals and disentanglement metrics. We complement these empirical findings with an information-theoretic analysis of the HRR unbinding channel. We prove that unbinding induces approximately independent symbol-value pairs and derive a per-slot capacity bound that quantifies how many distinct symbolic concepts can be reliably encoded, giving a quantitative account of the inductive bias toward disentanglement. The resulting representations differ from standard autoencoder-based models, in that their latent units are vectors that are summed together, rather than scalar dimensions of a low-dimensional latent vector. We show that this HRR representation is more robust to noise than other disentangled representations and maintains reconstruction quality across a range of SNRs.