VAE 的后验坍塌是生成模型老难题，EAE 用隐式先验巧妙绕开，做生成模型或表示学习的开发者值得一看。
变分自编码器（VAE）普遍存在后验坍塌问题，即潜在变量被忽略。本文提出熵自编码器（EAE），仅以重构损失为显式目标，通过自由能最小化的编码器集成隐式生成潜在变量的先验。EAE 能学习非高斯、多模态的潜在分布，从而缓解后验坍塌，生成多样且与数据一致的样本。在反应扩散过程、MNIST 和 CelebA 数据集上，EAE 分别捕捉了低维动力学、隐式类别区分和面部层次结构。该框架为生成模型提供了一种新的训练范式。
Entropic Auto-Encoding via Implicit Free-Energy Minimization
Despite their ubiquity, variational autoencoders (VAEs) inherently suffer from posterior collapse, a failure mode in which latent variables are effectively ignored. This failure arises because explicit prior imposition drives optimization toward loss landscape regions corresponding to uninformative latent representations. Here, we introduce Entropic Autoencoders (EAEs), a framework in which reconstruction loss is the only explicit objective, and entropy generates the latent variables' prior implicitly through a free energy-minimizing ensemble of encoders. This ensemble biases learning toward high-volume regions of near-optimal solutions, while decoder updates direct the search trajectories toward informative latent representations. We demonstrate that EAEs mitigate posterior collapse by learning non-Gaussian, multimodal latent distributions that yield diverse, data-consistent generations and preserve different forms of underlying structure in the data. As a proof-of-concept, we show that an EAE captures a superposition of the known low-dimensional dynamics of a reaction-diffusion process. Then, we show that an EAE identifies implicit categorical distinctions in MNIST latent representations, and displays a hierarchical understanding of facial structure on the CelebA dataset, from an "all-human" face to individual-dependent features.