VAE训练中常见的恒定崩溃问题终于有了可量化的检测和预防手段,做生成模型或自编码器研究的团队可以直接用这个证书方法监控训练过程,避免模型失效。
该研究针对变分自编码器(VAE)中的精确恒定崩溃问题——即编码器均值变得与输入无关——提出了一种可预先设计、训练中可监控、训练后可认证的解决方案。通过保持标准高斯先验,并为潜在均值附加一个固定的单纯形见证头,研究者定义了一个教师-学生对齐损失,其精确常数预测基线等于教师信息。当对齐损失低于该基线时,潜在均值不可能出现输入无关的恒定崩溃。该方法还提供了封闭形式的逆变换,可将任意全支撑教师后验嵌入潜在空间,并解释了何时对齐损失可以很小。这项工作将恒定崩溃从训练后的病理现象转化为可设计和可认证的问题。
A Simplex Witness Certificate for Constant Collapse in Variational Autoencoders
This note studies exact constant collapse in variational autoencoders, where the encoder mean becomes independent of the input. The goal is to make this specific failure mode pre-designable, monitorable during training, and certifiable after training. The prior is kept as the standard Gaussian. Given a fixed teacher posterior, we attach to the latent mean a fixed simplex witness head. The resulting teacher-student alignment loss has an exact constant-predictor baseline equal to the teacher information. If the alignment loss is below this baseline, the latent mean cannot be input-independent constant collapsed. The simplex witness also has a closed-form inverse. Any full-support teacher posterior can be represented by embedding its centered log-odds into the latent space. This gives an explicit latent energy cost and explains when the alignment loss can be made small. A computable view gap handles the case where teacher targets are computed from a different view. Thus exact constant collapse is converted from an after-the-fact training pathology into a design-and-certificate problem.