JEM复现研究：PC采样器与SGLD在CIFAR-10上实际无差异
Canonical Joint Energy-Based Model on CIFAR-10: failure modes and practical indistinguishability of Predictor-Corrector and SGLD samplers
这篇论文复现JEM，对比PC和SGLD采样器，结果在CIFAR-10上几乎没差别，AUROC差小于0.007，还暴露了训练发散问题。搞能量模型的人可以看看。
论文复现了无归一化层的WideResNet-28-10上的联合能量模型（JEM），两个独立运行达到92.88%测试精度和44.
46的buffer-FID，而原基准为92.9%和38.
40。研究记录了两种失败模式：后期训练中通过异常缓冲机制引发的发散，以及SVHN OOD检测行为的运行依赖性。
在约130个训练轮次中，Predictor-Corrector（PC）采样器替换SGLD并未带来任何方法级优势：所有十个检查点-OOD对上的AUROC绝对差低于0.007，FID差低于0.
5。分层种子-图像自助抽样给出宏平均AUROC差值的95%置信区间包含零，种子级等价检验无法建立正式等价。
Canonical Joint Energy-Based Model on CIFAR-10: failure modes and practical indistinguishability of Predictor-Corrector and SGLD samplers
Joint Energy-Based Models (JEM) unify classification and generation within a single network and support out-of-distribution (OOD) detection. Canonical JEM training relies on stochastic gradient Langevin dynamics (SGLD); a theoretically motivated alternative, the Predictor-Corrector (PC) sampler, has not previously undergone a systematic replication test on the canonical model. We reproduce canonical JEM on WideResNet-28-10 without normalisation layers on two independent runs and test whether PC retains its theoretical advantage without an annealed noise schedule, across three protocols: PC replacing SGLD throughout the roughly 130 training epochs; cold-start generation (FID); and refinement-style multi-OOD detection (AUROC). The reconstruction reaches 92.88% test accuracy and buffer-FID 44.46 (canonical: 92.9% and 38.40). We document two failure modes: catastrophic late-training divergence via the canonical outlier-buffer mechanism (both SGLD runs and, with the same signature, both PC runs), and run-dependent SVHN OOD-discrimination dynamics. No method-level advantage of PC over SGLD is observed on any protocol: at inference the absolute AUROC difference stays below 0.007 across all ten checkpoint-OOD pairs and the FID difference below 0.5; on the training protocol a hierarchical seed-by-image bootstrap gives a 95% confidence interval on the macro-averaged AUROC difference that contains zero, while a seed-level equivalence test with two runs per method cannot establish formal equivalence. The data are consistent both with equivalence and with a small directional effect. This practical indistinguishability is theoretically expected: under fixed noise the PC predictor step degenerates by construction, so its guarantees do not transfer to canonical JEM.
This report is categorized under AITOP's "Papers" desk. Key entities and technical topics under continuous tracking include JEM, CIFAR-10, SGLD, 能量模型.
Source attribution: curated from arXiv cs.LG (@Dmytro Knopov).
