Marginal Matching:面向 Flow Matching 的训练式预测-校正框架
Correcting Flows with Marginal Matching
做生成模型的人可以看看这篇:作者给 Flow Matching 补上了扩散模型早已成熟的预测-校正玩法,思路是把前向 ODE 的样本往反向 ODE 的边缘分布上校正。在 CIFAR-10 和 ImageNet-64 上有效,额外计算开销小,追 ODE 生成管线的团队用得上。
一篇 arXiv 论文提出针对 Flow Matching 模型的训练式预测-校正框架,核心思路命名为 Marginal Matching。Flow Matching 属于基于 ODE 的生成模型,通过将简单源分布逐步变形为目标分布来生成样本,此前在计算效率与生成保真度之间的平衡上缺乏有效手段,而预测-校正方案在扩散模型中已被证明有效,但在 Flow Matching 中研究不足。
该方法利用 ODE 易于反向求解的特性,在前向 ODE 的特定时间步上,将样本校正到与反向 ODE 匹配的边缘分布,并证明该方案可缩小数据分布与学习到的分布之间距离的上界,据此给出两种校正函数实现。在 CIFAR-10 和 ImageNet-64 基准上,该方法提升了样本质量,且计算时间开销很小。
Correcting Flows with Marginal Matching
Flow matching models, ODE-based generative models, generate samples by gradually morphing a simple source distribution into a target distribution. However, these models still face challenges in optimally balancing computational efficiency with generation fidelity. While predictor-corrector schemes have been proven to help address this issue in diffusion models effectively, their application to flow matching models remains under-explored. In this work we present a novel training-based predictor-corrector framework for flow matching, which exploits the fact that the ODE can be easily reversed. Our key idea, termed Marginal Matching, is to correct samples at a particular time step in the forward ODE towards a matching marginal distribution of the backward ODE. This scheme proves to reduce a bound on the distance between the data distribution and the learned distribution, motivating two different implementations for the correction function. We show that our proposed method improves sample quality on CIFAR-10 and ImageNet-64, with minimal overhead in computation time.
This report is categorized under AITOP's "Papers" desk. Key entities and technical topics under continuous tracking include Flow Matching, diffusion models, ImageNet-64.
Source attribution: curated from arXiv cs.LG (@Meital Bojan, Nadav Bojan Sellam, Ev Zisselman, Aviv Tamar).