这篇论文提出了ZID,一个更全面的生成模型评估方法,它不仅检测偏离,还能诊断问题,对于想要深入了解生成模型评估的人来说是个好资源。
FID和KID常用于评估生成模型,但FID的总结可能遗漏分布差异,且仅凭标量差距无法准确测试采样变化。本文提出ZID,结合标准化位置和分散敏感的臂和高斯核,提供三个相关输出:排名偏离程度指数、分布平等测试的排列p值和有符号分散读数。ZID在控制实验中检测到广泛的偏离,其得分与相应扫描的严重程度增加一致,包括FID平坦或反转的情况。
Generative models are commonly ranked by Fréchet Inception Distance (FID) and Kernel Inception Distance (KID), yet FID's first-two-moment summary can miss distributional differences, and a reported scalar gap alone is not a calibrated test against sampling variation. FID's moment restriction has concrete consequences: on ImageNet, visually unrecognizable images optimized only to match the reference Inception mean and covariance obtain FID $24.7$ versus $58.6$ for held-out real images (lower is better). Moreover, FID and KID are scalar discrepancies that are unchanged when the two samples are exchanged and therefore do not encode the direction of a dispersion change: under-dispersion, as can occur in mode collapse, versus over-dispersion. We introduce \textbf{ZID} (\emph{Z-resolved Integrated Diagnostic}), which combines six standardized location- and dispersion-sensitive arms from a rank graph (RISE) and Gaussian kernels (GPK at two bandwidths). Rather than asking one scalar to serve incompatible roles, ZID reports three linked outputs: an index for ranking departure magnitude, a permutation $p$-value for testing distributional equality, and a signed dispersion readout for diagnosis. In controlled experiments, ZID detects a broad range of departures, and its score tracks increasing severity along the corresponding sweeps, including cases in which FID is flat or reversed. On DiT-XL/2 and SiT-XL/2 guidance sweeps, ZID detects departure from real data, and its signed readout labels the high-guidance diversity collapse as under-dispersion.