这篇论文提出了AutoBackSwap,只用几百张图片的标注就能让模型不依赖背景,效果比之前的方法都好,做图像分类的值得一看。
深度神经网络分类器常因依赖训练数据中的虚假背景特征而导致泛化失败。新方法AutoBackSwap通过辅助网络分离前景和背景,并用填充方式生成完整背景,然后将不同前景与填充背景组合来增强训练数据。仅需几百个样本的补丁标注即可训练辅助网络,并自动增强整个数据集。在多个具有虚假背景的图像分类任务上,AutoBackSwap始终优于先前方法,即使在训练数据中没有任何打破虚假相关性的样本时也有效。
Automated Background Swapping for Robustness against Spurious Backgrounds
Classifiers based on Deep Neural Networks exhibit strong performance across domains, yet can fail catastrophically if they rely on spurious correlations, i.e., features that are predictive of the target label in the training data but are not causally linked and thus fail to generalize. For the vision domain, many such spurious correlations manifest themselves within the background of the image, where only the foreground is predictive of the class label. In this paper, we introduce Automated Background Swapping (AutoBackSwap) to reduce the reliance of classifiers on such spurious backgrounds. AutoBackSwap uses a secondary network to disentangle the foreground and background, followed by infilling to synthesize complete backgrounds, and finally combines different foregrounds and inpainted backgrounds to augment the training data. We find that patch-wise labeling of just a few hundred samples suffices to train the secondary network and automatically augment the full training dataset on challenging image classification tasks. In contrast to many previous methods, AutoBackSwap proves very effective even if there is not a single sample in the training data breaking the spurious correlation. Across a range of image classification tasks with spurious backgrounds, AutoBackSwap consistently outperforms prior methods.