这篇论文提出了一个名为ICON分解的新方法，用于更准确地评估深度神经网络中各个概念的重要性，对于模型审计和可解释性研究具有重要意义。与现有的基线方法相比，它能够提供更可靠的解释，对于理解模型的决策过程非常有帮助。
深度神经网络常利用训练数据中的虚假关联，称为捷径学习。基于概念的可解释性方法通过测试网络层中是否可以解码出如患者性别或扫描设置等概念来筛选捷径。由于每个概念都是单独评估的，这些方法可能会将概念之间的相关性误认为是模型使用它们的证据。我们引入了ICON分解，它通过考虑所有其他概念和结果，量化了每个概念解释了层中多少方差。在具有已知真实标签的合成数据上，ICON比七种替代基线方法更准确地恢复概念重要性。在皮肤病变和脑成像模型上，它隔离了模型真正依赖的概念，量化了任何提供概念都无法解释的表示，并产生了稀疏的解释，我们通过重新训练和分布外测试来验证这些解释。
ICON Decomposition: Multivariate Concept-Level Explanations of Deep Representations for Model Auditing
Deep neural networks often exploit spurious associations in their training data, a failure known as shortcut learning. Concept-based explainability methods screen for shortcuts by testing whether concepts such as a patient's sex or scanner settings can be decoded from a network layer. Because each concept is evaluated in isolation, these methods can mistake correlations between concepts as evidence that the model uses them. We introduce ICON decomposition, which instead quantifies how much of a layer's variance each concept explains after accounting for all other concepts and the outcome. On synthetic data with known ground truth, ICON recovers concept importance more accurately than seven alternative baseline methods. On skin-lesion and brain-imaging models, it isolates the concepts on which a model genuinely relies, quantifies the representation unexplained by any of the supplied concepts, and yields sparse explanations that we validate by retraining and out-of-distribution testing.