Papers

arXiv 论文:基于 AUC 的临床预测模型公平性指标如何正确使用

Fairness Metrics based on the AUC in Clinical Predictive AI

Why we picked it

做医疗预测模型公平性评估的可以看看,作者逐个分析了八种 AUC 公平性指标,哪些能用、哪些有坑都讲清楚了,还给了 EuroSCORE II 的模拟例子。

一篇 arXiv 论文检视了临床预测模型中八种基于 AUC 的公平性指标,包括 within-group AUC parity、xAUC parity、gap-to-overall AUC 等。作者指出 AUC 是不可折叠指标,子群 AUC 与总体 AUC 不能直接比较。论文以 EuroSCORE II 住院死亡率预测模型做模拟,展示各指标行为差异。最终建议报告包含总体 AUC 与所有组内、跨组 AUC 成分的 AUC 矩阵,并推荐以 within-group AUC parity 为主、xAUC parity 为辅。gap-to-overall AUC、equity-scaled AUC 和 minimum-based 指标因比较对象不当,不建议使用。

Original · arXiv cs.AI

Fairness Metrics based on the AUC in Clinical Predictive AI

Fairness evaluation of clinical prediction models for binary outcomes often involves fairness metrics that compare the area under the receiver operating characteristic curve (AUC, or c-statistic) across subgroups. However, the AUC has been shown to be a non-collapsible metric, meaning that its subgroup specific and overall values are not directly comparable as often implicitly assumed. This raises important questions about how existing AUC-based fairness metrics should be used and interpreted. We examine eight AUC-based fairness metrics proposed in the literature, including: within-group AUC parity; cross-group AUC (xAUC) parity; gap-to-overall AUC; equity-scaled AUC; minimum-based metrics; $\mathrm{xAUC}^1$; $\mathrm{xAUC}^0$; and $\mathrm{xAUC}^1$ or $\mathrm{xAUC}^0$ parity. We characterise their estimands, relate them to the decomposition of overall AUC (into within- and cross-group components), and critically appraise them. We then illustrate their behaviour using simulations based on the EuroSCORE II, a clinical prediction model for in-hospital mortality. We recommend reporting an AUC matrix, containing the overall AUC and all within- and cross-group AUC components, alongside population characteristics. When fairness metrics are needed, we recommend reporting the within-group AUC parity, with cross-group AUC parity providing supplementary information but requiring caution due to its potential conflict with calibration. Metrics involving $\mathrm{xAUC}^1$ and $\mathrm{xAUC}^0$ have merit, summarising within- and cross-group AUCs into single values, but their interpretation is non-trivial, and they should be used with caution. Gap-to-overall AUC, equity-scaled AUC, and minimum-based metrics compare quantities that should not be compared and it is unclear how to operationalise them, making them not fit for purpose and generally not recommended.