Papers

研究提出人机分类决策模型:相关性忽视与信任如何影响AI辅助效果

AI-Assisted Classification under Correlation Neglect and Trust

Why we picked it

这篇论文解释了为什么更聪明的 AI 有时反而帮倒忙:人会重复计算重叠证据。还用 ChatBench 数据拆出了两种损失各占多少,量化很扎实。

一篇 arXiv 论文建模了人类在使用 AI 建议做二元决策时的行为,核心参数是对证据重叠的感知与对 AI 的信任。分析显示,存在相关性忽视时,更强的 AI 反而可能让人准确率下降,因为共享证据被重复计算。在 ChatBench 上,AI 辅助使准确率提升 21 个百分点,但受试者只给 AI 信息约四分之一的贝叶斯理性权重。实验进一步把损失拆为相关性忽视成本与信任不足成本:校准信任可让 GPT-4o 场景挽回约 2.6 个点,对 Llama-3.1-8b 则几乎无效。

Original · arXiv cs.AI

AI-Assisted Classification under Correlation Neglect and Trust

When does an AI assistant improve human decision-making, and when does it hurt? We argue that the answer depends not just on AI accuracy but also on how much its evidence overlaps with the human's and how the human combines the two. A worker makes a binary decision by observing her own signal and receiving a recommendation from an AI. We develop a human behavioral model with two primitives: her awareness of the overlap and her trust in the AI. Our analysis gives a closed-form phase diagram over AI capability and overlap, with regions of complementarity, impairment, and automation. A worker who neglects the overlap (correlation neglect) can be made less accurate by a more capable AI, double-counting the shared evidence. Trust reshapes the regions - undertrust may compensate for correlation neglect, so that the trust level maximizing accuracy depends on the deployment parameters. Empirically, we identify the two model parameters by pairing the same workers with different AIs, and break the accuracy loss into a correlation-neglect cost and an undertrust cost, one tied to reducing overlap on the AI side, the other to calibrating trust on the human side. On ChatBench, AI assistance raises accuracy by 21 points, yet workers apply only about a quarter of the Bayes-rational weight to AI information; GPT-4o falls in the automation region and Llama-3.1-8b stays at the complementarity boundary; and calibrating trust recovers about 2.6 points for GPT-4o but almost nothing for Llama-3.1-8b.