TRACE 框架揭示音频语言模型证据排序缺陷：8 个模型中 6.3% 正确答案证据不一致
Right Answers, Inconsistent Evidence: Source-Level Counterfactual Auditing of Large Audio-Language Models
这篇论文不测模型答得对不对，而是删掉声源后看打分会不会乱，思路挺巧妙。8 个音频语言模型里都有这个问题，做语音识别或音频检索的开发者可以拿来给自己的模型做体检，代码和数据都开源了。
一篇发表于 arXiv（编号 2611.15463）的论文提出 TRACE 框架，用于审计大型音频语言模型（LALM）的证据一致性。核心发现是：模型答对'场景里有没有猫'这类问题，不代表它真正区分了目标声音的有与无。
TRACE 的方法是对混合音频做源级反事实删除：逐一移除被查询的声源和竞争声源，检查所有包含目标声音的视图得分是否都高于删除后的视图。在 8 个 LALM、3,456 个受控混合音频上测试，6.3% 的正确答案违反这一排序——目标不存在视图的得分反而高于某些目标存在视图，且这种冲突无法用统一阈值修复。只做目标独占得分检查会漏掉其中 42.5% 的不一致案例。
不一致率随信噪比变化明显：目标 SNR 从 +5 dB 降到 -5 dB 时，不一致率从 4.9% 升至 8.4%，即使在各信噪比下都答对的单元中同样存在，且经过响度控制和重新配对控制后依然成立。论文结论是准确率只能证明答案对，TRACE 才能审计模型是否把'有'排在'没有'之上。代码和数据已在 GitHub（yikise/TRACE）开源。
Right Answers, Inconsistent Evidence: Source-Level Counterfactual Auditing of Large Audio-Language Models
A large audio-language model (LALM) can answer "is a cat present?" about a mixture correctly, and its confidence can drop when the cat is removed, yet this does not show that it can tell the cat's presence from its absence. We introduce TRACE. It deletes the queried source and each rival in turn and asks whether every view still containing that sound outscores the view with it removed. Across 8 LALMs and 3,456 controlled mixtures, 6.3% of correct answers fail this ordering: the target-absent view scores at or above some target-present view, a conflict no shared threshold repairs. A target-only score check misses 42.5% of them. The share rises from 4.9% to 8.4% as target SNR falls from +5 to -5 dB, even on units correct at every level, and survives loudness and re-pairing controls. Accuracy certifies the answer; TRACE audits whether it orders presence above absence. Code and data: https://github.com/yikise/TRACE.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include TRACE, LALM, 音频语言模型, 模型评测.
Source attribution: curated from arXiv cs.AI (@Yi Zhang, Yi Li, Hongwei Du).