新研究提出决策审计方法：模型执行成功不代表产品决策正确
The Delegation Blind Spot: Auditing Product Decisions from Agent Choices
这篇论文用 4,800 次请求证明模型跑得准不等于选对了产品方向，还给出不用调用模型的提取器，做 Agent 评估的可以看看。
arXiv 论文提出一种针对 Agent 决策的审计方法，将声明的观察通道与产品价值对比映射为兼容区间和见证人群。
冻结实验向两个固定版本的模型快照发出 4,800 次请求，36 个保守主区间全部未获解决。后续 2,400 次调用仅解决每模型九项对比中的三项，而确定性提取器在不调用模型的情况下解决了九项中的七项。
论文还提供带来源标签的决策回执和离线查看器，建议在收集更多遥测数据前先诊断决策为何无法确定。
The Delegation Blind Spot: Auditing Product Decisions from Agent Choices
Successful agent execution need not identify which future product improvement its user would value. We present a decision-specific audit that maps a declared observation channel and product-value contrast to compatible intervals and witness populations. Its foundations are established identification and decision theory; the contribution is an executable measurement workflow and a controlled study of its limits. A frozen experiment makes 4,800 requests to two pinned model snapshots on shared synthetic tasks. All 36 conservative primary intervals remain unresolved despite different execution accuracy. An exploratory 2,400-call follow-up records supplied preferences and resolves three of nine comparisons per model. A deterministic extractor resolves seven of nine without model calls or calibration observations, exposing unnecessary uncertainty introduced by model-generated reports. A further 14,400 controlled multinomial simulations distinguish structural ambiguity from weak identification and finite calibration precision. We propose a source-labeled decision receipt and provide an offline viewer for inspecting the audit. These results motivate preserving decision-relevant structured input and diagnosing why a decision is unresolved before collecting more telemetry. The study contains no human participants or real customer outcomes. Full proofs, raw model provenance, controlled experiments, and reproducible analyses accompany the report.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include 智能体, 决策审计, 论文, 评估方法.
Source attribution: curated from arXiv cs.LG (@Shivam Gupta).
