这篇论文解决了超人类AI系统的监督难题，做AI安全和对齐的研究者可以直接参考其理论保证和实验验证。
论文提出校准集体监督（CCO）方法，通过聚合多个辅助评分函数形成惩罚项，衡量AI行为对保守基线的偏离。CCO受可达到效用保留启发，实现集体保守主义：当监督者认为行为无异议时，高效用行为仍被选中，仅在担忧累积时被覆盖。该方法利用共形决策理论在线校准保守程度，确保不良结果低于用户指定阈值，且无需分布假设。在修改版SWE-bench上，较弱监督者成功约束了对抗性更强的智能体；在MACHIAVELLI环境中，CCO在保持奖励的同时大幅减少伦理违规。
Calibrating Conservatism for Scalable Oversight
Agentic AI systems capable of autonomous planning and extended environmental interaction pose a fundamental control problem: how can humans maintain meaningful oversight of systems that may exceed their own capabilities? Existing approaches to scalable oversight rely on complex assumptions, remain largely heuristic, or lack practical methods for sequential settings with statistical guarantees. We introduce Calibrated Collective Oversight (CCO), which aggregates diverse auxiliary scoring functions into a penalty measuring deviation from a conservative baseline. Inspired by Attainable Utility Preservation, CCO enables collective conservatism: actions face a penalty proportional to overseer concern, so high-utility actions are still selected when overseers find them unobjectionable and overridden only when concern accumulates. CCO calibrates this conservatism online using Conformal Decision Theory, ensuring that undesirable outcomes remain below a user-specified target threshold with finite-time bounds and no distributional assumptions. On a modified version of SWE-bench, weaker overseers successfully constrain an adversarially misaligned stronger agent; on MACHIAVELLI, CCO substantially reduces ethical violations while preserving reward. In both settings, empirical violation rates closely match the specified targets, as predicted by the theory.