预测市场从业者终于有了可落地的AI裁决方案——独立聚合投票比单模型更准,混合路由策略能平衡成本与精度,做预言机或去中心化应用的团队值得参考。
预测市场依赖可靠的裁决机制,但现有方案在自动化速度与人工准确性间难以平衡。该研究评估了多智能体LLM架构(独立聚合与协商共识)在1189个已解决预测市场问题上的表现,对比GPT-5 Nano、DeepSeek V3和Llama-3.3-70B单模型基线。独立聚合(置信度加权投票)以83.43%准确率胜出,比最佳单模型高1.01个百分点;而协商共识因错误传播导致准确率降至76%。模型间错误相关性(0.529-0.689)限制了集成方法的理论上限。研究提出混合AI-人类裁决系统:仅自动裁决一致高置信度问题,可在47%数据上达到97.87%准确率,其余由人工审查。
Design and Evaluation of Multi-Agent AI Oracle Systems for Prediction Market Resolution
Prediction markets aggregate collective intelligence to forecast uncertain events, but their utility depends on reliable outcome resolution. Existing oracle systems tradeoff fast but brittle automation against accurate but costly human arbitration. Single-LLM oracles achieve meaningful accuracy but inherit all failure modes of their underlying model with no self-correction mechanism. We evaluate whether multi-agent LLM architectures can improve oracle resolution accuracy over single-model baselines. We compare independent aggregation and deliberative consensus against single-LLM baselines (GPT-5 Nano, DeepSeek V3, and Llama-3.3-70B) on 1,189 resolved prediction market questions from KalshiBench. All agents share a common evidence layer through Exa, with retrieval filtered by publication date to isolate reasoning from retrieval quality. Independent aggregation with confidence-weighted voting achieves the highest accuracy at 83.43 percent, outperforming the best individual model by 1.01 percentage points. Deliberative consensus degrades accuracy to approximately 76 percent, below every single-model baseline, attributed to error propagation during debate where confidently wrong models flip correct ones. Error correlations across models (0.529-0.689) explain why aggregation gains fall short of the theoretical Condorcet ceiling, placing a fundamental limit on ensemble approaches. Many questions resist correction by any multi-agent architecture, motivating escalation to human arbitration. We propose routing criteria for hybrid AI-human oracle systems: auto-resolving only unanimous, high-confidence questions yields 97.87 percent accuracy on 47 percent of the dataset, with inter-agent disagreement flagging the remainder for human review.