论文精选

DeepSeek V3.2智能体架构在ARC-AGI-1上达67%准确率

Cost-Effective Agent Harnesses for Abstract Reasoning and Generalization on ARC-AGI-1

精选理由

开源模型DeepSeek V3.2套上智能体架构,在ARC推理任务上以不到1美元成本拿下67%准确率,比零样本强了50多个点,硬件开销很低。

AI 摘要

该研究使用开源模型DeepSeek V3.2(非推理模式)在严格预算下构建了智能体框架。提出的Explorer-Definer Pipeline将模式发现与程序合成分离,在ARC-AGI-1 400任务测试集上达到57.50% pass@2,每任务成本0.25美元。Reflective Orchestrator扩展了该流程,通过自主探索新变换,将准确率提升至67.25% pass@2,每任务成本0.62美元。该架构无需基准特定微调或大量测试时计算,将15.50%的零样本基线提升了约52个百分点。

原文 · arXiv: DeepSeek

Cost-Effective Agent Harnesses for Abstract Reasoning and Generalization on ARC-AGI-1

Recent progress on ARC-AGI-1 from disclosed architectures has come broadly from two regimes: heavy test-time compute over frontier models (evolutionary search, exhaustive sampling, extended chain-of-thought), or benchmark-specific training in which small models are fine-tuned on ARC data, often with task-specialized architectures. We study a third regime: an open-weight model in non-thinking mode (DeepSeek V3.2) under a strict budget, with no ARC-specific fine-tuning. We study what is recoverable through architecture alone, building agentic harnesses that decompose pattern-discovery and program-synthesis stages explicitly. First, we introduce an Explorer-Definer Pipeline that separates pattern discovery from executable transformation synthesis, implemented as a two-stage agent pipeline. Next, we present the Reflective Orchestrator, which augments the pipeline with autonomous exploration of new transformations when previous hypotheses fail on training pairs. On the ARC-AGI-1 public 400-task evaluation set, the pipeline reaches 57.50% pass@2 at \$0.25 per task, and the orchestrator reaches 67.25% pass@2 at \$0.62 per task. Together these architectures lift a 15.50% one-shot baseline by ~52 points without benchmark-specific training or heavy test-time compute. Furthermore, the orchestrator-driven lift tests a falsifiable diagnostic the pipeline produces; unbiased pass@k analysis suggests the pipeline is generation-bound, not selection-bound (selection via training-pair accuracy captures ~95% of the candidate ceiling) and predicts that significant improvement requires broader generation, not better ranking. The orchestrator implements this prediction via adaptive re-exploration and confirms it (unbiased pass@1 lift +9.81 pp, matching selection-mediated pass@2 lift). An additional pipeline ablation identifies its think tool as a significant component, with removal reducing pass@2 by 5.75 pp.