想评估智能体能不能从经验里进步?GDPevo用企业真实流程搭了120个任务,最高涨16.44个点,开源可跑。
GDPevo是一个基于GDP相关企业工作流的智能体自我进化评估基准,覆盖CRM、ERP、金融、医疗、法律和数据工作流。其规则杂交机制把每个流程拆成原子业务规则,训练与测试任务重组规则,使测试收益可归因于训练经验。V1包含12组120个任务,每组5个训练任务和5个保留测试任务;流水线可在两天内扩展到V2的24组240个任务。研究者在四种监督类型下评估了四个智能体,自我进化使保留测试准确率最高提升16.44个百分点。但最优进化智能体仍远低于91.6%的全知上限,说明当前智能体的自我进化能力尚未充分实现。
GDPevo: Evaluating Agent Self-Evolution on Real Business Tasks
Agent self-evolution updates an agent's persistent state from prior experience and reuses it to solve related tasks more effectively. Evaluating self-evolution is difficult: existing benchmarks provide limited coverage of economically valuable task domains, do not always design training and test tasks such that test-time gains can be attributed to training experience, and remain vulnerable to data contamination. We present GDPevo, an evolution-native benchmark grounded in GDP-related enterprise workflows, together with the fully automated data pipeline that generates it. Its core mechanism, rule hybridization, decomposes each enterprise workflow into atomic business rules, distributes subsets of these rules across training tasks, and recombines them in held-out test tasks so that test-time gains are attributable. GDPevo spans CRM, ERP, finance, healthcare, legal, and data-centric workflows. Its V1 release contains 120 tasks in 12 groups, with five training and five held-out test tasks per group. Full automation enables the pipeline to expand the suite to 240 tasks in 24 groups (V2) within two days, providing a practical response to contamination. Using GDPevo, we evaluate four agents, each comprising a harness and a model, under four supervision types. Self-evolution consistently improves held-out accuracy by up to 16.44 percentage points. But the best evolved agents remain far below the fully informed oracle ceiling of 91.6%, indicating that the self-evolution ability of current agents remains far from fully realized. We publicly release the pipeline, benchmark, and full evaluation results at https://github.com/Prism-Shadow/GDPevo.