研究提出 DCPS:简单生成-选择即可匹敌复杂提示词优化器
Simplicity Goes Far: Auditing Prompt Optimizers with Demonstration-Conditioned Prompt Search
很多人以为提示词优化必须上 PPO 或进化搜索,这篇论文用实测数据说明:简单生成-选择就能在低 32% 成本下打平 GEPA 和 PPO。做 Agent 或 DSPy 管线的人应该先跑这个免费对照再决定要不要上复杂优化器。
一篇新论文提出 Demonstration-Conditioned Prompt Search(DCPS),将 Automatic Prompt Engineer(APE)流程改造成机制归因控制:冻结生成器、基于随机采样的示例生成候选提示词、在验证集上打分并保留最优,不涉及任何学习更新、反思或进化搜索。
作者用 DCPS 审计了两类主流优化器。在 StablePrompt 中,DCPS 替换 Proximal Policy Optimization(PPO)后,分类准确率差距在 0.
5 个百分点以内,成本约为原方法的三分之一;在基于 DSPy 的复合 AI 系统中,DCPS 在 GPT-4.
1-mini 上以 55% 更少的 rollout、平均 32% 更低的 API 成本,匹配了 GEPA 的 macro accuracy。审计揭示出任务依赖的模式:生成-选择机制在结构化与指令跟随任务上能复现大部分复杂优化器的性能,而在部分开放式与重推理任务上,额外机制仍保有优势。
作者建议基准测试应将提示词优化器与生成-选择对照比较,并报告 rollout 数量、token 用量与美元成本。
Simplicity Goes Far: Auditing Prompt Optimizers with Demonstration-Conditioned Prompt Search
Automatic prompt optimization is growing more elaborate. Recent methods stack policy-gradient updates, trace reflection, and evolutionary search. These methods share a generate-and-select step; without controlling for it, reported gains may conflate the contributions of optimizer mechanisms and larger sampling budgets. We isolate this step as Demonstration-Conditioned Prompt Search (DCPS), repurposing the Automatic Prompt Engineer (APE) recipe as a mechanism-attribution control: it freezes the generator, conditions it on randomly sampled demonstrations, scores candidates on validation data, and keeps the best---no learned update, reflection, or evolution. Across two optimizer families, DCPS replaces Proximal Policy Optimization (PPO) in StablePrompt, matching classification accuracy within 0.5 pp at roughly one-third the measured cost; in DSPy-based compound AI systems, under the reported budgets, it matches GEPA's macro accuracy on GPT-4.1-mini while using 55% fewer rollouts at 32% lower measured API cost on average. The audit reveals a task-dependent pattern: proposal and selection recover much of heavier optimizers' performance on structured and instruction-following tasks, while additional mechanisms retain advantages in some open-ended and reasoning-heavy settings. We recommend benchmarking prompt optimizers against generate-and-select controls and reporting rollout counts, token usage, and dollar costs alongside accuracy. Code and artifacts: https://github.com/bhc6/DCPS.
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include GEPA, DSPy, APE, prompt engineering.
Source attribution: curated from arXiv cs.AI (@Bohe Chen).