Microsoft CASD prompt optimization beats GEPA using agent logs
The shift from search loops over small trajectory batches to holistic log analysis changes the economics of prompt optimization for anyone running agents in production. Teams can apply CASD without building validation sets or wiring up environment access, making it practical for real-world agent debugging. Its 22x cost advantage over validation-gated search makes it a concrete alternative to GEPA for budget-conscious projects.
A Microsoft paper introduces CASD, a prompt optimization method where a coding agent reads an agent's full set of trajectory logs, computes statistics, identifies recurring failure modes and writes the findings as rules into a single prompt.
Unlike search-based methods, CASD needs no environment access and no validation data.Across ALFWorld, tau2-bench retail and telecom, and Spreadsheet Bench-Verified, one pass improves the unoptimized baseline by 16.
6 points on average, versus 10.9 for GEPA and 5.
3 for SkillOpt.Each optimized prompt costs about $1.
60, more than 22x cheaper than validation-gated search.
Banger paper from Microsoft on prompt optimization.
(bookmark it)
The claim that a coding agent reading your logs beats GEPA at prompt optimization
The overall finding is that you want to give a coding agent your full set of agent logs and let it write the analysis code, instead of running a search loop over small batches of trajectories.
CASD has an off-the-shelf coding agent compute statistics over the whole trajectory corpus, find recurring failure modes, read representative episodes and write the findings as rules in one prompt. It needs no environment access and no validation data.
Across ALFWorld, tau2-bench retail and telecom, and Spreadsheet Bench-Verified, one pass improves the unoptimized baseline by 16.6 points on average. GEPA improves it by 10.9 and SkillOpt by 5.3.
Each optimized prompt costs about $1.60, more than 22x cheaper than validation-gated search.
Paper: https://t.co/9gp4KNGUTJ
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Microsoft, prompt engineering, agents, GEPA.
Source attribution: curated from DAIR.AI (@dair_ai). Cross-source corroboration: 2 independent sources have covered this development (including shao__meng), confirming industry-wide signal strength.
- shao__meng09-24 01:53Source
