Jerry Liu 提出评测驱动开发：先定义 eval 再爬坡优化
LlamaIndex 的 Jerry Liu 说了个新打法：别手写 agent 流程，先写个 eval 然后对着它爬坡优化就行，挺值得试试。
LlamaIndex 创始人 Jerry Liu 在 X 上提出 eval 驱动开发的思路：多数任务可以先定义一个 eval，再围绕它做 hillclimbing 优化，而不必直接编写确定性的 agentic workflow。他认为数据供应商的核心工作是为各类经济活动定义 eval，让 frontier 模型能覆盖更多任务。开发者要做的是定义要解决的问题以及如何衡量好坏，复杂流程仍需显式 workflow builder。
eval driven development Jerry Liu @jerryjliu0 These days, you can pretty much solve any task by defining an eval and hillclimbing over it, instead of directly defining the deterministic/agentic workflow to solve it. The data provider companies' entire job is to define evals for all economic activity to make the frontier models capable of doing anything. Your job then becomes pointing frontier intelligence in the right direction - defining what to solve, and how to measure what good looks like. Agent application interfaces will evolve to capture this. The most complex processes will still need some sort of explicit workflow builder interface, but most tasks can be compressed into goals and eval instructions. 🔗 View Quoted Tweet 💬 4 🔄 1 ❤️ 15 👀 1503 📊 5 ⚡
- Jerry Liu10-09 05:58原文
