LangChain的Viv分享了一套改进智能体的实战思路，核心是用生产数据挖Evals，还给了Harness-PostTrain的开源模型配方。
LangChain的Viv在AI Engineer World Fair上分享持续改进智能体的方法论。他认为智能体会遵循Evals中编码的行为，因此高质量Evals来自大规模挖掘生产数据。他还提出了用开源模型搭配Harness与PostTrain的“三明治”配方来定制智能体系统。演讲强调不存在通用模型或通用Harness，针对具体任务优化模型与框架组合效果更好。
Improving agents is a data mining problem ⬇️
Improving agents is a data mining problem ⬇️ Viv @Vtrivedy10 the @aiDotEngineer World Fair always one of the best events every year to talk to builders at the frontier of Research, Agents, Evals, Systems, etc A few weeks ago I gave a talk on - Continually Improving Agents - building Agents to understand data from other Agents - & a walkthrough of some of our latest work on data agents & post-training experiments some fun takes: - Every Continual Learning company will be an Observability & Eval company (and vice versa) - Environments & Evals are the currency of agent improvement. Agents are literally following the behaviors encoded in Evals. The best way to make good evals is mining Production data at scale - A good recipe to own your intelligence is using a Harness Eng - PostTrain - Harness sandwich with open models - Model-Harness-Task fit! There is no universal model or universal harness. You can always build a better agent system by optimizing the model and harness for a given task if your team is looking to understand your data at scale, build environment/evals, or just improve your agents - reach out, hmu would love to work with you! 🚀 youtu.be/CvRngaQZQ3Y?is… 0 🔗 View Quoted Tweet 💬 1 🔄 1 ❤️ 2 👀 1491 📊 2 ⚡