Matei Zaharia刚透露Databricks实测：用Pi harness成本省一半，效果和Opus、GPT一样好；GLM 5.2这个开源模型也追上来了。
Databricks联合创始人Matei Zaharia在推文中分享了内部编码智能体基准测试结果。Pi harness在Opus和GPT模型上获得相同成功率，但成本降低2倍。GLM 5.2被描述为开源编码智能体性能的一次重大进步。测试表明包括开源在内的许多模型已达到真正具有竞争力的水平。
Highly-recommended read. Aligns with what I see in my own harness: > Pi harness got the same succe...
Highly-recommended read. Aligns with what I see in my own harness: > Pi harness got the same success rate as harnesses from the LLM vendors with Opus and GPT, but at 2x less cost > GLM 5.2 was a major step forward in open-source coding agent performance The harness bloat/rot is real! Matei Zaharia @matei_zaharia We benchmarked coding agents on our own internal tasks at Databricks and learned a lot! There are many surprising opportunities to lower cost and increase quality, and many models including open source ones are truly competitive now. 🧵 🔗 View Quoted Tweet 💬 3 🔄 1 ❤️ 9 👀 965 📊 4 ⚡