Meta paper reports ML engineering agents running 1,097 search experiments
The non-obvious takeaway is that harness design and objective calibration, not model strength, drove the jump in acceptance rate from 15.6% to 29.6%. Practitioners building agent-driven experiment loops can copy the three mitigations directly: parallel runs, online-calibrated objectives, and disciplined context engineering. Engineering leaders weighing agent adoption in ML pipelines get a rare production-scale, A/B-tested reference point instead of a demo.