SFResearch Study Tests Fragility of Memory-Based Self-Improving Agents
The takeaway is that agent self-improvement may be more fragile than reported scores suggest, because evaluation conditions differ sharply from production task streams. Teams building memory-based agents should run multi-seed evaluations, randomise task order, and enrich memory construction with explicit rubrics and feedback. Code and data are released openly, so practitioners can replicate the findings directly.
A study re-evaluated two established memory-based agent methods across WebArena, VisualWebArena, and SCUBA, running each experiment multiple times and shuffling task order. Results varied widely across runs, suggesting single-run reporting can overstate progress. Default benchmark orderings follow an easy-to-hard curriculum, while real-world requests arrive in any order. Adding rubrics, environment feedback, and clearer prompts to memory construction closed 31% of the performance drop under shuffled task order.
On the Fragility of Self-Improving Agents: Variance, Task Order, and Underspecification: https://t.co/uSh4hMOp6G
Memory-based agents learn from a stream of tasks and save lessons for future work. We re-evaluated two established methods across WebArena, VisualWebArena, and SCUBA, running each experiment multiple times and shuffling task order to reflect how requests arrive in practice.
What we learned:
📊 Results can vary widely across runs, so reporting multiple runs gives a clearer picture of real progress.
🔀 Default benchmark orderings follow an easy-to-hard curriculum, while real-world requests arrive in any order.
🧭 Clearer specification leads to more useful memories. Adding rubrics, environment feedback, and clearer prompts to memory construction closed 31% of the performance drop under shuffled task order.
The findings point toward stronger evaluation practices, memory validation, and interfaces that support human oversight. Code and data are available as open science.
#FutureOfAI #EnterpriseAI
