Microsoft Research and Georgia Tech release MANDATE agent failure taxonomy
The counterintuitive finding is that the failures users report most often, like agents burning compute without progress, are the ones researchers study least, leaving a blind spot for anyone deploying agents in production. Teams building or evaluating agent products can use the 23-category taxonomy as a checklist for user-facing risk testing and post-incident review.
Researchers from Microsoft Research and Georgia Tech analysed 163,487 Reddit posts about AI agents and surveyed 73 experienced agent users. They produced MANDATE, a taxonomy of 23 failure categories across seven groups, covering cases where an agent completes a task in a way the user did not expect. Users rated the most severe failures as the hardest to notice, and people who had personally experienced a failure spotted it more easily than those who only considered it possible. The most commonly reported failures, such as agents that keep running and spend compute without progress, have the thinnest research coverage.
Must-read paper from Microsoft Research and Georgia Tech.
This one has lots of great insights.
They used 163,487 Reddit posts about AI agents and surveyed 73 experienced agent users to map how delegated agent work goes wrong.
The result is MANDATE, a taxonomy of 23 failure categories in seven groups. Each describes an agent that finishes the job in a way the user didn't expect.
Users rated the most severe failures as the hardest to notice.
People who had already run into a failure found it easier to spot than people who only thought it was possible.
The most common failures are often the least studied.
Agents that keep running and spend time and compute without making progress are among the most commonly reported failures, and they have the thinnest research base.
https://t.co/UCgw7HZYfo
Chat with Paper: https://t.co/UDWpuOHsvu
