产品精选

DigitalOcean 推出 Signals 和 Simulations,捕捉智能体静默失败

精选理由

DigitalOcean 出了一对调试智能体的工具:Signals 负责从线上抓出静默失败的会话,Simulations 用模拟用户重放验证修复,生产故障自动变测试集。

智能体出问题时往往各项检查都正常:比如工具调用打错了数据库区域,返回 200 状态码,用户只拿到更慢更差的答案。DigitalOcean 推出两个新工具解决这个问题。Signals 用普通规则而非 AI 模型监控线上会话,捕捉用户重复提问、要求转人工、工具循环重试等异常信号。命中的会话交给 Simulations,用一个模拟用户对修复后的智能体重放整段对话,验证修复有效后再上线。相比过去靠 LLM 给每条对话打分或人工读记录,真实生产故障直接变成了测试集。

原文 · rohanpaul_ai

Agent failures often pass every health check. A tool call hits the wrong database region, returns 200, and the user gets a slower, worse answer. Nothing alerts.

DigitalOcean’s new tools flag agent sessions that fail silently, then replay them against a fix before release.

Because, the clue behind an agent bug is often sitting in something nobody can search.

Signals watches every live session with cheap rules, not an AI model, that look for human trouble signs such as a user repeating themselves, asking for a human, or a tool being retried in a loop, then passes those sessions to Simulations, which replays them with a fake user against your fixed agent to check the whole conversation works before the fix ships.

Until now teams had to pay an LLM to grade every conversation, read transcripts by hand, or invent test cases from imagination, while here the real failures from production automatically become the test set.

https://t.co/6Wkqd4oULX