Agentic Harnesses:为机器人自主性引入LLM驱动的验证层

Agentic Harnesses: LLM-Driven Verification Layers for Robot Autonomy

精选理由

这篇论文给机器人规划加了个AI裁判,能拦下不安全或不合规的动作,对抗攻击遏制率97%,做机器人安全的值得看看。

AI 摘要

该论文提出在机器人规划与执行之间加入LLM驱动的验证层,评估动作的可行性。采用LLM-as-a-Judge集成,结合多模型链式推理与专家输出合成,类似专家混合与自一致性方法。该层作为中间件,在计划到达MCP服务器前进行门控,支持批准、拒绝或升级人工审查。系统在批准/升级/拒绝类别上达到近85%的精确度,对对抗性攻击的遏制率达97%。错误主要出现在升级边界,接受与拒绝任务间的误差可忽略。

原文 · arXiv cs.AI

Agentic Harnesses: LLM-Driven Verification Layers for Robot Autonomy

Advances in advanced artificial intelligence tools have sparked research in robot autonomy, but the development of such systems has largely focused on execution rather than verifying the feasibility actions planning models propose. Like general-purpose LLMs, robotics planning models carry risks: biased toward user-specified goals, they may suggest actions misaligned with scientific ethics, they may be unsafe due to an inability to "remember" prior safety risks, or they may be vulnerable to adversarial attacks on the autonomy ecosystem. We propose a LLM-driven verification layer between planning and execution to evaluate action permissibility. Our LLM-as-a-Judge ensemble combines chain-of-thought reasoning across models and synthesizes those expert judge outputs, mirroring a combination of a mixture of experts and self-consistency approach. This layer serves as middleware, gating plans from the server's planning module before they reach the MCP server and therefore the robot's low-level controls: plans are approved, rejected for reformulation, or escalated for human review. With this system, we achieve near 85% precision across accept/escalate/reject categories 97% containment of adversarial attacks, with negligible errors between accepting and rejecting tasks, and errors mostly manifesting at the escalate boundary.