SA-CUA：为屏幕操作智能体构建防间接提示注入的信任边界
Towards Safe Autonomy for Computer Use
教屏幕操作智能体防住提示注入的论文：SA-CUA 攻击成功率压到 0，效用还比 CaMeL-CUA 高。
针对 Computer Use Agent（CUA）直接在屏幕上视觉操作、无法区分第一方界面与攻击者内容的问题，论文提出双 LLM 框架 SA-CUA。该框架从无障碍树（accessibility tree）推导保守的确定性规则，在特权 LLM 的截图中划分可信区域，将可信与不可信内容隔离开。人类只需在任务开始时确认任务描述，运行期间由经可信渠道验证的背书通道在不可信域与可信域之间传递数据。测试结果显示 SA-CUA 的攻击成功率为 0，同时效用明显高于基线 CaMeL-CUA。
Towards Safe Autonomy for Computer Use
The safety of LLM-based agents currently rests on alignment training and safety classifiers, neither of which can rule out that a sophisticated indirect prompt injection (IPI) goes through and causes significant harm. Recent work therefore builds agents with information-flow guarantees that rely on structured tool interfaces to interact with the environment. Computer Use Agents (CUAs) act visually on the computer screen, which does not provide a boundary between trusted first-party interfaces and untrusted, potentially attacker-authored content. Actions are equally ambiguous: a click may land on an application toolbar or a button controlled by an attacker. Treating the user as the source of trust and confirming each sensitive decision produces confirmation fatigue, with the risk of missing harmful actions. We introduce SA-CUA, a dual-LLM framework that generates a reliable trust boundary for CUAs by deriving conservative deterministic rules from the accessibility tree for unmasking trusted regions from the privileged LLM's screenshot, keeping trusted and untrusted content separated. Human interaction is only required for proper task specification in the beginning. During the task a robust endorsement channel allows values to pass from the untrusted to the trusted domain by verifying information via trusted channels. This allows the agent to autonomously handle tasks involving untrusted information. SA-CUA maintains much higher utility than CaMeL-CUA while keeping the attack success rate at zero.