Safe Personal RSI 框架让个人健康助手安全地自我修改指令
搞智能体安全的别错过:Kicker 助手能自己改指令,但每条改动都要过 29 个用例加注入攻击探针的门槛,GPT-5.6 Luna 修复后违规率直接归零。做自定义 Agent 或个人助理的开发者可以参考这套门控思路。
researchers 团队为 Kicker 个人健康助手构建了 Safe Personal RSI 框架,让智能体只能修改自身指令文本,模型权重和运行时代码保持固定。该框架的一次更新门控在 21 条自提议修改中拒绝了 20 条高风险改动。
技术机制上,每条提议的更新都要在 29 个可执行用例上测试,覆盖邮件、日历、健康和市场交易场景;另设独立探针检测授权复用、记忆污染和工具输出注入攻击。失败用例会重复执行,并通过临时回滚定位是哪一条编辑导致失败,若没有任何版本通过全部测试,就保留当前指令不变。
实测结果显示,一次针对 GPT-5.6 Luna 的修复将攻击违规率从 10.7% 降到 0%,安全任务完成率从 89.7% 提升到 100%;在更强攻击下,Qwen2.5-7B 的修复方案迁移到 Qwen3-14B 后,违规率从 69.0% 降到 17.9%。
Self-improving agents are one of the most discussed ideas in AI right now.
(bookmark it)
This update gate for a personal health assistant rejected 20 of 21 risky self-proposed updates.
Researchers built Safe Personal RSI for the Kicker assistant. The agent can only change its own instructions. Model weights and runtime code stay fixed.
Each proposed update is tested against the current instructions on 29 executable email, calendar, health, and marketplace cases. Separate probes test authorization reuse, memory, and tool-output injection. Failing cases are repeated, and changes are temporarily undone to find which edit caused the failure. If no version passes, the current instructions stay.
One admitted repair for GPT-5.6 Luna cut attack violations from 10.7% to 0% and raised safe task completion from 89.7% to 100%. Under stronger attacks, a repair moved from Qwen2.5-7B to Qwen3-14B, cutting violations from 69.0% to 17.9%.
https://t.co/iJB46F8sig
Chat with Paper: https://t.co/hnH5BSgbCv
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Safe Personal RSI, Kicker, GPT-5.6, Qwen3.
Source attribution: curated from DAIR.AI (@dair_ai).