arXiv 论文提出带风险认证的 LLM 守栏提示词优化框架 CPS 与 CRISP
Risk-certified prompt optimization for LLM-as-a-judge guardrails
安全团队常用系统提示词做 LLM 审查守栏,但普通提示词优化只追平均分。这篇论文的 CPS 和 CRISP 给漏检攻击率加了统计认证上限,四个数据集验证有效,做内容安全或 agent 守栏的值得看看。
arXiv 论文将 LLM-as-a-judge 守栏的系统提示词选择建模为约束优化问题:在将漏检攻击率控制在用户指定水平 α 之内的前提下最大化良性通过率,并提供有限样本统计认证。论文给出两种实现:CPS 从训练数据生成固定提示词池,用同时上置信界在留出评估集上认证候选,选出效用最高的合规提示词;CRISP 则把风险反馈融入自适应搜索,通过难例引导改写和可复用留出集接口向可行效用边界推进。
实验覆盖提示词注入、系统提示词泄露和敏感信息披露四类数据集,结果显示经风险认证选出的提示词在留出测试上稳定不超目标漏检水平,放宽风险额度后效用随之提升;当固定池搜索找不到可认证的高效用候选时,CRISP 更为有效。对依赖系统提示词做守栏控制的团队,这提供了区别于平均性能优化的安全约束路径。
Risk-certified prompt optimization for LLM-as-a-judge guardrails
LLM-as-a-judge guardrails are often controlled primarily through system prompts, yet standard prompt optimization methods target average performance rather than explicit safety constraints. This paper studies system-prompt selection as a constrained optimization problem: maximize benign pass rate while keeping the missed-attack rate below a user-specified level α with finite-sample statistical control. The framework combines LLM-based judging, automatic prompt generation, and distribution-free risk certification. This work instantiates it in two procedures: CPS and CRISP. CPS generates a fixed pool of prompts from training data, then certifies candidates on a held-out evaluation set using simultaneous upper confidence bounds and selects the certified prompt with the highest benign utility. CRISP instead integrates risk feedback into an adaptive search, using hard example-guided rewriting and a reusable-holdout interface to steer generation toward the feasible utility frontier. Experiments on four datasets covering prompt injection, system-prompt leakage, and sensitive-information disclosure show that risk-certified selection consistently respects the target missed-attack level on heldout tests, while utility improves as the permitted risk is relaxed. CRISP is especially useful when fixed-pool search fails to produce certifiable high-utility candidates.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include LLM-as-a-judge, prompt engineering, AI safety, prompt injection.
Source attribution: curated from arXiv cs.LG (@Hussein Jawad, Alexandre Landeau, Guy Fleury Iradukunda, Paul Beaumont, Théodore Rezel, Sofiane Schaack, Nicolas J-B. Brunel).