通过沉默获胜:LLM计划评估中的删除非单调性、自主利用与类型状态门控

Win by Silence: Deletion Non-Monotonicity, Autonomous Exploitation, and Typed-State Gating in LLM Plan Evaluation

精选理由

这篇论文深挖了一个漏洞:计划评估器可能因为计划含糊就给高分。他们用数学证明并在26条路线上验证了。搞LLM安全或评估的值得一读。

AI 摘要

一篇论文研究了LLM计划评估器的漏洞:计划可通过省略必要工作提高得分。在26条路线组成的队列中,所有57次合法删除都匹配了分析恒等式,且每条路线至少有一次得分提升的删除。评分优化器在21/26条路线中发现了未被覆盖的结构。GATE机制在26/26条静默路线上拒绝得分释放,0次诚实暂停;之后47/54次修订转为覆盖结构,严格覆盖改进从1/26升至13/26。自适应编译器感知合著者揭示了注册表-来源边界:义务通道规避在v1/v1.5条件下保持6/6,而delta索引成本下限将诚实路线从6/6降至3/6,静默可融资性从5/6降至0/6。

原文 · arXiv cs.AI

Win by Silence: Deletion Non-Monotonicity, Autonomous Exploitation, and Typed-State Gating in LLM Plan Evaluation

Plan evaluators can reward a strategic plan for becoming less explicit. This paper studies that failure in a staged expected-value scorer for LLM-generated venture routes. Proposition 1 gives the score change from deleting an interior transition while retargeting its predecessor and retaining downstream value: Delta_k = (prod_{i<k} p_i)[c_k + (1 - p_k)R_{k+1}]. On a frozen 26-route cohort, all 57 admissible deletions matched the analytic identity and threshold sign, and every route had at least one score-improving deletion. A score-seeking optimizer, allowed to restructure routes but not told the exploit mechanism, found baseline-beating uncovered structures in 21/26 routes. GATE refused score release for 26/26 silenced routes with 0/26 honest suspensions; after refusal, 47/54 next revisions repaired to a covered structure, and strict covered improvement rose from 1/26 to 13/26. An adaptive compiler-aware co-author exposed the registry-provenance boundary: obligation-channel evasions remained 6/6 across all four v1/v1.5 conditions, while delta-indexed cost floors reduced beat-honest routes from 6/6 to 3/6 and fundability-by-silence from 5/6 to 0/6 without establishing semantic completeness. If a plan scores better only because it omits necessary work, the plan did not improve; the evaluation created an omission incentive. PCSC detects and neutralizes post-hoc omission splices over model-mediated typed-state records. In the cooperative setting tested, GATE acts as a deterministic search-shaping constraint, not merely a post-hoc filter. It does not verify the semantic completeness or real-world quality of arbitrary LLM-generated strategies.

通过沉默获胜:LLM计划评估中的删除非单调性、自主利用与类型状态门控 · AI 热点