研究人员发布LongPIBench基准，测试了长上下文场景下的提示注入防御效果，发现现有防御存在显著漏洞。
LongPIBench是首个针对长上下文的提示注入攻击基准，涵盖论文评审、简历筛选、代码审查和邮件摘要4个场景。该基准包含合成数据集和真实数据集，上下文长度从数千到数万token不等。评估结果显示，即使在长上下文场景下，简单的启发式提示注入攻击仍能取得高成功率，并经常绕过最先进的防御措施。
LongPIBench: A Long-Context Benchmark for Prompt Injection
Prompt injection attacks pose a serious security risk to large language models in real-world applications. However, existing prompt injection benchmarks primarily focus on short-context inputs, leaving the attacks and defenses in long-context settings largely unexplored. This gap leads to a substantial overestimation of the effectiveness of current defenses. In this paper, we bridge the gap by introducing LongPIBench, a long-context benchmark for prompt injection covering 4 realistic application scenarios: paper peer review, resume screening, code review, and email summary. For each scenario, we construct a synthetic dataset and a real-world dataset, with context lengths ranging from thousands to tens of thousands of tokens. The evaluation results on LongPIBench reveal significant vulnerabilities of prompt injection defenses under long-context settings: even simple heuristic prompt injection attacks achieve high success rates and frequently bypass state-of-the-art defenses. We hope LongPIBench can serve as a practical benchmark for systematically evaluating prompt injection defenses in realistic long-context scenarios.