这篇论文讲了一种真实的预训练数据投毒方式——通过论坛评论注入恶意内容，还给出了分析工具HalfLife，搞AI安全的值得看。
该论文提出通过公共讨论界面（如论坛评论）向大语言模型预训练数据注入恶意内容的攻击方法。作者引入HalfLife分析工具，用于评估网络爬虫数据中是否包含对抗性内容。实验表明，基于维基百科等传统数据源的投毒假设不适用于真实的大规模异构预训练语料库。研究强调第三方网页内容可能成为攻击语言模型预训练的潜在向量。
Pretraining Data Can Be Poisoned through Computational Propaganda
Poisoning pretraining data can introduce harmful behaviors to LMs that are difficult to detect and mitigate. Prior work on poisoning pretraining data has largely exploited established data sources such as Wikipedia, which do not represent the large scale and heterogeneity typical of pretraining corpora, and has ignored the interaction between poisoned data and data curation pipelines. We demonstrate that poisoning attacks on pretraining data are feasible beyond this limited setting through an existing web-scale content injection mechanism: public discussion interfaces. Additionally, to measure whether malicious content is included after web crawling and data curation, we introduce HalfLife, a novel analysis for estimating adversarial content inclusion in web-crawl based LM training data. We use HalfLife to explore the feasibility of poisoning pretraining corpora at web scale through open discussion interfaces. Our analysis demonstrates the importance of estimating whether poison injections are included in pretraining data, and establishes third-party webpage content as a possible vector for attacking language model pretraining.