文本水印原理详解：KGW方法、破解与工程难点

这篇推文关于文本水印原理讲的很清楚。 生成文本的时候，模型每要输出下一个 token，就拿前面已经生成的所有词加上一个密钥，算出一个哈希值。这个哈希值会把词表里的词随机分成两组，比方说绿组和红组。然...

这篇推文关于文本水印原理讲的很清楚。 生成文本的时候，模型每要输出下一个 token，就拿前面已经生成的所有词加上一个密钥，算出一个哈希值。这个哈希值会把词表里的词随机分成两组，比方说绿组和红组。然后模型在选下一个词时，悄悄提高绿组词被选中的概率。最终输出的文本看起来完全正常，但统计上会偏向绿组词。 检测的时候反过来。拿着同一个密钥，对文本里的每个词重新算一遍哈希，还原出当时的绿组和红组。如果整篇文本中绿组词的占比显著超过 50%，就判定这段文本带有水印。 几个常见疑问： 1. 改写能破解水印吗？ 大部分情况下可以。因为你一改词，前面的 token 序列变了，哈希也跟着变，绿组红组就对不上了。但也有改进方案用统计模型而不是确定性函数来生成哈希，让水印对改写有一定的适应能力。 2. 水印会降低文本质量吗？ 理论上会，因为模型不再完全自由地选最优词，而是被约束在绿组里选。但 Google 在两万条文本的人类反馈实验中称，大多数人感知不到质量下降，因为表达同一个意思的词有很多种组合。不过在短文本或者高度确定的输出上（比如1+1=2），水印确实会失效。 3. 密钥会被逆向破解吗？ 理论上需要指数级数量的样本才能还原绿组红组的划分，所以直接暴力破解不现实。但如果检测器公开了，攻击者可以通过不断试探来摸索出规律。 工程难点 第一，大模型是流式输出文本的，一个词一个词往外蹦，没法像学术方案那样等整段写完再调整。 第二，如果密钥泄露，水印就废了，所以需要多组密钥轮换。 第三，代码类输出不能随意换词，否则代码会出错，水印只能加在注释、变量名这类可替换的部分。 如果 Anthropic 公开检测器，等于给了攻击者一个免费的练习靶，水印会被快速找到绕过策略。 如果像 Google 的 SynthID 那样把检测器留在内部，更安全一些，但学术界已经有论文展示了不需要检测器也能零样本破解水印的方法。 密集改写（同义词替换加句法重组）可以有效消除水印，免费的改写工具就能绕过 Google 的 SynthID。 前沿实验室对此心里有数，也没指望水印能拦住刻意绕过的人。大多数普通用户不会专门去除水印，而欧盟监管方也只需要足够好就行。 水印更多是一种应对欧盟监管的合规动作。 Alex Cui @alexcdot Claude's watermark probably doesn't work how you think. As the CTO of GPTZero, I'll explain how Anthropic, Google and OpenAI are building text watermarking in this brief explainer and whether it can be defeated. Almost all forms of watermarking that are fast and cheap enough for a frontier lab have the same formula, following the KGW method: In generation: 1. Let's say you've generated n tokens so far. Take those n tokens + a secret key to generate a random hash 2. Use that hash to randomly reweight the probabilities for the n+1 token, and then sample from that new distribution. In the simple case, you could split 50% of all English words into a green or red set based on your hash, and boost the probability of words in the green set. For watermark detection: 1. For each token, see if it was in the green or red set. 2. To do this, recreate the hash based on the secret key and the text preceding the current token. Then, recreate the green and red set of words. 3. Once you've checked all the words in the text, if the next token is selected disproportionally from the green set more than 50% of the time, you claim the text has the watermark. I can tell you want to ask the following: 1) Isn't it easy to mess up the hash if you paraphrase the text? The answer is mostly yes, however, you can use a statistical model to get your hash instead of a deterministic function (SIR, Adaptive Watermark). Since the entire watermark is probabilistic, this is fine. 2) Doesn't this make the text much worse? The answer is yes, it does - Yes, it does – but for most people, it's imperceptible (Google claims in human feedback study with 20,000 texts), since there are exponentially many ways to write the same paragraph. DiPmark does something more sophisticated to avoid shifting the text distribution on average. Of course, watermarks fail on short text or highly predictable texts like "2+2=4". 3) Shouldn't it be easy to figure out the green and red sets? The answer is no. You would need an exponentially large number of samples from the watermarker to reconstruct those sets exactly, but it's a risk if the detector is open to the wild (Watermark Stealing) Still, there are couple challenges that a frontier lab needs to overcome: 1. Their watermark needs to work token-by-token because they are streaming their text to users. Many watermark methods plan sentences or paragraphs at a time, or change the text after its entirely written, in order to make their watermark robust to paraphrasers, and a frontier lab cannot afford to do this yet (SemStamp, PostMark) 2. If the secret key leaks, the watermark is busted. To avoid a large blast damage from this, you need to have a couple secret keys in rotation. 3. There are some texts, like code, that cannot be arbitrarily changed, otherwise the code will break. In those cases, the watermark needs to selectively change words in parts of the text that can tolerate synonyms (i.e. like variable naming) - see SWEET, EWD, Invisible Entropy. 4. They will need to educate their users on how to deal with false positives and false negatives of a detector, which is a big challenge (one we put a lot of effort into) So, how do I see this playing out in the next 6 months? 1. If Anthropic releases the watermark detector publically, I think they defeat their own watermark. People find reliable watermark removal strategies by testing against Anthropic (AI detectors like GPTZero have an advantage here because they can train against these adversaries once they become popular). 2. If they keep the detector private to the government, like Google has done, it's "safer". However, there are some papers showing trained approaches that work robustly to zero-shot break watermarks without any data, simply because they try to write the text just like a human (Zhang et al. 2024, Watermarks in the Sand). Also, making your detector makes it battle-tested and stronger long-term (my experience). 3. In my testing, the watermarks don't survive intense paraphrasing (especially if you combine word choice and syntax attacks), or human text substitution (rewrite your AI text by plagiarizing human authors). The free paraphrasers I've tried have quickly bypassed Google Deepmind's SynthId for what it's worth. 4. All-in-all, frontier labs are likely okay with this because they expect most users to not attack the watermark, and also because they + European regulators likely don't care past a certain point - its good enough. 5. Overall, I think users of frontier LLMs will not really care about this, because 1) they don't realize watermarks are there, 2) EU will force everyone to conform, 3) this seems more like regulatory hoop-jumping than an earnest effort from frontier labs to expose LLM use Lastly, people's first concern shouldn't be watermarking, it should be AI detectors! If you're posting, "its not X, its Y!!", I don't think the watermark is going to make a difference :) 🔗 View Quoted Tweet 💬 1 🔄 0 ❤️ 1 👀 661 📊 1 ⚡