文章背景与核心概要

在这篇文章中，作者 Sebastian Raschka 深入探讨了 Anthropic 为 Claude 引入的全新文本水印机制。不同于对大语言模型（LLM）架构进行昂贵且复杂的重新训练，该水印技术巧妙地介入了文本生成的采样阶段，利用密钥引导词元（token）选择过程中的随机数种子，从而在输出中嵌入一种不可见但可被验证的“数字签名”。

文章详细拆解了 LLM 从分词、获取对数几率（logits）、概率归一化到最终采样的全过程，并进一步剖析了类似 Google "SynthID-Text" 的“锦标赛采样”（tournament sampling）机制。这种方法极大地降低了水印检测的计算成本，无需重新运行庞大的底层模型即可高效验证文本来源。通过此次详尽的讲解，读者可以全面理解 AI 水印的技术原理、安全性以及对抗篡改的实际表现。

Claude 如何为 AI 生成的文本添加水印

In a recent Substack note, I discussed Anthropic’s new watermarking process for Claude. Due to the high level of interest and the complexity of the mechanism, I have expanded my explanation into a comprehensive lecture.

Originally planned as a 10-minute overview, this deep dive grew into a 48-minute presentation covering the technical "under-the-hood" mechanics of how LLMs generate text and how watermarking is integrated into that process.

• Watch the YouTube Lecture
• Download the Slide Deck (PDF)

Subscribe to the newsletter for more technical deep dives.

在最近的一篇 Substack 笔记中，我讨论了 Anthropic 针对 Claude 推出的全新水印机制。由于大家对此高度关注且该机制较为复杂，我将解释内容扩展成了一堂全面的技术讲座。

这场深度解析最初计划作为一个 10 分钟的概述，最终扩展成了一场 48 分钟的演示，涵盖了大语言模型（LLM）如何生成文本以及水印如何融入该过程的技术“底层”机制。

• 观看 YouTube 讲座
• 下载幻灯片 (PDF)

订阅新闻通讯以获取更多技术深度解析。

Summary

This lecture explains that watermarking is not a separate, expensive layer added to an LLM, but rather a subtle modification to the sampling stage of text generation. By using a secret key to influence the random seed during token selection, the model creates a detectable "signature" in the output. The lecture also covers the "tournament sampling" method used to make watermark detection computationally efficient without needing to rerun the original LLM.

总结

本次讲座解释了水印并非添加到 LLM 中的独立且昂贵的层，而是对文本生成采样阶段的微妙修改。通过使用密钥在词元选择过程中影响随机种子，模型在输出中创建了一个可检测的“签名”。讲座还涵盖了“锦标赛采样”方法，该方法使得水印检测在计算上高效，而无需重新运行原始 LLM。

Video Transcript

(Note: The following transcript has been edited for readability while preserving the flow of the lecture.)

视频文稿

（注：为保证可读性并保留讲座的流畅度，以下文稿经过了编辑。）

How Claude’s Text Watermarking Works

Anthropic recently announced they will watermark Claude’s text outputs. Many users are concerned about whether this degrades text quality or what the actual benefits are. Understanding the mechanism is key to evaluating these concerns.

Claude 的文本水印是如何工作的

Anthropic 最近宣布他们将为 Claude 的文本输出添加水印。许多用户担心这是否会降低文本质量，或者其实际好处是什么。理解这一机制是评估这些担忧的关键。

I believe in building things "from scratch" to understand them deeply. While this isn't a coding tutorial, understanding the sampling process of an LLM is essential to seeing how watermarking is applied.

我坚信通过“从头构建”事物来深刻理解它们。虽然这不是一个编码教程，但理解 LLM 的采样过程对于了解如何应用水印至关重要。

The goal of watermarking is to identify AI-generated text. It is invisible to the user, but allows the provider to verify the origin of the text using a secret key.

添加水印的目标是识别 AI 生成的文本。它对用户是不可见的，但允许提供商使用密钥来验证文本的来源。

How LLM Text Generation Works

Text generation involves converting input text into token IDs, passing them through the model, and obtaining a score distribution (logits) for the next token.

LLM 文本生成是如何工作的

文本生成涉及将输入文本转换为词元 ID（token IDs），将其输入模型，并获得下一个词元的得分分布（logits）。

When you ask, "The capital of Germany is," the model calculates probabilities for every token in its vocabulary.

当你问出“德国的首都是”时，模型会计算其词表中每个词元的概率。

The model generates a score distribution across the entire vocabulary. "Berlin" will have the highest score.

模型在整个词汇表上生成得分分布。“柏林 (Berlin)”将获得最高得分。

Tokenization is the first step, converting text into IDs that the model can process.

分词（Tokenization）是第一步，将文本转换为模型可以处理的 ID。

The LLM then outputs a score distribution for the next token.

随后，LLM 会输出下一个词元的得分分布。

Logit values represent raw scores. We can convert these into probabilities, but the core process is selecting the next token from this distribution.

Logit 值代表原始得分。我们可以将其转换为概率，但核心过程是从该分布中选择下一个词元。

The distribution spans the entire vocabulary. For a specific prompt, the model is highly confident in the correct token.

该分布跨越了整个词汇表。对于特定的提示词，模型对正确的词元具有高度的信心。

We sample a token from this distribution.

我们从这个分布中采样一个词元。

The token is detokenized back into text.

该词元通过反分词（detokenize）转换回文本。

This process repeats in a loop until the response is complete.

这个过程在循环中重复，直到响应生成完毕。

How Next-Token Sampling Works

Most LLMs don't use "greedy decoding" (always picking the highest score) because it leads to repetitive, memorized outputs. Instead, they sample from the distribution.

下一个词元采样是如何工作的

大多数 LLM 不使用“贪婪解码”（即总是选择最高得分），因为这会导致重复、死记硬背般的输出。相反，它们会从分布中进行采样。

We convert scores into probabilities using a softmax function.

我们使用 softmax 函数将得分转换为概率。

Once normalized, we sample using a random number generator.

一旦归一化，我们就使用随机数生成器进行采样。

Repeated sampling confirms that the model consistently selects the most probable token.

重复采样证实，模型始终如一地选择概率最高的词元。

From Sampling to Watermarking

Watermarking is essentially a way to influence this sampling process.

从采样到水印

水印本质上是一种影响该采样过程的方法。

In cases where multiple tokens are equally plausible (e.g., "overcast" vs "gray"), random sampling creates variation.

在多个词元同样合理的情况下（例如，“overcast”与“gray”），随机采样会产生变体。

Setting a random seed makes the sampling deterministic and reproducible.

设置随机种子（random seed）可使采样具有确定性和可复现性。

With a fixed seed, the model will consistently pick one specific token over another.

固定种子后，模型将始终如一地选择某个特定词元而不是另一个。

Changing the seed changes the deterministic output.

改变种子会改变确定性的输出。

Watermarking uses a secret key and the previous tokens to derive a random seed, effectively controlling the sampling process in a way that only the key-holder can verify.

水印利用密钥和先前的词元推导出一个随机种子，从而有效地控制采样过程，只有密钥持有者才能进行验证。

Without watermarking, there are many possible valid outputs.

如果没有水印，可能会有许多种合理且有效的输出。

A fixed random seed reproduces one of the plausible texts.

固定的随机种子会复现其中一种合理的文本。

With watermarking, the key controls which plausible text is generated.

引入水印后，密钥会控制生成哪种合理的文本。

This is applied at the sampling stage, meaning no retraining of the LLM is required.

这应用于采样阶段，这意味着不需要对 LLM 进行重新训练。

How Watermark Detection Works

Detection is only possible if you have the secret key.

水印检测是如何工作的

只有拥有密钥，才能够进行检测。

Watermarks can be defeated by editing the specific tokens that were chosen by the watermarking process.

可以通过编辑由水印过程选定的特定词元来消除水印。

Because the user doesn't know which tokens are watermarked, they must guess, which may lead to awkward or lower-quality text.

由于用户不知道哪些词元带有水印，他们必须去猜测，这可能会导致文本显得生硬或质量下降。

SynthID Text and Tournament Sampling

To make detection cheaper, Anthropic uses techniques like those found in Google's "SynthID-Text."

SynthID Text 与锦标赛采样

为了降低检测成本，Anthropic 使用了类似于 Google "SynthID-Text" 中采用的技术。

Tournament sampling allows for efficient scoring without needing to rerun the LLM.

锦标赛采样（Tournament sampling）允许进行高效的评分，而无需重新运行 LLM。

We evaluate candidate tokens based on their probability and their "signature" from random watermarking functions.

我们根据候选词元的概率及其来自随机水印函数的“签名”来评估它们。

The tournament structure pits tokens against each other based on these signatures.

锦标赛结构根据这些签名让词元相互竞争。

This tournament process ensures the final selected token carries the watermark signature.

这一锦标赛过程确保了最终选定的词元带有水印签名。

Each token gets a bit signature from the watermark functions.

每个词元从水印函数中获得一个位签名（bit signature）。

These signatures are the basis for the tournament.

这些签名是锦标赛的基础。

Tokens are paired up, and the winner is determined by the watermark function.

词元被两两配对，由水印函数决定胜者。

The tournament proceeds until one token remains.

锦标赛一直进行，直到只剩下一个词元。

This process is repeated for every token generated.

生成的每个词元都会重复此过程。

The main benefit is the ease of detection.

主要的好处是检测变得更加容易。

To detect, we simply re-run the watermark functions on the text and check the scores.

为了进行检测，我们只需在文本上重新运行水印函数并检查得分即可。

We average the scores to get a final metric.

我们对得分取平均值以获得最终指标。

Comparing the average score against a threshold determines if the text is watermarked.

将平均得分与阈值进行比较，即可确定文本是否带有水印。

This thresholding is the final step in detection.

这种阈值判定是检测的最后一步。

Tournament sampling is a clever way to make detection computationally feasible.

锦标赛采样是一种巧妙的方法，使检测在计算上切实可行。

The key difference is the use of the watermarking key during the sampling stage.

关键区别在于在采样阶段使用了水印密钥。

Detection requires the secret key and the specific watermark functions.

检测需要密钥和特定的水印函数。

Removing the watermark is difficult because you don't know which positions are watermarked.

移除水印很困难，因为你不知道哪些位置带有水印。

This may lead to "edited" AI text that is lower quality.

这可能会导致“编辑后的”AI 文本质量变低。

Users might simply use local, non-watermarked models to edit the text, creating a more complex pipeline.

用户可能会简单地使用本地无水印模型来编辑文本，从而创建一个更复杂的处理管道。

I hope this behind-the-scenes look was useful!

希望这次幕后探秘对大家有所帮助！

PS: If you like more explainers in this style, I have accumulated over 300 videos over the years, which you can find on my YouTube channel.

附言：如果你喜欢这种风格的更多解析文章，我在过去几年中积累了 300 多个视频，你可以在我的 YouTube 频道上找到它们。