Shadowgraph
TrendingAbout
Sign in

ToForkDrankXs/Text-WaterMark-PaperList

This repo is to record some text watermark papers.

1

51 commits

updated Aug 31, 2026

See the code

README

Text-WaterMark-PaperList

Text watermark is an important research direction for the misuse of LLM-generated text. This repo will have the followings:

1.BackGround

1.1 Application

  • Deep Fake Detection: Detect whether the target text is generated by a language model;
  • Deep Fake Attribution: Determine which model/user generated the target text;
  • IP Protection: Protect valuable text and model

1.2 Classification

1.2.1 Bit Count

Distinguished based on the amount of information that can be embedded in the watermark.

  • Zero-Bit: Only determine whether to add watermark, no additional information. This makes the watermark only detect, no attribution effect.
  • Multi-Bit: Embed and extract multi-bit information, you can freely add the content you want to embed, such as time, user ID, etc. Multi-Bit watermark is necessary for deep fake attribution.

1.2.2 Embedding Phase

Text watermark research existed in the last century, at which time it was only possible to add watermarks to already written text, and was classified as Post-hoc. However, with the development of language models, some new stages of text watermark emerges.

  • Backdoor/A-Priori: By employing specific methods to utilize watermark information to form a toxic dataset, fine-tuning or training a language model, modify the model weights directly can result in a language model with watermark. The introduction of watermark information occurs before the language model generates text.
  • Generating-Process: The introduction of watermark information will interfere with the generation process of the language model, resulting in the language model producing text with embedded watermarks. The addition of such watermarks does not require modification of the language model, but it necessitates access to the complete language model.
  • Post-Hoc: By altering the text to introduce watermark information, the watermarked text is constructed. This kind of method does not require the original generative language model; it only requires the text generated by the model.

1.2.3 Generating-Process

This category represents a further subdivision within Category Embedding Phase→Generating-Process.

  • Token-Level: Watermark information is introduced when the language model generates a token each time.
  • Sentence-Level: The embedding of watermark information is used when the language model generates each sentence. This requires a semi-controlled sampling process when generating: 1.No constraint is imposed until a sentence is generated; 2.Multiple candidates must be obtained using beam search during generation; 3.When generated, it must be generated sentence by sentence.

1.2.4 Token-Level

This category represents a further subdivision within Category Embedding Phase→Generating-Process→Token-Level.

During each step of token generation, the language model undergoes the following processing: Get Logits on the vocabulary $\rightarrow$ Get Probability Distribution(Weight) over the vocabulary $\rightarrow$ Sample from the probability distribution

  • Logits Bias: The watermark information is converted into a logits distribution on the vocabulary, which is added to the logits distribution generated by the original language model to interfere with the generation phase.
  • Reweighting: The watermark information will guide reweighting probability distribution. This typically involves scrambling the vocabulary and selecting a re-weighting interval.
  • Sampling: Watermark information is embedded by influencing the generation results through sampling, which prohibits the commonly used decoding methods and restricts the decoding approach.

1.2.5 Different Scenarios

  • Completion: Normal Generation. Given a prompt and complete it. Most watermarks are configured for this type of scenario.
  • Condition: Conditional text generation. Such tasks are typically in the form of question-and-answer (QA) and summarization tasks.
  • Code: Code generation. Generating executable code.

1.3 Necessary Features

The addition of watermarks often involves a trade-off among multiple aspects.

  • Detectability: Detectability refers to the capability of a watermarking method to distinguish between watermarked text and non-watermarked text. Metric: Typically, it is a binary classification metric, and in the case of multiple bits, there are corresponding multi-bit comparison metrics.
    • TPR/TNR/FPR/FNR
    • AUROC
    • Accuracy
    • TPR@FPR=X%
    • Bit Accuracy/Bit Error Rate(Multi-Bit)
  • Invisibility/Text Quality: Invisibility refers to the impact of the watermark on the quality of the generated text. Metric: Typically, the text quality metrics of the original generated text and the watermarked text are compared.
    • Perplexity(PPL)
    • BLEU
    • ROUGE(-n)
    • BERTScore
    • Entailment Score(ES)
    • Sentence Similarity
    • Human Evaluate
    • Ent-31
    • Rep-32
  • Imperceptibility: Imperceptibility indicates the impact of watermark addition on the overall token distribution, focusing more on the statistical differences between watermarked and non-watermarked texts, which requires a large amount of corresponding text for overall estimation. Metric: Imperceptibility-related metrics often involve token-level statistics for the overall generated text. It assesses whether there are differences in the token distribution between watermarked texts and non-watermarked texts.
    • Word Frequency
  • Robustness: Robustness refers to the ability of watermarked text to still be recognized as watermarked after undergoing watermark removal attacks. Metric: Robustness-related metrics are represented by comparing the changes in detectability indicators caused by attacks.
  • Usability: Usability refers to the additional time and memory consumption caused by the addition of watermarks. These costs must be within an acceptable range for the user. Metric: Usability metrics typically consist of corresponding values for watermarking time and memory usage.
    • Time Cost
    • Memory Cost

1.4 Attack Function

1.4.1 Word Level

The watermark is removed by modifying the word.

  • Insert: Insert words randomly according to proportion.
  • Delete: Delete words randomly according to proportion.
  • Exchange: Swap the positions of two words in a sentence.
  • Replace: Similar substitutions are usually chosen based on the semantics of a particular word.
    • Synonym: Choose word substitutes according to the calculated thesaurus or the trained word vector.
    • Context: According to the MLM task of Masked Language Model, candidate words are determined by context.
  • Emoji: The model is induced to insert emojis according to certain rules.
  • Exception Word Removal: ONION, In the inference stage, GPT-2 pre-trained model is used to prevent the activation of backdoors by detecting and removing abnormal words such as "cf" in test samples.
  • HELM Perturbation: Deliberately use word variations to remove watermarks.
    • Structure: do not --> don't
    • Case: Large --> large
    • Spell/Ghost-Word: sun --> sunn; sun --> san

1.4.2 Sentence Level

The watermark is removed by modifying the sentence.

  • Human-Modify: Humans manually modify it directly.
  • Paraphrase: Use language models to rewrite sentences. Model: DIPPER;pegasus_paraphrase...
    • Bigram-Paraphrase3: When overwriting, several candidates are generated and the candidate with the least overlap of tokens is selected.
  • Back-Translate: Translate into another language, translate back.
  • Re-Watermark: Re-add watermark when overwriting.

1.4.3 Spoof Attack

The method of cracking the watermark is to try to forge the watermark text in a certain way.

2.Watermark Paper

About Key Word:

3.Survey

4.Attack

In this part, the core of the paper is the research of the watermarking attack method, and sometimes the corresponding solution is proposed.

5.Improvement for Watermarking

6.Evaluation & Benchmark

7.Finding Research

8.Other Related Work

Reference

Footnotes

  1. Generating informative and diverse conversational responses via adversarial information maximization. ↩

  2. Neural text generation with unlikelihood training ↩

  3. SemStamp: A Semantic Watermark with Paraphrastic Robustness for Text Generation ↩

Contributors

DrankXs

51 commits

Similar repos

plll4zzx/Awesome-LLM-Watermark

A collection list for Large Language Model (LLM) Watermark

70

hzy312/Awesome-LLM-Watermark

UP-TO-DATE LLM Watermark paper. 🔥🔥🔥

380

Xianjun-Yang/Awesome_papers_on_LLMs_detection

The lastest paper about detection of LLM-generated text and code

292

THU-BPM/MarkLLM

[EMNLP 2024 Demo] MarkLLM: An Open-Source Toolkit for LLM Watermarking

1,074

LKRCharon/LLM-Watermark-Attack-Papers

This repository is a curated collection of research papers, attack strategies, and tools related to…

1

junchaoIU/LLM-generated-Text-Detection

A survey and reflection on the latest research breakthroughs in LLM-generated Text detection,…

84

QingYuan-L/Awesome-LLM-Fingerprint

8

and-mill/Awesome-GenAI-Watermarking

A curated list of watermarking schemes for generative AI models

145

ToForkDrankXs/Text-WaterMark-PaperList

This repo is to record some text watermark papers.

1

51 commits

updated Aug 31, 2026

See the code

README

Text-WaterMark-PaperList

Text watermark is an important research direction for the misuse of LLM-generated text. This repo will have the followings:

1.BackGround

1.1 Application

  • Deep Fake Detection: Detect whether the target text is generated by a language model;
  • Deep Fake Attribution: Determine which model/user generated the target text;
  • IP Protection: Protect valuable text and model

1.2 Classification

1.2.1 Bit Count

Distinguished based on the amount of information that can be embedded in the watermark.

  • Zero-Bit: Only determine whether to add watermark, no additional information. This makes the watermark only detect, no attribution effect.
  • Multi-Bit: Embed and extract multi-bit information, you can freely add the content you want to embed, such as time, user ID, etc. Multi-Bit watermark is necessary for deep fake attribution.

1.2.2 Embedding Phase

Text watermark research existed in the last century, at which time it was only possible to add watermarks to already written text, and was classified as Post-hoc. However, with the development of language models, some new stages of text watermark emerges.

  • Backdoor/A-Priori: By employing specific methods to utilize watermark information to form a toxic dataset, fine-tuning or training a language model, modify the model weights directly can result in a language model with watermark. The introduction of watermark information occurs before the language model generates text.
  • Generating-Process: The introduction of watermark information will interfere with the generation process of the language model, resulting in the language model producing text with embedded watermarks. The addition of such watermarks does not require modification of the language model, but it necessitates access to the complete language model.
  • Post-Hoc: By altering the text to introduce watermark information, the watermarked text is constructed. This kind of method does not require the original generative language model; it only requires the text generated by the model.

1.2.3 Generating-Process

This category represents a further subdivision within Category Embedding Phase→Generating-Process.

  • Token-Level: Watermark information is introduced when the language model generates a token each time.
  • Sentence-Level: The embedding of watermark information is used when the language model generates each sentence. This requires a semi-controlled sampling process when generating: 1.No constraint is imposed until a sentence is generated; 2.Multiple candidates must be obtained using beam search during generation; 3.When generated, it must be generated sentence by sentence.

1.2.4 Token-Level

This category represents a further subdivision within Category Embedding Phase→Generating-Process→Token-Level.

During each step of token generation, the language model undergoes the following processing: Get Logits on the vocabulary $\rightarrow$ Get Probability Distribution(Weight) over the vocabulary $\rightarrow$ Sample from the probability distribution

  • Logits Bias: The watermark information is converted into a logits distribution on the vocabulary, which is added to the logits distribution generated by the original language model to interfere with the generation phase.
  • Reweighting: The watermark information will guide reweighting probability distribution. This typically involves scrambling the vocabulary and selecting a re-weighting interval.
  • Sampling: Watermark information is embedded by influencing the generation results through sampling, which prohibits the commonly used decoding methods and restricts the decoding approach.

1.2.5 Different Scenarios

  • Completion: Normal Generation. Given a prompt and complete it. Most watermarks are configured for this type of scenario.
  • Condition: Conditional text generation. Such tasks are typically in the form of question-and-answer (QA) and summarization tasks.
  • Code: Code generation. Generating executable code.

1.3 Necessary Features

The addition of watermarks often involves a trade-off among multiple aspects.

  • Detectability: Detectability refers to the capability of a watermarking method to distinguish between watermarked text and non-watermarked text. Metric: Typically, it is a binary classification metric, and in the case of multiple bits, there are corresponding multi-bit comparison metrics.
    • TPR/TNR/FPR/FNR
    • AUROC
    • Accuracy
    • TPR@FPR=X%
    • Bit Accuracy/Bit Error Rate(Multi-Bit)
  • Invisibility/Text Quality: Invisibility refers to the impact of the watermark on the quality of the generated text. Metric: Typically, the text quality metrics of the original generated text and the watermarked text are compared.
    • Perplexity(PPL)
    • BLEU
    • ROUGE(-n)
    • BERTScore
    • Entailment Score(ES)
    • Sentence Similarity
    • Human Evaluate
    • Ent-31
    • Rep-32
  • Imperceptibility: Imperceptibility indicates the impact of watermark addition on the overall token distribution, focusing more on the statistical differences between watermarked and non-watermarked texts, which requires a large amount of corresponding text for overall estimation. Metric: Imperceptibility-related metrics often involve token-level statistics for the overall generated text. It assesses whether there are differences in the token distribution between watermarked texts and non-watermarked texts.
    • Word Frequency
  • Robustness: Robustness refers to the ability of watermarked text to still be recognized as watermarked after undergoing watermark removal attacks. Metric: Robustness-related metrics are represented by comparing the changes in detectability indicators caused by attacks.
  • Usability: Usability refers to the additional time and memory consumption caused by the addition of watermarks. These costs must be within an acceptable range for the user. Metric: Usability metrics typically consist of corresponding values for watermarking time and memory usage.
    • Time Cost
    • Memory Cost

1.4 Attack Function

1.4.1 Word Level

The watermark is removed by modifying the word.

  • Insert: Insert words randomly according to proportion.
  • Delete: Delete words randomly according to proportion.
  • Exchange: Swap the positions of two words in a sentence.
  • Replace: Similar substitutions are usually chosen based on the semantics of a particular word.
    • Synonym: Choose word substitutes according to the calculated thesaurus or the trained word vector.
    • Context: According to the MLM task of Masked Language Model, candidate words are determined by context.
  • Emoji: The model is induced to insert emojis according to certain rules.
  • Exception Word Removal: ONION, In the inference stage, GPT-2 pre-trained model is used to prevent the activation of backdoors by detecting and removing abnormal words such as "cf" in test samples.
  • HELM Perturbation: Deliberately use word variations to remove watermarks.
    • Structure: do not --> don't
    • Case: Large --> large
    • Spell/Ghost-Word: sun --> sunn; sun --> san

1.4.2 Sentence Level

The watermark is removed by modifying the sentence.

  • Human-Modify: Humans manually modify it directly.
  • Paraphrase: Use language models to rewrite sentences. Model: DIPPER;pegasus_paraphrase...
    • Bigram-Paraphrase3: When overwriting, several candidates are generated and the candidate with the least overlap of tokens is selected.
  • Back-Translate: Translate into another language, translate back.
  • Re-Watermark: Re-add watermark when overwriting.

1.4.3 Spoof Attack

The method of cracking the watermark is to try to forge the watermark text in a certain way.

2.Watermark Paper

About Key Word:

3.Survey

4.Attack

In this part, the core of the paper is the research of the watermarking attack method, and sometimes the corresponding solution is proposed.

5.Improvement for Watermarking

6.Evaluation & Benchmark

7.Finding Research

8.Other Related Work

Reference

Footnotes

  1. Generating informative and diverse conversational responses via adversarial information maximization. ↩

  2. Neural text generation with unlikelihood training ↩

  3. SemStamp: A Semantic Watermark with Paraphrastic Robustness for Text Generation ↩

Similar repos

plll4zzx/Awesome-LLM-Watermark

A collection list for Large Language Model (LLM) Watermark

70

hzy312/Awesome-LLM-Watermark

UP-TO-DATE LLM Watermark paper. 🔥🔥🔥

380

Xianjun-Yang/Awesome_papers_on_LLMs_detection

The lastest paper about detection of LLM-generated text and code

292

THU-BPM/MarkLLM

[EMNLP 2024 Demo] MarkLLM: An Open-Source Toolkit for LLM Watermarking

1,074

LKRCharon/LLM-Watermark-Attack-Papers

This repository is a curated collection of research papers, attack strategies, and tools related to…

1

junchaoIU/LLM-generated-Text-Detection

A survey and reflection on the latest research breakthroughs in LLM-generated Text detection,…

84

QingYuan-L/Awesome-LLM-Fingerprint

8

and-mill/Awesome-GenAI-Watermarking

A curated list of watermarking schemes for generative AI models

145

Contributors

DrankXs

51 commits