An Entropy-based Text Watermarking Detection Method

1 Introduction

• Low-entropy watermarked text may be mistaken for human writing because it contains fewer green tokens and receives a smaller detection score.The watermark strength cannot simply be increased excessively to compensate without risking output text quality.Source: §1 · pp. 1–2, Figure 1 · p. 1
• SWEET excludes tokens below a manually set entropy threshold but treats all remaining tokens equally, and its performance gain is limited.The threshold is set by analyzing human-written code samples.Source: §1 · p. 2
• EWD assigns token influence weights according to entropy, using a monotonically increasing continuous function to distinguish higher- from lower-entropy tokens.The authors state that the detection-side weighting is training-free and avoids a human dataset or manual operations.Source: §1 · p. 2
• The paper also provides a theoretical analysis of detection accuracy in the low-entropy setting.Source: §1 · p. 2
• EWD improves detection accuracy in low-entropy scenarios while providing similar performance to other methods on high-entropy text.The introduction also reports robustness to back-translation watermark-removal attacks.Source: §1 · p. 2

2 Related Work

• Watermarking existing text commonly substitutes words, but the extent of modification is limited to protect text quality, reducing capacity and effectiveness.Source: §2 · p. 2
• LLM watermarking can modify token sampling or logits; many methods extend the logits-based approach for payload, robustness, or unforgeability.The section names methods including SIR and X-SIR, and describes defenses against editing and cross-lingual removal attacks.Source: §2 · p. 3
• For low-entropy code generation, Lee et al. (2023) proposed selective watermarking, but it requires human code data to set the entropy threshold manually.Source: §2 · p. 3

3.1 Text Generation Process of LLMs

• At each generation step, an LLM uses the prompt and previously generated tokens to produce a probability distribution over the vocabulary for the next token.Source: §3.1 · p. 3

3.2 Text Watermarking

• KGW adds watermark strength δ to green-list token logits, biasing generation toward green tokens; detection compares the green-token count with a threshold.The green-list ratio γ is the fraction of the model vocabulary assigned to the green list.Source: §3.2 · p. 3

3.3 Token Entropy and Low-entropy Scenario

• Watermark generation more readily affects high-entropy tokens, whereas low-entropy tokens are harder to change and yield fewer green tokens.Consequently, low-entropy watermarked text can be detected as human-written.Source: §3.3 · p. 3

4.1 Motivation

• Entropy affects how readily watermark generation produces green tokens, yet KGW gives every token equal influence during detection.Source: §4.1 · p. 4

4.2 Entropy-based Text Watermarking Detection

• EWD makes a token’s detection influence proportional to its entropy, so high-entropy tokens count more and low-entropy tokens count less.Source: §4.2 · p. 4
• EWD uses a monotonically increasing, continuous function to customize weights from token entropy; unlike SWEET’s threshold bands, this reflects entropy differences within each band.C0 is the minimum spike entropy used to normalize the input.Source: §4.2 · p. 4
• The detector computes token entropy from model logits, assigns weights, applies the standard KGW procedure to identify green tokens, then calculates a z-score for thresholding.The process is fully automated and eliminates SWEET’s manual operations.Source: §4.2 · pp. 4–5

5 Theoretical Analysis

• Assuming independent token sampling, the analysis approximates the green-token sum with a normal distribution and compares its probability of crossing the detection threshold.Source: §5 · p. 5

5.1 Type-I Error

• At a detection threshold of 2, all methods have the same theoretical Type-I error: 2.28%.Source: §5.1 · p. 5

5.2 Type-II Error

• The Type-II analysis models low-entropy spike entropy with a power-law distribution averaging 0.608, compared with KGW’s reported 0.807 for sampled news scripts.Source: §5.2 · p. 5
• False negative rates exceed 33.4% for EWD, 41.7% for SWEET, and 84.1% for KGW under their z = 2 detection thresholds.Source: §5.2 · p. 6
• EWD achieves the lowest theoretical Type-II error among the methods while maintaining the same Type-I error in low-entropy detection.Source: §5.2 · p. 6, Table 2 · p. 6

6.1 Experiment Settings

• Experiments use KGW watermark generation across low-entropy code tasks and high-entropy news and data-to-text tasks.HumanEval and MBPP use StarCoder; news generation uses OPT-1.3B, while Rotowire uses a fine-tuned T5-small model.Source: §6.1 · p. 6

6.2 Main Results

• EWD improves TPR over SWEET by 2.5% on HumanEval and 9.4% on MBPP while keeping FPR below 5%, and performs similarly to baselines on Rotowire and C4.The main results compare KGW, SWEET, and EWD across the datasets.Source: Table 3 · p. 7, §6.2 · p. 7

6.3 Empirical Analysis

• EWD produces higher watermarked-text z-scores and slightly lower human-text z-scores, with a larger separation in code-generation tasks than Rotowire.The analysis links green-token probability to token weight in watermarked texts, but reports no such relationship in human texts.Source: §6.3 · p. 7, Figure 2 · p. 8
• Two monotonic alternatives to linear weighting, Sigmoid and Exponential, can surpass SWEET in detection F1 under 1% FPR.The tested function strengths change how weight differences are distributed across token entropy levels.Source: §6.3 · p. 8, Figure 3 · p. 8
• With a general prompt replacing original prompts for MBPP entropy calculation, EWD still outperforms baselines, although both EWD and SWEET lose performance.The comparison is reported in Table 4.Source: §6.3 · p. 9, Table 4 · p. 9
• EWD detection takes 0.0831 seconds per text, versus 0.0830 for SWEET and 0.0391 for KGW.The authors describe the difference from SWEET as virtually insignificant, while EWD takes about twice KGW's time.Source: §6.3 · p. 9, Table 5 · p. 9
• After English-to-French-to-English back-translation, EWD achieves the best detection accuracy under 1% FPR and results similar to KGW under 5% FPR.The attack evaluation uses watermarked texts generated from C4.Source: §6.3 · p. 9, Table 6 · p. 9

7 Conclusion

• EWD customizes token detection weights to reflect the positive relation between entropy and influence.The paper describes this as fully considering token entropy during watermark detection.Source: §7 · p. 9
• The paper provides theoretical analysis of detection properties for different methods in low-entropy scenarios.Source: §7 · p. 9
• Experiments validate EWD on low-entropy code generation with multiple weight functions, while high-entropy detection accuracy, speed, and robustness are reported as similar to previous work.Source: §7 · p. 9

• Low-entropy evaluation uses only two code-generation datasets.Source: Limitations · p. 9
• Future work aims to include more low-entropy tasks and datasets.Source: Limitations · p. 9
• Broader watermark-method effectiveness is theoretical, with implementation on additional frameworks left for future work.Source: Limitations · p. 9

• The theoretical mean μ is the weighted sum of green tokens in the watermarked text, with μ = 5.79 under the simulated spike-entropy values.The proof represents the weighted number of green tokens from each token k using |s|Gk.Source: A Proof of Type-II Error for EWD · p. 12
• The proof obtains σ = 0.56 from the simulated spike-entropy moments E[SE4] = 0.24 and E[SE3] = 0.158.Source: A Proof of Type-II Error for EWD · p. 12
• 33.4% is the Type-II error when the detection threshold z′ is 2, corresponding to |˜s|G = 5.55.The stated probability is that |s|G ∼ N(5.79, 0.31) falls below 5.55.Source: A Proof of Type-II Error for EWD · p. 12