PaperlayerView exact revision on arXiv

Source-linked AI summary

TripPattern: A Pattern-based Text Watermarking Method for Large Language Models

Sangjun Moon, Dasom Choi, Jingun Kwon, Hidetaka Kamigaito, Taro Watanabe, Manabu Okumura

arXiv:2609.12472v1cs.AI

TL;DR

Green/red watermarking can reduce text quality by restricting token choice. TripPattern instead alternates between two pattern groups while allowing neutral-token sampling, and evaluations across four multilingual datasets report preserved generation quality with robust detectability.

  • Problem

    Green/red token partitioning can restrict vocabulary flexibility and degrade text quality and naturalness.

    Source: §1 · p. 1

  • Method

    TripPattern partitions vocabulary into two alternating pattern groups and a neutral group, then detects watermarks by testing adjacent pattern-group alternations.

    Source: §1 · p. 2

  • Results

    Across four multilingual datasets, TripPattern preserves LLM generation quality while achieving robust watermark detectability.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    TripPattern provides a favorable balance between text quality and watermark detectability compared with conventional two-group partitioning methods in multilingual settings.

    Source: §6 · p. 9

    Strong context-aware paraphrasing such as Dipper can disrupt TripPattern's local token-level patterns while preserving meaning.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Text watermarking techniques have gained significant attention for identifying machine-generated text and mitigating risks from large language models (LLMs). Existing methods typically divide an LLM's vocabulary into green and red tokens, but encouraging generation toward green tokens can reduce text quality and naturalness. To address this, we propose TripPattern, a watermarking framework that formulates text watermarking as a pattern-based matching task using three vocabulary partitions. TripPattern divides the vocabulary into one neutral group and two pattern groups. During generation, the model alternates token selection between the two pattern groups to embed detectable patterns, while neutral tokens are selected independently to improve flexibility and preserve naturalness. For detection, TripPattern uses pattern-based statistical tests that provide interpretable p-values by measuring how often adjacent tokens alternate between the pattern groups. Theoretical analysis and empirical evaluations on four multilingual datasets show that TripPattern maintains LLM generation quality while achieving robust watermark detectability.

1 Introduction

Existing green/red watermarking can reduce text quality by restricting token choice. TripPattern uses two alternating pattern groups and a neutral group, and experiments across four multilingual datasets show preserved generation quality with robust detectability.

1 Introduction

  • Green/red watermarking biases generation toward green tokens, restricting vocabulary flexibility and often degrading quality and naturalness.This trade-off is especially consequential in multilingual and practical settings where linguistic diversity and fluency matter.Source: §1 · p. 1
  • TripPattern partitions vocabulary into two pattern groups and one neutral group, alternating between pattern groups while sampling neutral tokens independently.Neutral-token sampling is intended to preserve fluency and naturalness.Source: Figure 1 · p. 1, §1 · p. 2
  • The detector tests how often adjacent pattern-group tokens alternate, using that statistic to identify the watermark.Neutral tokens do not carry watermark signals and are excluded from detection.Source: §1 · p. 2
  • Across four multilingual datasets, TripPattern preserves LLM generation quality while achieving robust watermark detectability.The paper also considers subword token length when constructing the neutral group.Source: §1 · p. 2

2 Method

The method contrasts conventional green/red token partitioning with TripPattern's three-group design. It embeds alternating patterns while leaving neutral tokens selectable, constructs that neutral group using subword length, detects alternation statistically, and proves a conditional flexibility advantage over KGW.

2.1 Preliminary

  • KGW partitions vocabulary into green and red sets, biases green-token logits, and detects watermarks by testing whether green tokens occur more often than expected.The verifier reconstructs the partition and computes a z-score against the null hypothesis that the text does not follow the green-list constraint.Source: §2.1 · p. 2

2.2 TripPattern: Three-group Partitioning

  • TripPattern divides vocabulary into two pattern groups and a freely selectable neutral group, biasing allowable tokens to encourage alternation between the pattern groups.Neutral tokens do not change the alternation state; the allowable set includes the opposite pattern group and the neutral group.Source: §2.2 · p. 3

2.3 Subword Length-based Neutral Group Construction

  • Shorter subword tokens receive priority for neutral-group assignment until the target neutral-group ratio is reached; remaining tokens are split between pattern groups.The partition is reconstructed at each decoding step using the same keyed hash and PRNG procedure.Source: §2.3 · p. 3

2.4 Detecting the Watermark

  • Detection reconstructs the three-way partition, removes neutral tokens, and tests adjacent pattern-token alternations under a permutation null conditional on pattern-group counts.A z-score exceeding a predefined threshold leads the detector to reject the null hypothesis and identify the watermark.Source: §2.4 · p. 3

2.5 Theoretical Analysis

  • TripPattern's biased token set is larger than KGW's when the stated condition holds, suggesting greater token-selection flexibility.The paper then empirically examines whether that flexibility translates into improved generation quality.Source: §2.5 · p. 4

3 Experiments

Experiments evaluate TripPattern’s fluency, reasoning accuracy, and watermark detection across four languages. Results show generally preserved text quality and reasoning performance alongside competitive detection.

3.1 Experimental Settings

  • Experiments cover English, Korean, German, and Spanish fluency plus GSM8K and MGSM reasoning, evaluating PPL, answer accuracy, AUC, and TPR@5.The tested models include Llama-3 3B/8B and Qwen-2.5 3B/7B; decoding is greedy, and the neutral-group length threshold is three.Source: §3.1 · p. 4

3.2 Main Results

  • Across four languages, TripPattern generally keeps PPL low with limited degradation versus non-watermarked text and competitive or better quality than watermarking baselines.Source: §3.2 · pp. 4–5, Table 1 · p. 5
  • The length-based neutral group generally improves average fluency versus TripPattern w/o Len, though effects vary by model and language.It assigns frequent short subword tokens, such as functional words and morphemes, to the neutral group.Source: §3.2 · p. 5
  • On GSM8K and MGSM, TripPattern maintains reasoning performance with limited accuracy degradation while retaining coherent reasoning steps and final answers.It better preserves reasoning accuracy in several settings while achieving competitive detection performance.Source: §3.2 · pp. 5–6, Table 2 · p. 5
  • Across tasks, TripPattern achieves competitive detection and a stronger overall quality–detectability balance, although KGW sometimes has higher TPR@5.Source: §3.2 · pp. 5–6, Table 1 · p. 5

4 Analysis

TripPattern’s quality advantage under hard partitioning comes with a detection penalty when the neutral group becomes too large. Further analyses examine the effects of watermark strength, token-length thresholds, decoding strategies, summarization, and attacks.

Flexibility of TripPattern

  • Under hard partitioning, TripPattern beats TripPattern w/o Len and KGW in text quality, with similar AUC; a large γ sharply weakens detection.The comparison uses separate γ sweeps because γ denotes the green-token ratio for KGW and the neutral-group ratio for TripPattern.Source: Figure 2 · p. 6, §4 · p. 6

Performance Trade-offs at Different δ

  • Across δ values, TripPattern preserves text quality more effectively than KGW while maintaining reliable watermark detection.Source: Figure 3 · p. 6, §4 · p. 6

Analysis of Subword Length

  • A subword-length threshold of three generally gives the best balance between text quality and detectability, performing well across models in English, Korean, and German.Source: §4 · p. 6

Performance on Text Quality

Effect of Decoding Strategy

  • Detection performance remains largely stable across the decoding strategies tested.Source: §4 · p. 7

Robustness against Attacks

  • TripPattern retains reasonable detection under word-level perturbations, slightly trails KGW at 30% WordDel and WordSub, and is more vulnerable to Dipper paraphrasing.The authors attribute the Dipper vulnerability to disruption of the local alternation patterns used for detection.Source: Table 5 · p. 7, §4 · p. 7

Investigating Softmax Probability

  • Multilingual datasets have substantially higher average top-1 next-token certainty than English, which the authors link to weaker watermark insertion and reduced detection performance.Source: Table 6 · p. 7, §4 · p. 7

5 Related Work

Prior watermarking approaches include post-hoc detection, text modification, and decoding-stage token biasing, while multilingual vocabulary-partitioning watermarking has received limited attention. TripPattern addresses that gap with three vocabulary groups and length-based neutral-group construction.

  • Post-hoc binary classifiers have become less effective, while rule-based and neural text-modification watermarks can degrade quality through substantial changes to text.Source: §5 · pp. 7–8
  • Figure 4 examines neutral-group length thresholds, watermark strength effects on PPL and AUC, and decoding-strategy effects on text quality and detection.Source: Figure 4 · p. 8
  • Decoding-stage methods modify token logits without changing the underlying LLM; KGW introduced green/red partitioning and green-token biasing, later extended in several ways.Source: §5 · p. 8
  • Prior work has paid limited attention to multilingual vocabulary-partitioning watermarking; TripPattern uses two pattern groups, a neutral group, and subword length to construct that neutral group.Source: §5 · p. 8

6 Conclusion

TripPattern uses three vocabulary groups and subword token length to balance text quality and watermark detectability. The authors report a favorable balance compared with conventional two-group partitioning in multilingual settings.

  • TripPattern partitions the vocabulary into two pattern groups and one neutral group, alternating selection between the pattern groups while using the neutral group to preserve quality and naturalness.Source: §6 · p. 8
  • The method uses subword token length to define the neutral group and evaluates four languages, addressing limited prior attention to multilingual watermark embedding.Source: §6 · p. 8
  • TripPattern provides a favorable balance between text quality and watermark detectability compared with conventional two-group partitioning methods in multilingual settings.Source: §6 · p. 9

Limitations

TripPattern’s limitations concern paraphrasing robustness, quality–detection trade-offs, concentrated multilingual distributions, and evaluation scope. The algorithm alternates pattern-group selection while retaining neutral tokens and detects alternation among non-neutral tokens.

Limitations

  • Strong context-aware paraphrasing such as Dipper can disrupt local token-level alternation patterns while preserving meaning.Robustness against semantic paraphrasing remains future work.Source: Limitations · p. 9
  • Larger watermark strength δ can improve detectability while reducing quality, whereas a larger neutral-group ratio γ increases selection flexibility but can dilute the signal.Optimal settings may vary by model, language, domain, and decoding strategy despite fixed cross-language hyperparameters.Source: Limitations · p. 9
  • Highly concentrated model distributions can weaken watermark insertion and degrade detection, potentially more in low-resource languages, domain-specific text, or models with different tokenization.The paper attributes this to higher next-token certainty making generation less responsive to watermarking bias.Source: Limitations · p. 9
  • Experiments cover instruction-tuned Llama and Qwen models and a limited set of languages and tasks, leaving broader generalizability unevaluated.The authors call for evaluation across more languages, model families, domains, and real-world deployment scenarios.Source: Limitations · p. 9

A TripPattern Algorithm

  • Generation reconstructs a step-specific vocabulary partition, targets the opposite pattern group from the last non-neutral token, and soft-biases that group plus neutral tokens.The alternation state is unchanged when a neutral token is sampled.Source: A TripPattern Algorithm · p. 12
  • Detection reconstructs each position’s partition, excludes neutral tokens, and tests whether adjacent pattern labels alternate more often than expected under a permutation null.The null is conditional on the counts of labels 1 and 2; a z-score above a predefined threshold identifies the watermark.Source: A TripPattern Algorithm · p. 13

B Proof

The proof compares the token-selection flexibility of TripPattern with the KGW two-group framework. Its formal result conditions the flexibility advantage on the relative group proportions.

B Proof

  • TripPattern’s detector maps non-neutral tokens to pattern labels, counts adjacent alternations, and computes a z-score under a permutation null conditional on label counts.It rejects the no-watermark null when the z-score exceeds a predefined threshold.Source: B Proof · p. 13
  • KGW consistently biases selection toward its green group, while TripPattern’s allowable set changes with the preceding pattern group and includes the neutral group.TripPattern selects the opposite pattern group after a non-neutral pattern token; initially, it targets Vp1.Source: B Proof · pp. 13–14
  • When γ_KGW ≤ (1 + γ_N)/2, TripPattern mathematically ensures greater token-selection flexibility than KGW-based methods.The paper connects this greater flexibility directly to generated-text naturalness and quality.Source: B Proof · p. 14

C Additional Experimental Results

The additional comparison reports that TripPattern detects watermarks substantially better than MorphMark across tested models and languages while maintaining practical generation quality.

C Additional Experimental Results

  • TripPattern achieves substantially higher detection performance than MorphMark across models and languages while maintaining practical generation quality.MorphMark adaptively scales bias according to token-level green probability mass on KGW; the comparison is reported in Tables 7 and 8.Source: C Additional Experimental Results · p. 14

D Case Study

The case study identifies Table 9 as an example of TripPattern-generated outputs from Llama-3.2-3B-Instruct.

D Case Study

  • Table 9 presents example outputs generated by Llama-3.2-3B-Instruct using TripPattern.The passage does not state a comparative outcome for the examples.Source: D Case Study · p. 14

E Bin-Wise Analysis

On C4, TripPattern consistently detects watermarks even when fewer tokens are generated. The bin-wise analysis compares AUC and TPR@5 across token-length bins.

E Bin-Wise Analysis

Loading 2609.12472v1…