OpenStamp: A Watermark for Open-Source Language Models

Miroojin Bakshi; Saksham Rastogi; Danish Pruthi

Source-linked AI summary

OpenStamp: A Watermark for Open-Source Language Models

Miroojin Bakshi, Saksham Rastogi, Danish Pruthi

arXiv:2608.27899v1Aug 28, 2026cs.CLcs.AIcs.LG

TL;DR

Open-source LLM watermarking needs to remain effective when users can bypass decoding-time controls and modify released models. OpenStamp embeds a designed watermark offset in the unembedding weights, achieving strong detection with minimal quality degradation and improved robustness to paraphrasing and post-hoc fine-tuning. Its detector is computationally more expensive than text-only methods and lacks a formal statistical-test interpretation.

Problem
Decoding-based watermarks can be disabled by users of open-source models, while existing approaches face limited robustness to paraphrasing and post-hoc model modifications.
Method
OpenStamp embeds watermarking logic by adding a designed offset matrix to the unembedding weights, biasing logits during generation.
Results
OpenStamp achieves near-perfect detection at 1% FPR with minimal text-quality degradation and outperforms open-source baselines at comparable perplexity levels.
Takeaways & Limitations
The watermark is integrated into generation without decoding-time intervention and shows improved robustness to paraphrasing and post-hoc fine-tuning.
Takeaways & Limitations
Detection requires a model forward pass and base-model token probabilities, while the LLR detector lacks a formal statistical-test interpretation and universally valid p-values.

Abstract

from arXiv · show

With the growing prevalence of large language model (LLM) generated content, watermarking is considered a promising approach for attributing text to LLMs and distinguishing it from human-written content. A prominent class of techniques embeds subtle but detectable signals in generated text by modifying token sampling probabilities. However, such methods are unsuitable for open-source models, where users have white-box access and can easily disable watermarking during inference. In this work, we introduce OpenStamp, a watermarking technique that encodes the watermarking logic directly into the model weights by modifying only the final projection, or unembedding, layer. Through experiments across two models, we show that OpenStamp achieves superior detection performance, with minimal degradation in model capabilities compared to prior methods. The implanted watermark is explicitly designed, and empirically confirmed, to be more robust to paraphrasing attacks and harder to scrub off through post-hoc fine-tuning than prior open-source watermarks. To enable developers to watermark their models, we release our code alongside watermarked versions of 4 popular open-source models.

1 Introduction

OpenStamp addresses the difficulty of watermarking open-source LLMs by embedding watermarking logic into model weights rather than relying on user-controllable decoding-time procedures. It reports strong detection, limited text-quality degradation, and improved resistance to paraphrasing and fine-tuning attacks.

LLM watermarking supports distinguishing generated text from human writing and attributing content to source models, promoting transparency and accountability.
Decoding-based watermarks are unsuitable for open-source models because users can control generation and disable the watermarking logic.
Prior open-source approaches face substantial training costs, reverse-engineering vulnerabilities, and weak robustness to paraphrasing attacks.Distillation requires considerable data and computation, while fixed token preferences can be reverse-engineered and rendered ineffective.
OpenStamp modifies the unembedding weights with a designed offset that biases logits before generation, embedding watermark signals directly into generated text.
TPR ≥99% at 1% FPR is achieved with minimal text-quality degradation, outperforming other open-source methods at comparable perplexity levels.The method also approaches the Pareto frontier of prominent decoding-based techniques and is reported as more robust to paraphrasing and fine-tuning.

2 Background and related work

Prior LLM watermarks commonly alter decoding or token preferences, but open-source settings require watermark logic embedded in model weights because users can bypass decoding-time controls. Existing approaches remain limited by computational demands, reverse-engineering risks, and poor durability under model modifications.

LLM watermarks embed imperceptible signals in generated text that can be algorithmically detected, often by altering decoding and testing token-frequency skew.
Paraphrasing attacks dilute or remove token-level statistical patterns, motivating methods that select tokens using semantic information for greater robustness.
Open-source watermarking must embed its logic in model weights because users can bypass any decoding-time mechanism.Distillation can make watermarking native to a student model but requires substantial computational resources.
Fixed perturbations to final-layer biases create context-independent token preferences analogous to fixed green lists, which are vulnerable to reverse-engineering.
Existing methods generally lack robustness to post-hoc modifications such as quantization, pruning, merging, and fine-tuning.The cited evidence states that no current methods remain detectable after such updates.

3 Methodology

OpenStamp embeds watermarking logic in the unembedding weights, using semantic projection, context-dependent green-list selection, and logit biasing. Detection uses a length-normalized log-likelihood ratio, while the design targets detectability, controllability, security, and paraphrasing robustness.

3.2 Watermarking via unembedding matrix modification: OpenStamp modifies the unembedding matrix so hidden states produce watermark logits that bias token probabilities during generation.The offset matrix transforms hidden states into added logits, whose accumulated bias embeds a detectable signal.
3.3 Linearized green list biasing: The offset matrix decomposes into semantic projection, soft green-list selection, and logit biasing through three sequential operations.P aligns hidden-state geometry with semantic similarity, S selects among L candidate lists, and G maps selections to watermark logits.
3.3 Linearized green list biasing: Semantic projection is trained to make semantically similar hidden states geometrically similar, supporting consistent watermark behavior across paraphrases.The projection matrix is trained by minimizing KL divergence between projected-state and target-embedding similarity distributions.
3.3 Linearized green list biasing: The selector dynamically varies watermark logits across multiple candidate green lists, making the watermark harder to reverse-engineer.S produces a soft selector over L lists, while γ and δ control the watermark’s strength and impact on text quality.
3.3 Linearized green list biasing: Because selectors are continuous, multiple green lists may be partially activated, weakening alignment with exact green-list behavior despite strong detection.The paper analyzes this approximation and its influence on detection in Appendix L.
3.4 Detection via log-likelihood ratio: Detection computes a length-normalized log-likelihood ratio between the watermarked and original models, using the full token probability distributions.Unlike frequency-based tests, this score is not a formal statistical test with one universally valid null distribution.

4 Experimental setup

The experiments evaluate OpenStamp and baselines on detection, robustness, model modifications, and downstream-task impact across Llama-2-7B and Mistral-7B. Watermarked completions are generated from RealNewsLike prompts, and detectability and quality are measured with thresholded detection metrics and perplexity.

Evaluation scope: Experiments cover detection, paraphrasing robustness, resistance to post-hoc model modifications, and downstream-task impact on Llama-2-7B and Mistral-7B.Detection is additionally evaluated on phi-4, Olmo-3-7B, Qwen2.5-7B, and SmolLM2-1.7B.
Reporting: Table 1 reports TPR@0.1%FPR and perplexity across multiple datasets, comparing detectability and text quality for each model.Bold values identify the best TPR at the lowest PPL for each dataset and model.
Baselines: OpenStamp is compared with Unremovable, GaussMark, KGW Distilled, conventional KGW, and KGW combined with the LLR detector.The first three are open-source-compatible approaches; KGW provides a decoding-based reference.
Data and generation: The generation setup uses 500 watermarked completions of 200 tokens from 50-token RealNewsLike prompts, with nucleus sampling at temperature 1.0.Detection is performed only on the continuation after each prompt; unwatermarked continuations come from the dataset.
Metrics: Detectability is measured by AUROC and TPR at 1% and 0.1% false-positive rates, while text quality is measured by mean perplexity.Thresholds are selected separately for each target false-positive rate because low false positives matter in applications such as plagiarism detection.
Experimental exclusions: The study omits RL-based watermarking because the model either produces repetitive sequences or has unacceptably low detection rates.Further details are reported in Appendix J.

5 Results

OpenStamp is evaluated for detection performance, paraphrasing robustness, resistance to post-hoc model modifications, and downstream capability preservation. Across these tests, it generally outperforms open-source baselines while maintaining model accuracy.

5.1 Detection performance: The LLR detector improves detectability relative to KGW’s native detector and similarly strengthens baseline detection.The gap between KGW and KGW+LLR in Figure 2 illustrates this detector-level improvement.
5.2 Robustness to paraphrasing attacks: OpenStamp achieves the highest TPR@1%FPR on paraphrased text for both evaluated models.Removing the projection matrix degrades performance, indicating its contribution to paraphrasing robustness.
5.3 Resistance to post-hoc model modifications: OpenStamp maintains higher detectability than GaussMark, KGW Distilled, and Unremovable during post-hoc fine-tuning.All methods degrade over time; the experiment measures TPR@1%FPR through 2,500 steps, approximately 60 million tokens.
5.3 Resistance to post-hoc model modifications: After instruction fine-tuning, OpenStamp retains the highest TPR@1%FPR on RealNewsLike and remains competitive on AlpacaEval.Detectability remains strong on general-domain text but drops for every method on instruction-following outputs.
5.3 Resistance to post-hoc model modifications: Quantization preserves high detectability for OpenStamp and the baselines under both NF4 and INT8.OpenStamp also causes minimal accuracy degradation across all Llama-2-7B benchmarks, with results statistically indistinguishable from the unwatermarked baseline.

6 Limitations

The paper identifies computational, access, and statistical-interpretation limitations in OpenStamp’s detection procedure.

6 Limitations: Detection requires a model forward pass and base-model token probabilities, and its score lacks a probabilistic interpretation.Unlike text-only statistical tests, the method does not provide confidence levels or false positive rates for detection uncertainty.

7 Conclusion

The paper concludes that OpenStamp embeds watermarking logic in unembedding weights while preserving text quality and improving robustness.

7 Conclusion: OpenStamp integrates watermarking into the generation process through the unembedding layer, avoiding decoding-time interventions.The paper reports stronger detection, paraphrasing robustness, and post-hoc fine-tuning robustness while preserving text quality.
7 Conclusion: Future work could improve offset matrix designs and develop detection methods with statistical guarantees.These directions address robustness and the absence of probabilistic detection guarantees.

Appendices

The appendix formalizes detection as a length-normalized log-likelihood ratio between original and watermarked models, thresholded to classify sequences.

Detection algorithm: Algorithm 1 extracts backbone hidden states and computes sequence log-likelihoods under the original and watermarked unembedding matrices.The watermarked model incorporates the offset matrix ∆W before producing token probabilities.
Detection algorithm: The detector returns a watermarked classification when the length-normalized LLR score exceeds threshold τ.The score is computed as (ℓwm − ℓorig)/(T −1).

B Mistral additional results

Additional Mistral-7B experiments evaluate robustness to fine-tuning, downstream accuracy, instruction tuning, model families, and distribution shifts. OpenStamp generally preserves detectability and text quality, though performance varies by dataset.

Post-hoc fine-tuning: OpenStamp maintains higher detectability than GaussMark, KGW Distilled, and Unremovable under post-hoc fine-tuning.All methods degrade over time, but OpenStamp degrades less.
Downstream task accuracy: OpenStamp exhibits minimal degradation in relative downstream task accuracy across benchmarks.
Instruction fine-tuning: OpenStamp retains the strongest TPR@1%FPR on RealNewsLike after Alpaca instruction fine-tuning, while AlpacaEval scores are lower across methods.
Generalizability across LLMs: OpenStamp achieves perfect TPR@1%FPR across phi-4, Olmo-3-7B, Qwen2.5-7B, and SmolLM2-1.7B, with watermarked perplexity reasonably close to baseline perplexity.
Distribution shift: OpenStamp retains strong detectability on Japanese and mathematical web text but degrades on Python code, where baselines degrade more severely.Unremovable trails OpenStamp across all shifted datasets, with the largest gap on code.
Hyperparameter effects: Varying γ and δ exposes a trade-off between watermark detectability and text distortion.

F Training the projection matrix P

OpenStamp trains a projection matrix to align model hidden states with semantic embeddings, improving consistency between original and paraphrased text. The projection is part of a broader watermark pipeline that selects and biases green lists.

Projection matrix P: The projection matrix P is trained from 1.5 million paired hidden states and Qwen3-Embedding-8B sentence embeddings extracted from OpenWebText.Hidden states come from the model’s final layer, with sequences capped at 512 tokens.
Projection matrix P: P is optimized with a contrastive loss using AdamW for 15 epochs, with learning rate 10^-5, weight decay 10^-2, and temperature Tsim = 0.1.
Projection evaluation: Projection substantially increases cosine similarity between original and paraphrased hidden states compared with raw hidden states.This places paraphrase-related hidden states in a more semantically aligned space.
Embedding-model sensitivity: Comparable paraphrasing robustness across Qwen3-Embedding-8B, multilingual-e5-large, and bge-m3 outperforms the ablation without semantic alignment.The benefit of alignment does not appear to depend on one embedding model.
Selector matrix S: The selector matrix S classifies projected hidden states into L distinct classes, created through incremental K-Means clustering.Clusters with fewer than 10 hidden states are discarded, and ridge regularization uses λ = 10^-3.
Paraphrase evaluation: Paraphrased completions retain high semantic similarity, with a mean cosine similarity of 0.92.

I Hyperparameter details for watermarking methods

The experiments use representative configurations for OpenStamp and several baselines, alongside specified fine-tuning and decoding setups. An RL-based watermarking comparison highlights a practical detectability–repetition trade-off.

OpenStamp: OpenStamp uses L = 254 green lists, γ = 0.25, and δ = 1.0 for experiments outside the Pareto evaluation.For the Pareto evaluation, δ ranges from 0.3 to 1.2.
Baseline configurations: GaussMark perturbs MLP up-projection weights in one decoder block, while Unremovable adds a fixed final-bias perturbation and KGW Distilled uses distilled logit biases.
Baseline configurations: The decoding-time KGW baseline uses k = 1, γ = 0.25, and representative δ = 1.5 for non-Pareto evaluations.
Evaluation metrics: The repetition metric seq-rep-3 is the proportion of duplicate 3-grams in a sequence.
RL comparison: OpenStamp preserves low repetition while maintaining strong detectability under both greedy and multinomial decoding, unlike the RL watermark.Greedy decoding makes the RL watermark detectable but causes severe repetition.
Fine-tuning setup: Post-hoc fine-tuning updates internal transformer linear layers and the unembedding layer, whereas instruction fine-tuning applies LoRA only to attention and MLP projections.

L Analysis of watermarking behavior

OpenStamp’s watermark behavior depends on multiple green lists, semantic projection, and continuous likelihood-ratio detection. Increasing list diversity improves variability and security-related properties while preserving detection, and full LLR scoring retains information lost by discretization.

Effect of L: Increasing L weakens alignment between top watermark-logit tokens and the intended green list.The measured overlap decreases as the number of green lists grows.
Effect of L: PPL remains stable across L values and stays below the GaussMark, KGW Distilled, and Unremovable baselines at TPR@1%FPR ≥ 0.90.
Effect of L: Mean Jaccard similarity between top-biased token sets decreases as L increases, with the decline plateauing for large L.This reflects diminishing marginal gains in watermark-logit variability.
Prompt independence: Detection scores are largely consistent with and without the original prompt, diverging only slightly at generation start.This suggests detection relies mainly on immediate local context.
Selector inversion: Reconstructing the selector matrix is challenging because assignments are sensitive to the dataset and K-Means initialization seed.Table 11 reports moderate agreement across selector matrices with the same L.
Detector ablations: Discretized LLR and binomial detectors perform noticeably worse than the full LLR detector because discretization loses mixed-green-list information.
LLR on baselines: Replacing baseline-native detectors with LLR improves detectability for GaussMark, KGW Distilled, and Unremovable on both evaluated models.The largest gain occurs for GaussMark, while generated texts remain unchanged.