PaperlayerView exact revision on arXiv

Source-linked AI summary

Watermarks Without Verification: AI Text Watermarking After the EU AI Act

Alexander Nemecek, Vipin Chaudhary, Erman Ayday

arXiv:2609.09604v1cs.CYcs.AIcs.CR

TL;DR

The paper asks how claims about deployed AI text watermarks can be verified when outsiders cannot inspect proprietary systems. It classifies the required evidence, evaluates open-source SynthID-Text on two open-weight models, and finds near-seed-level prose effects, small or undetectable code-quality costs, and near-chance code detection. It concludes that withheld access and missing institutions—not watermarking alone—are the central governance problem.

  • Problem

    The paper addresses the lack of independent evidence for users’ objections and providers’ assurances about deployed text watermarking.

  • Method

    The paper classifies contested claims by verification requirements and evaluates open-source SynthID-Text on two open-weight models because deployed systems cannot be publicly tested.

  • Results

    The evaluation finds that prose effects do not exceed sampling-seed variation, code correctness falls by 3.1 points on Llama and 0.4 points on Gemma, and code detection remains near chance.

  • Takeaways & Limitations

    The paper maps unresolved claims to matched outputs, configuration disclosure, independent audits, shared evaluation protocols, and interoperable detection.

Abstract

from arXiv · show

On August 2, 2026, the obligations of Article 50 of the EU AI Act took effect, requiring generative AI providers to mark the content their systems produce and ensure it can be detected as AI-generated. Days later, Anthropic disclosed that every Claude model released after that date embeds a watermark based on SynthID-Text in all generated text, enabled by default with no user opt-out; Google has deployed SynthID-Text in Gemini since 2024. Users objected that the watermark degrades quality, particularly for code, that it secretly encodes identifying information, and, in mutual contradiction, that it is easily removable and inescapable; the vendor answered with assurances of unchanged quality, no identifying information, and robustness to light editing. In this work, we argue that neither the objections nor the assurances can currently be verified and that this unverifiability, rather than watermarking itself, is the substantive governance failure. We sort the contested assertions by what it would take to settle each and evaluate the open-source SynthID-Text implementation on two open-weight models, because no public tool can test the deployed systems. On prose, the measured effect of the watermark does not exceed that of changing the sampling seed. On code, the cost is three points of correctness on one model and below measurement on the other, while detection remains near chance, a limitation of detectability rather than quality. The remaining gaps trace to withheld access or missing institutions and we map each to a requirement: release of matched outputs, configuration disclosure, accredited audits, a shared evaluation protocol, and interoperable detection.

I. INTRODUCTION

The paper examines a post–EU AI Act debate over deployed text watermarking, where users’ objections and providers’ assurances cannot yet be independently verified. It evaluates what can be tested publicly and argues that missing verification institutions, rather than watermarking itself, create the central governance failure.

  • Article 50 requires generative AI providers to mark produced content in machine-readable form and ensure that it can be detected as AI-generated.
  • Anthropic disclosed default SynthID-Text watermarking for all Claude models released after August 2, while Google had already deployed it in Gemini.
  • Users alleged quality degradation, covert identification, easy removal, and inescapability, although the removal and inescapability claims contradict one another.
  • Providers assured users that the watermark is imperceptible, has no practical quality impact, carries no identifying information, and survives light editing.
  • The paper evaluates public implementations and classifies contested claims by whether they require public tools, vendor cooperation, or new institutions to settle.
  • The paper argues that unverifiability, rather than watermarking itself, is the substantive governance failure.

II. CLAIMS IN CONFLICT

The section sorts watermarking assertions by the evidence needed to resolve them. Some claims are publicly testable, others require vendor access, and traceability claims remain unresolved without standards and audit institutions.

  • Table I groups assertions into publicly testable claims, claims requiring vendor cooperation, and claims unverifiable absent standards and audit institutions.
  • The deployment’s absence of hidden characters and extra token consumption is externally testable, and the vendor’s account holds in that case.
  • Matched watermarked and unwatermarked outputs would settle Claude-specific quality and perceptibility questions, while confidence-scored detection would test removability claims.
  • Traceability cannot be resolved externally because per-customer keys could identify a source without encoding information in the text.
  • Claims that the watermark is easily removable and inescapable cannot both be true: paraphrasing removes the mark, while light editing does not.

III. WHAT THE LITERATURE ALREADY SAYS

The literature review narrows its focus to distinguishing settled questions in sampling-based watermarking from claims that remain unresolved in the 2026 deployment debate.

  • The section uses Table I’s properties to identify which public claims re-litigate established results and which questions remain open.

A. Distortion-Based and Distortion-Free Watermarks

Sampling-based text watermarks fall into distortion-based schemes that alter token probabilities and distortion-free schemes that preserve the next-token distribution while encoding the signal in sampling randomness. SynthID-Text belongs to the latter family but offers adjustable strength and can operate in a distortionary configuration.

  • Distortion-Based and Distortion-Free Watermarks: Distortion-based watermarks raise logits for key-selected green-list tokens, changing the sampling distribution and typically increasing perplexity.
  • Distortion-Based and Distortion-Free Watermarks: Distortion-free watermarks leave the next-token distribution unchanged and encode the mark through keyed pseudorandom sampling choices.
  • Distortion-Based and Distortion-Free Watermarks: Distortion-free signals weaken when the next token is nearly determined, including in code, structured output, and short factual answers.
  • Distortion-Based and Distortion-Free Watermarks: SynthID-Text samples candidate tokens, assigns keyed g-values, selects through tournament layers, and detects watermarks using the mean g-value score.
  • Distortion-Based and Distortion-Free Watermarks: Multi-bit watermarks can carry payloads such as model versions or account identifiers, whereas published SynthID-Text is zero-bit and returns only a detector score.
  • Distortion-Based and Distortion-Free Watermarks: Post-hoc detectors inspect unmarked text for statistical or stylistic signs of machine authorship but lack an embedded signal and secret key.

B. What Has Been Measured

Existing evidence largely supports small quality costs and context-dependent removability, but detectability and attribution remain important weaknesses. The evidence is also limited for mixed human–AI text, multilingual effects, and disparate impact.

  • Quality and imperceptibility: Quality costs are small for both watermark families, with modest perplexity increases for distortion-based schemes and parity for distortion-free schemes.Google’s live Gemini comparison found no difference in user feedback, though the comparison was vendor-run and reported only in aggregate.
  • Detectability: Detection weakens on short or highly predictable text because detector scores accumulate with passage length and token entropy.A code study measured this directly, making low entropy a central constraint for code detectability.
  • Removability: Token-level edits and mixing watermarked spans with human text preserve detection when enough watermarked tokens remain, whereas paraphrasing and translation substantially reduce the signal.A full rewrite marks the practical boundary described by the evidence, although removal depends on passage length and detector threshold.
  • Attribution and fairness: Published error rates are sparse for human-written text edited by a model because such text is neither fully watermarked nor fully unwatermarked.Google’s detector uses watermarked, unwatermarked, and uncertain states, while deployment-scale calibration and population error rates remain undisclosed.
  • Attribution and fairness: Evidence that post-hoc classifiers flag non-native English writers at elevated rates does not transfer directly to watermark detectors, which score a key rather than writing style.Uneven detectability across languages remains a possible concern, but the supplied evidence does not establish it for watermarking.

C. What Remains Open

Open questions concern deployment and institutions rather than the watermark mechanism itself. The main gaps are multilingual deployment evidence, comparable evaluation standards, and coordination across providers.

  • Deployment evidence: Multilingual deployment-scale behavior remains unreported, including quality and detectability disaggregated by language or user group.The family-level evidence is confined to open-weight models.
  • Evaluation standards: Existing benchmarks and toolkits use different metrics, text lengths, attack suites, and false-positive rates, preventing reliable comparison across schemes or vendor assurances.The paper identifies an agreed evaluation protocol as a missing requirement.
  • Provider coordination: Providers use separate watermarking schemes and keys without a shared detection interface, leaving multi-provider text checking unresolved.Watermarks and signed provenance metadata may also diverge after file editing.
  • Scope of the open questions: These remaining gaps concern deployments and institutions rather than the watermark mechanism.The paper locates them in Sections V and VI.

IV. IS THE SKEPTICISM PROPORTIONATE?

The evaluation finds that SynthID-Text has little measurable effect on prose beyond ordinary seed variation, while code detection remains near chance despite limited quality cost. These family-level results do not directly verify the deployed product because its configuration and outputs remain inaccessible.

  • Evaluation design: Matched watermarked and unwatermarked outputs were generated alongside a different-seed control on two open-weight models.The design isolates watermark effects from ordinary sampling drift and fixes equivalence bounds in advance.
  • Prose: On prose, the watermark’s measured effect did not exceed changing the sampling seed.Perplexity changed by −0.3% on each model, while Llama’s seed control drifted +1.5%; preference win rates were 0.501 and 0.515.
  • Code quality: On code, correctness fell 3.1 points on Llama but was equivalent on Gemma, where watermarked samples scored 61.8% versus 61.4% unwatermarked.The reported non-distortionary configuration therefore produced a small cost on one model and none on the other.
  • Detectability: Code detection stayed near chance at every tested length, with detector scores of 0.55 and 0.57 on the two models.By contrast, prose detection strengthened with length, reaching 39% true-positive rate at 200 tokens and 56% to 59% at 400 tokens.
  • Scope: The evaluation bounds claims about the watermark family, not Anthropic’s deployment, whose strength, entropy cutoff, and calibration cannot be tested externally.The same three-arm design would directly settle the quality rows on the deployed system if access were provided.

V. WHAT CANNOT BE VERIFIED

Verification is blocked by two distinct gaps: vendors withhold deployment access, and institutions needed for trustworthy cross-provider verification do not yet exist. The paper maps these gaps to concrete requirements, while arguing that their absence fuels contradictory public claims and untestable assurances.

  • Withheld access: Withheld matched outputs, scored detection, and deployed configuration prevent external testing of product-specific quality, removability, and calibration claims.These are resources the vendor could supply, and their absence reflects a product decision rather than production cost.
  • Configuration: The entropy cutoff is undisclosed, leaving uncertainty about where deployment disables watermarking for low-entropy content such as code.Watermarking every token can cost correctness, whereas the non-distortionary configuration leaves code detection near chance.
  • Traceability: Traceability claims require auditing infrastructure, including key counts, assignment, rotation, and custody, that external parties cannot inspect.Output releases cannot distinguish whether identifying information is encoded because the decisive evidence concerns infrastructure rather than text.
  • Institutions: Even full vendor cooperation would not resolve verification without a shared protocol, accredited independent auditors, and coordinated reporting across providers.Existing benchmarks differ in metrics, passage lengths, attack suites, and operating points, while no shared detection interface exists.
  • Consequence: The combined absences leave public assurances untestable, encouraging worst-case reasoning, contradictory objections, and unvalidated removal tools.The paper locates the governance failure in the inability to check the deployment rather than in watermarking alone.

VI. MOVING FORWARD

The paper formalizes five requirements for making watermarking claims externally verifiable: vendor releases and configuration disclosure, accredited audits, a shared protocol, and interoperable detection. The first two lie within a vendor’s power, while the final three require independent institutions or collective coordination.

  • VI. MOVING FORWARD: Five requirements address the evidence gaps: matched outputs, configuration disclosure, accredited audits, a shared evaluation protocol, and interoperable detection.These requirements are assigned to parties positioned to supply the missing evidence.
  • VI. MOVING FORWARD: Matched watermark-on and watermark-off outputs, plus scored detector access, would make deployment-specific quality, removability, and attribution claims testable externally.The release would also allow outside measurement of false flags on human text edited by the model.
  • VI. MOVING FORWARD: Configuration disclosure should report the deployed variant, watermark strength, entropy cutoff, threshold, and below-threshold output share by content type.These parameters do not reveal the key or ease removal, but make coverage assurances externally checkable.
  • VI. MOVING FORWARD: Accredited audits are needed for traceability and detector governance, including key access, calibration, population-scale error rates, and attribution disputes.Public-key designs can permit verification without forgery, while intermediate designs can provide scored access to accredited parties.
  • VI. MOVING FORWARD: A shared protocol must define disaggregated reporting by language and content type because existing benchmarks disagree on evaluation quantities.Neither current NIST guidance nor the EU process has yet produced a sufficiently specific protocol.
  • VI. MOVING FORWARD: Interoperable detector interfaces should return scored answers at stated error rates across providers and specify consistency between text watermarks and signed provenance metadata after editing.The interface is necessary for checking text that has passed through multiple models.
  • VI. MOVING FORWARD: The first two requirements are within a single vendor’s power, whereas audits, protocols, and interoperable venues cannot be supplied by vendors alone.

VII. OUTLOOK

Watermarking may either mature into trusted provenance infrastructure or become a compliance ritual. The outcome depends on access, disclosure, and institutions surrounding watermarking schemes, including independent evaluation and interoperable detection.

  • VII. OUTLOOK: Watermarking has two possible trajectories: trusted provenance infrastructure under common protocols and external audits, or compliance ritual driven mainly by legal requirements.
  • VII. OUTLOOK: Access, disclosure, and institutions surrounding watermarking schemes—not the schemes alone—will determine which trajectory emerges.
  • VII. OUTLOOK: Existing protocols, benchmarks, and toolkits already evaluate quality, detectability, and robustness, while Article 50 does not require shared evaluation, independent audits, or vendor-independent detection interfaces.A small-scale family-level comparison required only an open-weight model, public tooling, and a configuration flag.
  • VII. OUTLOOK: Verification mechanisms must be established while public trust remains available to protect.
Loading 2609.09604v1…