PaperlayerView exact revision on arXiv

Source-linked AI summary

MARCO: The Radioactive Watermark for Protein Generative Models

Huajie Chen, Xin Guo, Yuchen Shi, Yuchen Zhong, Minhui Xue, Chi Liu, Congcong Zhu, Kun Gao, Minfeng Qi, Tianqing Zhu

arXiv:2610.08316v1cs.CRcs.AI

TL;DR

Proprietary PGMs face model-extraction and biosecurity risks, while protein watermarks must preserve structural geometry. MARCO adds a watermark during reverse diffusion with the PGM weights frozen and trains for recovery under structural attacks. The reported results show strong fidelity and robustness, and the watermark transfers to pirate models trained on protected structures.

  • Problem

    PGMs face economic risks from proprietary model extraction and biosecurity risks from attempts to engineer toxins or pathogens, while protein watermarks must preserve conformation and withstand SE(3) transformations.

    Source: §1 · p. 1

  • Method

    MARCO uses an encoder-decoder and discriminator with a frozen PGM to embed watermarks during reverse diffusion and recover them from stochastically distorted structures.

    Source: §1 · p. 2, §4.2 · p. 5

  • Results

    MARCO leads most fidelity metrics and outperforms all four baselines in robustness, maintaining BA > 0.90 and DR > 0.99 under every perturbation.

    Source: §5.1 · p. 8, §5.2 · p. 8

  • Takeaways & Limitations

    Watermarks embedded during reverse diffusion shift the output distribution so pirate models trained on protected samples inherit a traceable signal.

    Source: §1 · p. 2

    The threat model gives the adversary black-box, query-only access to the victim PGM but unlimited access to watermarked structures; an adaptive attacker may have white-box access to its pirate model.

    Source: §3 · p. 4

Abstract

from arXiv · show

Protein Generative Models (PGMs) have revolutionized structural biology by enabling the design of complex 3D protein structures from sequence data. However, this breakthrough introduces a dual-use challenge, exposing high-value PGMs to economic risks like unauthorized model extraction and biosecurity threats such as biohazard synthesis. To mitigate these threats, we propose \textbf{MARCO} (\textsc{COnformation waterMARk}), the first radioactive watermarking framework specifically tailored for PGMs. MARCO establishes a Dual-Layer defense that simultaneously protects intellectual property and ensures the forensic traceability of potential biosecurity misuses. (i) To preserve efficiency, MARCO iteratively embeds watermarks during diffusion reverse denoising via an auxiliary encoder-decoder, allowing the original PGM parameters to remain frozen for broad compatibility. (ii) To preserve biophysical fidelity and maximize robustness, we employ specialized loss functions targeting $C_\alpha$-atom pairwise distances and torsion angles ($\psi, \phi$) within an adversarial training framework integrated with stochastic attack simulations. (iii) Crucially, MARCO exhibits ``radioactivity'' where the watermark automatically transfers to the outputs of any pirate models trained on the watermarked data, effectively countering model extraction attacks. Comprehensive experiments demonstrate that MARCO achieves superior fidelity and robustness while successfully validating watermark transferability.

1 INTRODUCTION

PGMs create economic and biosecurity risks, while protein watermarks must preserve biological geometry and withstand rigid transformations. MARCO addresses these constraints with a frozen-model watermark that transfers to derivative models trained on protected structures.

1 INTRODUCTION

  • PGMs can enable pathogen or toxin engineering, while extraction of proprietary models threatens commercial advantages and drug-discovery investment.The paper argues that these risks make tailored PGM protection necessary.Source: §1 · p. 1
  • Protein watermarks must preserve conformation and withstand SE(3) transformations, requiring geometry-aware signals rather than coordinate noise.The cited constraints include folding stability, binding affinity, and conformational flexibility.Source: §1 · p. 1
  • MARCO is presented as a generalized, architecture-agnostic module that adds robust signatures without fine-tuning or retraining the pretrained PGM.Its stated contributions include high-fidelity signatures robust to SE(3) transformations and malicious attacks.Source: §1 · p. 2
  • MARCO intervenes throughout reverse diffusion with an encoder-decoder and discriminator, using fidelity constraints and stochastic distortions to train watermark recovery.The frozen PGM predicts intermediate structures; the encoder perturbs them, the decoder recovers signals, and the discriminator distinguishes clean from watermarked structures.Source: §1 · p. 2
  • A pirate model trained on MARCO-protected samples learns their biased distribution, causing its outputs to inherit the watermark and remain traceable.MARCO shifts the output distribution by watermarking during reverse diffusion rather than modifying only a static structure.Source: §1 · p. 2

2 PRELIMINARIES AND RELATED WORK

PGMs generate structures as rigid frames and use diffusion or flow matching to refine noisy structures. Existing diffusion watermarking approaches bias generated outputs, but protein-specific geometric and biological constraints limit direct transfer of image watermarking methods.

2.1 PROTEIN GENERATIVE MODELS

  • PGMs model joint distributions over protein sequences and structures, representing each structure as residue-level rotations and Cα positions with SE(3)-equivariant predictions.Equivariance makes predictions coordinate-frame independent while supporting physical validity and conformational generalization.Source: §2.1 · p. 2
  • Diffusion and flow-matching models corrupt structures into noise and learn to recover clean structures; sampling iteratively predicts clean backbones and updates translations and rotations.RFdiffusion's stated objective weights Cα coordinate error and orientation-matrix error.Source: §2.1 · pp. 2–3

2.2 WATERMARKING DIFFUSION MODELS

  • Diffusion watermarking either injects signals at inference time or imprints them during model training, biasing output distributions for verification while aiming to preserve fidelity.Examples include watermarking initial noise, moving it toward a watermarked latent region, and fine-tuning a decoder.Source: §2.2 · p. 3
  • Image watermarking methods do not transfer directly to variable-length 3D proteins, which require SE(3)-aware signals that preserve distances, torsions, stereochemistry, stability, and function.The cited protein methods either fine-tune PGM weights or lack multi-bit support or evidence of resistance to adversarial attacks.Source: §2.2 · p. 3

3 THREAT MODEL

The threat model includes a victim who owns and can inspect a proprietary PGM, and an adversary seeking to hide the provenance of protected outputs while retaining their biological utility. MARCO's training embeds and extracts watermarks under simulated attacks and adversarial discrimination.

3 THREAT MODEL

  • The victim has white-box access to its proprietary PGM and computing resources for biophysical validation, and embeds a recoverable watermark in generated structures.The victim's goal is for the decoder to recover the watermark from the protected structure.Source: §3 · p. 3
  • MARCO training uses a frozen PGM with encoder, decoder, attack simulator, and discriminator; the encoder embeds geometric perturbations and the decoder recovers the watermark after simulated attacks.The discriminator distinguishes clean predictions from watermarked structures while the encoder learns perturbations intended to deceive it.Source: §3 · p. 4, Figure 1 · p. 4
  • An adversary succeeds only if watermark detection fails while the transformed structure retains structural integrity and biological function.The adversary knows the watermarking algorithm and architecture, has black-box query-only access to the victim model, and has unlimited access to watermarked structures.Source: §3 · p. 4
  • An adaptive adversary may optimize structures toward thermodynamic minima, use DPSGD to scrub a watermark from a pirate model, or forge a watermark to obscure ownership.The described adversary knows MARCO and has white-box access to its pirate model.Source: §3 · p. 4

4 MARCO METHOD

MARCO adds a watermark to protein structures through a frozen PGM and an encoder-decoder trained to balance recovery, structural fidelity, and covertness. It generates watermarked structures during reverse diffusion and validates suspect structures by statistically testing recovered bits against the original watermark.

4.2 TRAINING MARCO

  • MARCO jointly trains an encoder, decoder, and discriminator around a frozen PGM, embedding and extracting watermarks while adversarial training promotes covertness.The training cycle comprises PGM prediction, watermark embedding, extraction, and adversarial training.Source: §4.1 · p. 4, §4.2 · p. 5
  • At each reverse-diffusion step, the encoder perturbs predicted translations and rotations, and the watermarked predictions guide the next denoising update.The encoder uses residual perturbations; rotational changes are represented in so(3)'s tangent space and mapped to SO(3) with a matrix exponential.Source: §4.2 · pp. 5–6, §4.1 · p. 5
  • Distance-matrix and torsion-angle losses preserve global Cα-distance topology and local backbone geometry while embedding the watermark.The torsion loss penalizes sterically implausible conformations.Source: §4.2 · pp. 6–7
  • A stochastic attack layer applies manipulations such as rotation, cropping, or noise before decoding, training recovery from distorted as well as unchanged structures.The attack is applied with probability 0.5; adversarial training encourages watermarked and clean structures to be statistically indistinguishable.Source: §4.2 · p. 7

4.3 EVALUATING MARCO

  • MARCO timestamps and registers model identity with a trustworthy ledger, then validates suspect structures by testing recovered watermark bits against the original.A detection threshold of z = 3.0 corresponds to a false positive rate of ≤10^-3.Source: Figure 2 · p. 6, §4.3 · p. 7

5 EVALUATION

MARCO is evaluated for fidelity and robustness against four baselines across pretrained PGMs. It leads most fidelity metrics and retains high watermark-recovery performance under the tested conformational perturbations.

5.1 FIDELITY

  • MARCO leads most fidelity metrics, with the lowest RMSD, near-perfect BLOSUM62 scores, BA > 0.95, and DR = 1.0 without attacks.The comparison uses four baselines across six pretrained PGMs; Figure 3 presents the fidelity evaluation.Source: Figure 3 · p. 8, §5.1 · p. 8
  • The comparison setup uses Cα-RMSD, Normalized BLOSUM62 score, Bit Accuracy, and Detection Rate on 1,000 clean-watermarked structure pairs.Watermarks are length 64, corresponding to a 2^64 identification space.Source: §5.1 · p. 8

5.2 ROBUSTNESS

  • BA > 0.90 and DR > 0.99 under every perturbation, with MARCO outperforming all four baselines in both metrics.The evaluation includes random rotation, cropping, and Gaussian noise injection.Source: §5.2 · p. 8, Figure 4 · p. 9
  • Rotation has relatively little effect, whereas cropping and noise sharply degrade baselines by removing structural information and globally distorting conformations, respectively.The authors suggest the attack layer is critical for learning persistent features and that adversarial training may diffuse the watermark through the structure.Source: §5.2 · pp. 8–9

6 DISCUSSION

MARCO is presented as a dual-layer response to economic and biosecurity risks from PGMs, combining watermarking with a digital chain of custody. Its fidelity, robustness, and identification capacity involve trade-offs, while cross-architecture experiments support watermark transfer to pirate models.

6 DISCUSSION

  • MARCO addresses model extraction and biosecurity misuse by using user-specific watermarks for intellectual-property management and a digital chain of custody for accountability.The discussion frames the latter as BWC-compliant.Source: §6 · p. 9
  • Geometric constraints preserve fidelity, while the attack layer and extraction loss enhance robustness; loss weights and manipulation probability can tune the trade-off.Source: §6 · p. 9
  • Increasing watermark length reduces fidelity and robustness, so payload capacity must be balanced against structural integrity and scalable user identification.Source: §6 · p. 9
  • Stepwise reverse-diffusion perturbations make the output distribution radioactive, and cross-architecture experiments confirm transfer to pirate models trained on protected structures.The discussion identifies this transfer as relevant to stolen-model identification and ownership validation.Source: §6 · p. 9
  • The discussion identifies protein language models and wet-lab verification as potential extensions of MARCO.Source: §6 · p. 9

7 CONCLUSION

The conclusion presents MARCO as a watermark for mainstream PGMs that leaves pretrained weights unchanged while balancing structural fidelity and robustness. It reports low geometric and sequence impact, robustness to attacks, and future work on fidelity and broader applicability.

7 CONCLUSION

  • MARCO adapts to mainstream PGMs without modifying their pretrained weights.Source: §7 · p. 9
  • Watermarked structures have negligible geometric deviation from clean predictions, and ProteinMPNN sequence validation indicates minimal influence on recovered amino acid sequences.Source: §7 · p. 9
  • The conclusion reports robustness to conformational manipulations and adaptive attacks, with high-capacity payloads and minimal performance trade-offs.Source: §7 · p. 10
  • Future work will seek greater conformational fidelity and applicability to more generative architectures and biological modalities.Source: §7 · p. 10

AI USE STATEMENT

The authors used generative AI only for language refinement, while retaining responsibility for the paper’s scientific content and final text.

AI USE STATEMENT

  • Generative AI tools were used to check grammar and improve flow and readability.The authors state that the tools did not generate the core concepts, experimental design, data analysis, or scientific conclusions.Source: AI USE STATEMENT · p. 10
  • No substantial portions of the text were AI-generated; the tools served solely as copy-editing assistance.The authors manually reviewed and verified all suggestions and take responsibility for the final publication.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

The ethics statement describes an offline, sandboxed study using public academic datasets, while the design properties specify the intended requirements for PGM watermarking.

ETHICS STATEMENT

  • The authors frame the work as theoretical and isolated, intended to protect model-owner copyrights and trace biosecurity misuse without harming people or organizations.The security assessment treats protein data and models as mathematical artifacts rather than attacking hosting infrastructure.Source: ETHICS STATEMENT · p. 10
  • The study confined its security assessment to a sandbox and reports that experiments ran offline without interacting with live services.The authors also state that they used public academic datasets and processed no personal data.Source: ETHICS STATEMENT · p. 10

A DESIGN PROPERTIES

  • Capacity, efficiency, and generalizability require adequate identification information, low embedding and extraction cost, and independence from particular architectures or protein topologies.These are presented as prerequisites for an effective PGM watermarking framework.Source: A DESIGN PROPERTIES · p. 15
  • Fidelity requires watermarked structures to retain structural integrity and biological function, with physical energy comparable to clean predictions.The section identifies a validated physical energy potential as the basis for this criterion.Source: A DESIGN PROPERTIES · p. 15
  • Robustness requires reliable watermark recovery after utility-preserving transformations or attacks, with bit error rate bounded by a tolerance.Examples include rigid-body rotation, noise injection, and energy minimization.Source: A DESIGN PROPERTIES · p. 15
  • Radioactivity requires a watermark to persist in outputs of a pirate model trained on watermarked structures.The section describes the signature as becoming encoded in the surrogate model’s parameters.Source: A DESIGN PROPERTIES · p. 15
  • Covertness requires the watermarked and clean model distributions to remain statistically indistinguishable, including in key geometric statistics.The listed statistics include bond lengths, torsion angles, and residue packing densities.Source: A DESIGN PROPERTIES · p. 15

B CERTIFIED ROBUSTNESS VIA RANDOMIZED SMOOTHING

Randomized smoothing provides a certificate for watermark-bit recovery under bounded structural perturbations, connecting an L2 radius to an RMSD threshold.

B CERTIFIED ROBUSTNESS VIA RANDOMIZED SMOOTHING

  • The certificate guarantees recovery of the correct watermark bit string under any perturbation bounded by radius r in L2 norm or RMSD.For protein structures, the section relates the L2 perturbation to an RMSD threshold within which a bit will not flip.Source: B CERTIFIED ROBUSTNESS VIA RANDOMIZED… · p. 16
  • If the most likely decoder bit has probability above 0.5, randomized smoothing certifies the smoothed classifier against perturbations within radius r.The radius is determined from the most- and second-most-likely class probabilities and the noise scale.Source: B CERTIFIED ROBUSTNESS VIA RANDOMIZED… · p. 16
  • The authors characterize this bound as an intrinsic geometric property enforced by noise-aware decoder training, not merely an artifact of attack simulations.The claim supplements their empirical robustness demonstrations against various attacks.Source: B CERTIFIED ROBUSTNESS VIA RANDOMIZED… · pp. 15–16

C EXPERIMENT SETTINGS

The experiments evaluate MARCO across several protein generative models, describe the data and baseline setup, and use structural, sequence, and watermark-recovery measures.

C EXPERIMENT SETTINGS

  • The evaluation spans structure-prediction networks and de novo backbone generators, including RFDiffusion, ESMFold, Chroma, FrameDiff, FrameFlow, and FoldFlow2.The passage characterizes these as a diverse suite of state-of-the-art models.Source: C EXPERIMENT SETTINGS · p. 16
  • The curated dataset contains 20,312 monomeric PDB structures, with sequences of 60–512 residues, resolution finer than 5 Å, and structures above 50% loop content excluded.The curation follows RFdiffusion protocols.Source: C EXPERIMENT SETTINGS · p. 17
  • Training freezes the pretrained PGM and optimizes MARCO’s encoder, decoder, and discriminator for 50 epochs using Adam at a 10^-4 learning rate and batch size 64.The reported experiments used eight NVIDIA A100 GPUs; loss weights were set by grid search in pre-experiments.Source: C EXPERIMENT SETTINGS · p. 17
  • The baselines adapt image-diffusion watermarking methods to proteins, while FoldMark uses watermark-conditioned LoRA and fine-tunes each PGM.The listed baselines are AquaLoRA, WaDiff, SleeperMark, and FoldMark; the study follows the original papers’ hyperparameter settings.Source: C EXPERIMENT SETTINGS · p. 17
  • Evaluation measures structural similarity with RMSD, sequence similarity with normalized BLOSUM62, and watermark recovery with bit accuracy and detection rate.The passage defines detection as successful when z ≥ 3, corresponding to BA ≥ 0.6875.Source: C EXPERIMENT SETTINGS · pp. 17–18
  • Figure 5 shows the watermarked structure highly aligned with the clean prediction, which the caption presents as evidence of high fidelity.The figure visualizes the clean prediction alongside the watermarked structure.Source: Figure 5 · p. 18

D.1 ROBUSTNESS

The evaluations report watermark retention across structural attacks and pirate architectures, while also identifying capacity trade-offs, component contributions, and limits to current scope.

D.1 ROBUSTNESS

  • During 100-ns molecular dynamics, fidelity and robustness stabilize from 80 ns; refolding retains the watermark while trading off sequential fidelity.The reported refolding test maps watermarked structures to sequences and refolds them with clean PGMs.Source: §D.1 · pp. 19–20
  • Average BA remains above 0.8 when RMSD < 2.5; under DPSGD with ε ≥4.0, BA ≥0.91 and DR ≥0.99.At sufficiently low scrubbing stiffness, watermark recovery can be compromised, but structural fidelity is also destroyed.Source: §D.1 · pp. 18, 19, 21
  • With 25% watermarked data, MARCO achieves 0.78 BA and 0.85 DR; it also outperforms MarkSplatter on PGM watermarking.Replacing clean structures with FoldMark-watermarked structures did not differ in its effect on removing MARCO’s watermark.Source: §D.1 · pp. 21–22

D.2 RADIOACTIVITY

  • Across source and pirate architectures, MARCO achieves BA ≥0.90 and DR ≥0.99, transferring its signal to pirate-model outputs.The comparison includes RFdiffusion, ESMFold, and Chroma as source PGMs and FrameDiff, FrameFlow, and FoldFlow2 as pirate models.Source: §D.2 · p. 22

D.3 CAPACITY

  • At watermark length l ≥96, RMSD and BA fluctuate significantly; l = 64 offers the reported trade-off for coding space and fidelity.Normalized BLOSUM62 and DR remain stable as length increases, while high-length BA reductions may compromise precise user identification.Source: §D.3 · p. 22

D.4 ABLATION STUDY

  • Removing Lwm, Ldm, or Ltor degrades fidelity; removing Lext eliminates robustness, while omitting the attack layer leaves the watermark vulnerable to conformational manipulations.With Ladv, FD is 2.27 versus 16.31 without it, which the authors say suggests improved covertness.Source: §D.4 · p. 23
  • Performance gains diminish beyond 25 epochs, with negligible improvement from 50 to 100; 50-epoch training takes about 16.71 hours on one A100 GPU.The decoder does not retrieve the predefined watermark from natural or clean predicted structures.Source: §D.4 · pp. 23, 25

E LIMITATIONS & FUTURE WORK

  • MARCO currently targets diffusion-based architectures, and watermark recovery from physically synthesized and re-digitized proteins has not been validated in wet-lab experiments.The authors identify discrete modalities such as protein language models as future work.Source: E LIMITATIONS & FUTURE WORK · p. 25
Loading 2610.08316v1…