Forging Tree-Ring: Reproducing and Instrumenting Black-Box Semantic Watermark Forgery

• The reproduction used free-tier dual T4 GPUs with 14.6 GB usable memory per device, substantially less per GPU than the original A40 setup.The compute constraint led to instrumentation work that a larger machine would not have required.Source: §1 · p. 1
• The paper reports a verified Reprompt reproduction against Tree-Ring on SDXL, with genuine, clean, and forged detection rates consistent with the original study.The reproduction was run under a fully pinned software and provenance configuration.Source: §1 · p. 1
• The released detector’s discarded non-central χ2 statistic was recovered exactly, and two derived scores produced materially different separations on identical samples.The paper treats score choice as a distinct measurement question rather than relying only on the detector’s returned output.Source: §1 · p. 1
• A mixed-precision compatibility fix for SDXL’s direct autoencoder calls preserved detector output on the tested sample.A controlled probe compared the patched path with full precision.Source: §1 · p. 1
• A four-gate feasibility protocol measured memory, cost, statistic recoverability, and hypothesis separation, and included one prediction refuted by the data.The paper also reports an exploratory uncertainty analysis that bounds how much the recovered separations support at this sample size.Source: §1 · p. 2

2.1. Semantic watermarking

• Tree-Ring embeds a key-derived ring pattern in the initial latent’s Fourier domain, while detection estimates that latent through DDIM inversion and tests masked coefficients against the key.The detector reports a non-central χ2 CDF and declares detection when p < τ; the estimated latent is only approximately invertible.Source: §2.1 · p. 2

2.2. Removal versus forgery

• Removal produces false negatives, whereas forgery produces false positives by making an image the marked model never generated appear attributable to it.Related work distinguishes these attack directions; robustness against distortion does not establish protection against forgery.Source: §2.2 · p. 2

2.3. The Reprompt attack

• Müller et al. [11] show that Reprompt transfers a watermark through inversion in a proxy model, allowing regeneration of an unrelated image accepted by the genuine detector.Their Imprint variant uses optimization and costs roughly two orders of magnitude more per run; this paper reproduces Reprompt only.Source: §2.3 · pp. 2–3

2.4. Reproduction as a research activity

• Reproduction studies report both broad artifact failures and informative divergences, including Hwang et al.’s [8] rotation result that differed sharply from the original Tree-Ring study.Olszewski et al. found that 60% of surveyed papers shipped no code and that 56% of existing artifacts would not run.Source: §2.4 · p. 3

3.1. Hardware

• The notebook’s P100 accelerator was unusable with the installed PyTorch build, leaving the dual-T4 configuration; neither T4 supported bfloat16 for FLUX.1-dev.The model exclusion is a hardware constraint, not evidence about watermark behavior on FLUX.1-dev.Source: §3.1 · p. 3

3.2. Models and configuration

• The target was SDXL base 1.0 and the attacker proxy SD 2.1 base, both run at 512 × 512 for 50 steps; the Tree-Ring key stayed fixed across six trials.The generation seed varied by trial, while the key seed was 999999; the SDXL threshold came from the original 1% false-positive calibration.Source: §3.2 · p. 3

3.3. Running the released attack

• The notebook invokes the released Reprompt entry point with SDXL as target, SD 2.1 as attacker, and resolution 512 with seed 123.The paper builds its reproduction and instrumentation work around that command.Source: §3.3 · pp. 3–4

3.4. Provenance

• The fork is pinned and cleaned to a recorded commit, while each run serializes model, software, device, dtype, and dependency-lock metadata.The notebook checks both the commit hash and a clean working tree before execution.Source: §3.4 · p. 4

3.5. Gates

• The feasibility protocol asks whether both pipelines fit, what one attack costs, whether the detector statistic is recoverable, and whether genuine, clean, and forged observations separate.The gates are framed as prerequisites for the reproduction, not as an experiment intended to establish a general finding about watermarking.Source: §3.5 · p. 4

4.1. Package removal is not idempotent in layered images

• Installing the released requirements over the notebook’s preinstalled stack left incompatible transformers files side by side, causing imports to fail with misleading missing-module errors.Recovery required removing package directories, reinstalling exact pins, and restarting the interpreter because broken modules remained cached.Source: §4.1 · p. 4

4.2. Cache paths are frozen at import

• The Hugging Face hub fixes its cache location on first import, so setting HF_HOME afterward did not redirect weights from the persistent volume.The paper sets the variable before any import and uses the ephemeral high-capacity volume for weights.Source: §4.2 · p. 4

4.3. Guards must interrogate state, not sentinel files

• A persistent lock file falsely signaled that packages were installed after a new session had lost them; the corrected guard checks installed distribution versions directly.The paper’s general lesson is to interrogate the state being protected rather than trust a marker with a different lifetime.Source: §4.3 · p. 4

4.4. Half precision needs the autoencoder handled explicitly

• SDXL’s direct autoencoder calls bypass the pipeline’s force_upcast handling and overflowed in fp16; the fork wraps those calls with an upcast helper.On the tested image, the patched mixed-precision path returned the same detector p-value as full precision: 1.6648 × 10^-36.Source: §4.4 · pp. 4–5

5.1. The detector discards its own statistic

• The released detector returns only the non-central χ2 CDF, while the reimplementation retains x, λ, df, and σ without changing the computation.The detector’s accuracy accessor returns 1 − p; the CDF compresses strong evidence into a narrow range near zero.Source: §5.1 · p. 5

5.2. Validating the reimplementation

• Across all 18 observations, recovered x reproduced released detector outputs with maximum absolute difference exactly 0, ruling out reimplementation error for Section 6.The validation reapplied the same CDF to recovered x and compared it with the released detector’s output.Source: §5.2 · p. 5

5.3. The choice of score is a modelling decision

• Forged-versus-clean AUC is 0.861 for −x and 0.972 for λ −x; genuine reaches 1.000 under both, so neither score is treated as canonical.The scores are not monotonically related, and choosing between them after seeing these samples would be selection on the test set.Source: §5.3 · p. 5

5.4. Measurement harness

• Each trial compares genuine and forged images with a clean Gaussian-noise null; attacker-model inversion carries the watermark pattern into unrelated-prompt regeneration.The clean arm supplies an empirical null for interpreting scores, beyond simply checking whether they cross a detection threshold.Source: §5.4 · p. 5

6. Evaluation

• Results reproduced to the last significant digit across two clean sessions eleven days apart, while all 18 observations shared one fixed Tree-Ring key.Wall-clock time was the only reported quantity that varied, by 2.2%.Source: §6 · p. 6

6.1. G1: memory feasibility

• The attacker peaked at 5.57 GB, the fp16 target at 7.36 GB, and the fp32 target at 13.68 GB; each fit on a 14.6 GB device.Half precision provided 6.32 GB of target-device headroom but was not required for Reprompt to run.Source: §6.1 · p. 6

6.2. G2: wall-clock cost

• One Reprompt run took 325–332 s on a T4, making forgery a sub-six-minute operation on free-access hardware.The authors report that this runtime is several times the original study’s published single-GPU range.Source: §6.2 · p. 6

6.3. G3: statistic recoverability and range

• Recovered x spanned 889.1–1802.2 and matched the detector’s output exactly, while p never reached the predicted zero and had a minimum of 5.45 × 10−49.The source-based prediction of exact-zero p-values was contradicted by the measurements.Source: §6.3 · p. 6

6.4. G4: hypothesis separation

• The forged arm occupied an intermediate statistic band, and the missed forgery’s x fell within the clean range, where the binary decision could not distinguish it from unwatermarked images.The detector computes the statistic but the thresholded decision discards distinctions visible in it.Source: Figure 3 · p. 7, §6.4 · pp. 7–8
• Forged-versus-clean AUC was 0.861 under −x and 0.972 under λ − x, while genuine-versus-clean AUC was 1.000 under both scores.Both candidate scores derive from the same 18 observations and differ by transformation.Source: Figure 5 · p. 7, §6.4 · p. 7
• At n = 6 per arm, tests had coarse resolution and uncorrected comparisons; the authors draw no inferential conclusion from these results.They rely instead on bootstrap intervals and descriptive effect sizes.Source: §6.4 · p. 7

7.1. Where the effort actually went

• About 45% of reproduction code builds an environment for the other 55%, making portability distinct from whether the released artifact executes.The released attack takes one command, while the reproduction notebook is 375 lines.Source: §7.1 · p. 8

7.2. Detector interfaces decide what can be asked

• Research detectors should expose already-computed intermediate quantities alongside decisions; otherwise, downstream users must reimplement and validate them.The released detector returns a CDF rather than the intermediate quantities used to compute it.Source: §7.2 · p. 8

7.3. Predictions that did not survive measurement

• The prediction that detector p-values were left-censored at zero was disproved by measurement, illustrating the risk of relying on code-reading assumptions.The authors connect this mistake to predictions that go unchecked when reasoning from apparent code behavior instead of measurement.Source: §7.3 · p. 8
• The half-precision probe validated the already-patched path, so it did not independently localize the original fault.The authors also corrected their attribution of the PSNR fix: it was commit d5885a3, not documentation-only commit 43eec7e.Source: §7.3 · p. 8

7.4. What this says about the attack

• 5/6 forged trials succeeded, supporting the original Reprompt conclusion at 325–332 s per attempt on free hardware.The reported result is from the first configuration tried.Source: §7.4 · p. 9
• The authors reproduced only Tree-Ring, attributing its vulnerability to recoverable-artifact similarity detection and reuse of the injection secret for verification.They do not generalize the finding to untested schemes and say schemes with distinct injection and verification secrets are not implicated.Source: §7.4 · p. 9

8. Limitations

• All eighteen observations use one Tree-Ring seed, so the reported separation is within-key rather than a distribution over watermarks.The paper identifies this as its most serious limitation; prior work found Tree-Ring behaves differently in the multi-key setting [3].Source: §8 · p. 9
• The study measures Tree-Ring detector outputs but has no forgery-detection baseline to distinguish forged from genuine images by other means.Lee et al. [9] propose a scheme-agnostic pre-verification detector that the authors identify as a natural next comparison.Source: §8 · p. 9
• Eighteen observations—six per arm—support reproduction and statistic recovery, but not distributional claims or calibrated AUC estimates.The 5/6 forged-detection rate’s difference from the reported 0.97 cannot be resolved at this sample size.Source: §8 · p. 9
• Testing covers Reprompt only; the authors did not replicate Imprint because its reported per-run cost is roughly two orders of magnitude higher.They make no claim about Imprint on this hardware.Source: §8 · p. 9
• The fp16-versus-fp32 probe found identical detection on one patched image, not general numerical neutrality of fp16.The original results used fp32 throughout, while this study used mixed precision with a patched autoencoder path.Source: §8 · p. 9
• The study excludes FLUX.1-dev because T4 GPUs lack bfloat16 and PixArt-Σ because its DPM scheduler would confound comparison with SDXL’s DDIM.The original work reports that scheduler choice substantially affects latent-recovery quality.Source: §8 · p. 9
• Only pristine images were tested; recompression, resizing, and screenshotting remain untested, and both populations need matched handling for comparison.Source: §8 · p. 9
• The SD 2.1 weights came from a mirror after the original repository was withdrawn; their behavior was checked, but not bitwise equivalence.Source: §8 · p. 9

9. Conclusion

• Genuine images were detected 6/6, clean images 0/6, and forged images 5/6 in the SDXL Reprompt reproduction, at 325–332 s per attempt.The clean arm’s mean p was 0.480 against 0.472 in the original work.Source: §9 · p. 9
• AUC 0.861 and 0.972 were obtained by two scores derived from the exactly recovered non-central χ2 statistic on the same eighteen observations.The released detector returns only the statistic’s CDF; the forged arm occupies a band between the clean null and genuine arm.Source: §9 · p. 9
• The predicted zero-censoring was contradicted: none of eighteen p-values was zero, and the minimum was 5.45 × 10^-49.The authors also found their half-precision assumption had been based on source inspection rather than a probe of the unpatched path.Source: §9 · pp. 9–10
• The stated next step is testing across watermark-key clusters and comparing genuine and forged populations after matched laundering.Source: §9 · p. 10

Availability

• The reproduction notebook, measurement artifacts, analysis and figure scripts, and paper source are released in the accompanying repository.The notebook was re-executed independently from a clean environment build, with matching fork commit and dependency-lock SHA-256 recorded for both sessions.Source: Availability · p. 10
• The modified attack code is pinned to commit 7f9e7ad, branching from upstream ca68950, with both hashes and the dependency-lock SHA-256 recorded.Source: Availability · p. 10