Source-linked AI summary
Copying As Defense: Protecting Images from Editing By Eliciting Model Copying
Salma Abdel Magid, Sanghyuk Chun, Eric Slyman, Kushal Kafle, Olga Russakovsky
TL;DR
Existing DiT-editor defenses suppress reference attention, but this can leave text and generated tokens free to follow edit instructions. CaD instead increases per-token target-to-reference attention in early blocks; across evaluated editors and benchmarks, it reduces edit success, though its evaluation is limited to two editors and robustness to adaptive preprocessing is untested.
Problem
Existing defenses for in-context DiT editors suppress reference attention, although attention can shift to text and target tokens that support instruction following.
Source: §1 · p. 2Method
Results
On Flux Kontext BodyBench, CaD reduces edit success to 5.1%, compared with 24.9% for DeContext and 44.2% for Diff-PGD; FIBO results vary by benchmark and metric.
Source: §6 · pp. 7–8Takeaways & Limitations
The findings suggest image protection can rely on making an editor use the reference more rather than hiding it from the model.
Source: §7 · pp. 9–10The evaluation covers two DiT-based editors and does not test robustness to adaptive preprocessing or attacks.
Source: §7 · p. 10
Abstract
from arXiv · showhide
Image editors built on diffusion transformers (DiTs), such as Flux Kontext, make it possible to turn a single photograph into a convincing manipulated image, including harmful and non-consensual edits. Proactive defenses aim to prevent such manipulations by adding an imperceptible perturbation before an image is shared. Existing defenses for DiT-based in-context editors make the output attend less to the protected reference image. But this leaves the model free to follow the edit instruction using the text and its own generated tokens, often producing a new, weakly grounded image rather than preventing the edit. We find that the opposite strategy is more effective: instead of leading the model to ignore the reference, we encourage it to copy it. Through controlled attention interventions, we show that concentrating target-token attention on the reference image in only a few early blocks is sufficient to collapse the editing behavior, causing the model to reproduce its input, often as a lossy copy, rather than execute the requested transformation. Based on this insight, we introduce Copying-as-Defense (CaD), which maximizes per-token target-to-reference attention mass in early transformer blocks. Since attention rows are normalized, increasing reference attention suppresses the attention allocated to the text and generated canvas. Restricting the objective to early blocks also lets us truncate the backpropagation path, keeping optimization cheap. Across editors and benchmarks, CaD reduces average edit success from 81.4% on unprotected images to 24.0%, a 70.6% relative reduction, and by a further 44.3% relative to DeContext, the strongest baseline (43.0%). CaD is also roughly 1.8-2.0x faster per optimization step than comparable methods.
1 INTRODUCTION
DiT editors jointly process reference images, instructions, and generated tokens, while existing protections suppress reference attention. The paper argues instead for inducing copying through stronger early reference attention, which motivates CaD.
1 INTRODUCTION
- Suppressing reference attention can redirect attention to text and target tokens, leaving the editor able to follow malicious instructions and produce weakly grounded images.The failure is especially visible when edits have a strong generative prior or the protected identity occupies few reference tokens.Source: §1 · p. 2
- Controlled interventions show that forcing target queries onto reference keys in a few early blocks makes the editor reproduce its input rather than execute the edit.The contrast also holds for a black reference: maximizing reference attention produces the black input, while suppressing it still permits a new image.Source: §1 · p. 2
- CaD drives per-target-token attention mass toward reference tokens in selected early double-stream blocks.Because attention rows are normalized, increasing reference attention reduces mass for text and target tokens; the per-token objective avoids washing out small identity regions.Source: §1 · p. 2
- Targeting early blocks lets CaD truncate backpropagation after the last selected block, reducing the cost of each optimization step.The paper presents improved protection over existing baselines, comparable imperceptibility, and reduced protection time as contributions.Source: §1 · p. 2
2 RELATED WORK
Image protections address different harms, and their effectiveness depends on the manipulation and model family they target. This section situates CaD among protections for modern in-context DiT editors and broader concerns about robustness.
2 RELATED WORK
- Image protection is not a single objective: defenses differ according to whether they target identity misuse, style mimicry, or general image manipulation.Protections effective against one manipulation may offer little protection against another.Source: §2 · pp. 2–3
- Proactive protections must account for evolving model families and perturbations weakened by ordinary transformations or deliberate purification.The paper argues evaluations should specify both the model and the manipulation being defended against.Source: §2 · p. 3
- Modern in-context DiT editors jointly process reference images, text instructions, and generated images, while DeContext is identified as the only proactive method designed specifically for them.Earlier protection methods can still be applied, but were not designed around this architecture.Source: §2 · p. 3
3 PRELIMINARIES
The paper frames protection as a per-image perturbation applied before an attacker submits an image and instruction to an editor. Its preliminaries describe the optimization template and the editors’ joint token processing.
3 PRELIMINARIES
- Protection adds a bounded perturbation to an image before release, and succeeds when the later edit fails to realize the requested transformation on the depicted subject.The attacker supplies the protected image with an editing instruction and, where required, a mask.Source: §3 · p. 3
- The standard protection template optimizes a per-image perturbation with projected gradient ascent, while prior methods target encoder representations, denoiser predictions, or attention patterns.The passage notes that baseline objectives and implementation details are provided in the appendix.Source: §3 · pp. 3–4
- Flux Kontext and FIBO jointly condition on instructions and reference images, with attention operating across text, target-image, and reference-image tokens.Increasing target-to-reference attention necessarily reduces the mass available to text and target tokens because each attention row is normalized.Source: §3 · p. 4
4 RETHINKING REFERENCE ATTENTION
Reference suppression can fail when the protected subject occupies few image tokens and because attention mass can shift to text and target tokens. Controlled interventions contrast removing reference attention with concentrating it, finding that the latter turns editing toward reproduction.
4.1 WHY SUPPRESSING REFERENCE ATTENTION IS NOT ENOUGH
- On full-body inputs, suppression can leave requested edits successful, unlike its disruption of identity-preserving edits on tightly cropped faces.For Flux Kontext, the person occupies 31% of image tokens and the face 4.8% in the full-body example, versus 92% subject tokens in the face crop; BodyBench face-and-body masks average 28.45% ± 6.06% of image area.Source: §4.1 · p. 4
- Image-averaged suppression can lower attention mainly through background tokens, leaving comparatively few identity-related interactions intact.The example uses 1024 tokens: 944 primarily cover the face-crop subject, while 322 cover the full-body person and 49 cover the face.Source: §4.1 · p. 4
- Suppressing reference attention reallocates mass to text and target tokens, and reference information can also reach output through text tokens that absorbed it in early blocks.Thus, removing direct target-to-reference interaction does not establish that reference information has left the computation.Source: §4.1 · pp. 4–5
4.2 MANUAL INTERVENTION: MINIMIZING VERSUS MAXIMIZING REFERENCE ATTENTION
- The controlled intervention holds other factors fixed while either removing target-to-reference attention or concentrating it on the reference.The minimize intervention sets reference mass to zero and renormalizes the remaining groups; the maximize intervention places all mass on reference keys while preserving their within-reference distribution.Source: §4.1 · p. 5, §4.2 · p. 5
- Forcing target-to-reference attention progressively changes outputs from edits into reference reproductions across more early blocks, while removing it still permits plausible instruction-following edits.The distinction also appears with a black reference, where forcing attention collapses the output toward black and suppressing it permits instruction-consistent content.Source: §4.2 · p. 5
- On 10 BodyBench images, PickScore stays high and nearly flat without intervention or with Target→Ref = 0, but drops substantially under Target→Ref = 1 as more blocks are included.This pattern indicates increasingly suppressed editing under stronger early reference attention.Source: §4.2 · p. 5
5 COPYING AS DEFENSE
CaD reverses reference suppression by maximizing target-to-reference attention for each target query in selected early double-stream blocks. The section describes the objective and its computational rationale, then outlines the evaluation setup.
5 COPYING AS DEFENSE
- CaD maximizes reference attention for every target query in selected early double-stream blocks, driving reference mass toward one per query.Attention normalization makes the reference compete with text and target tokens, while the per-token objective prevents identity-critical regions from being hidden by an image-level average.Source: §5 · p. 5
- Figure 2 contrasts removing reference attention with forcing target queries onto reference keys across increasingly deep prefixes of Flux Kontext’s early double-stream blocks.The caption reports that forced reference attention increasingly reproduces the natural or black reference, while suppression still permits instruction-consistent generation.Source: Figure 2 · p. 6
- Restricting the objective to early blocks allows the forward pass to stop after the selected prefix and avoids backpropagating through the rest of the transformer.The early-block choice serves both as the copying mechanism and as a way to reduce computation.Source: §5 · pp. 5–6
5.1 EXPERIMENTAL SETUP
- Protection is assessed with edit success, PickScore, CLIP directional similarity, ArcFace, LPIPS, PSNR, and optimization-step and per-protection runtime measures.The reported main comparison is at ϵ = 12/255; Table 1 covers datasets and editing models, with runtime measured per optimization step.Source: §5.1 · pp. 6–7, Table 1 · p. 7
- The evaluation uses FLUX.1 Kontext-dev and FIBO Edit, comparing CaD with PhotoGuard, DeContext, and Diff-PGD across perturbation budgets of 4, 8, 12, and 16/255.Each configuration uses 20 InpaintGuardBench images and 30 BodyBench images, with 15 edits per BodyBench image.Source: §5.1 · pp. 6–7
6 RESULTS
CaD protects images across editors and perturbation budgets while reducing optimization cost. Ablations show that increasing target-to-reference attention, particularly in early blocks, is more effective than suppressing it.
6 RESULTS
- On BodyBench with FLUX Kontext, CaD cuts edit success to 5.1%, versus 24.9% for DeContext and 44.2% for Diff-PGD.Source: §6 · p. 7, Figure 3 · p. 8
- On InpaintGuardBench with FIBO Edit, CaD reduces edit success from 80.0% on unprotected images to 18.3%, versus 52.7% for Diff-PGD and 59.7% for DeContext; on BodyBench, Diff-PGD performs slightly better on CLIPdir and edit success.Source: §6 · p. 8
- On Flux Kontext, CaD takes 0.647 s per step and 10.78 min per protection, versus 1.232 s/16.43 min for DeContext and 1.245 s/16.60 min for Diff-PGD.Source: §6 · pp. 8–9
- Across budgets, CaD achieves substantially lower edit success at comparable LPIPS; at ϵ = 8/255 on BodyBench, success is near zero while DeContext and Diff-PGD remain above 50%.Source: §6 · p. 8, Figure 4 · p. 9
- By 300 steps, CaD outperforms the final 800-step baselines across all four protection metrics, requiring about 3.2 minutes on Flux Kontext.Source: Figure 5 · p. 9, §6 · p. 9
6.1 ABLATIONS
- Extending the objective through block 4 sharply decreases PickScore, CLIPdir, and edit success; additional layers may not continue to improve protection.Source: §6.1 · p. 9
- Maximizing target-to-reference attention reduces edit success to 5.1%, whereas minimizing it changes success only from 81.6% to 80.7% under matched settings.Source: §6.1 · p. 9
7 CONCLUSION
CaD reverses the usual defense strategy: making editors rely more on the reference can itself protect images from editing. The evaluation is limited in scope and requires white-box, per-image optimization.
7 CONCLUSION
- CaD shows that making an editor rely more on the reference can itself defend against editing.Source: §7 · pp. 9–10
- Figure 6 describes protection as strong using early double-stream blocks, with the objective applied cumulatively from block 0 through each selected block.Source: Figure 6 · p. 10
- Evaluation covers only two DiT-based editors and assumes white-box access and per-image optimization; robustness to common transformations and adaptive attacks was not evaluated.Source: §7 · p. 10
B PROTECTION METHODS AND THEIR INSTANTIATION ON EACH EDITING MODEL
The protection methods perturb the context image using a shared PGD procedure and differ in the loss they optimize. The section specifies the common optimization setup and model token notation.
B PROTECTION METHODS AND THEIR INSTANTIATION ON EACH EDITING MODEL
- All four methods perturb the context image with the same PGD procedure and differ only in their optimized loss.The comparison covers PhotoGuard, DeContext, Diff-PGD, and CaD on FLUX.1 Kontext-dev and FIBO Edit.Source: B PROTECTION METHODS AND THEIR… · p. 13
- Each step applies a signed-gradient update, then projects the perturbation into the ℓ∞ ball of radius ε around the original image and the valid image range.Images are resized and center-cropped to 512 × 512, normalized to [−1, 1], and the perturbed image remains float32 while models run in bfloat16.Source: B PROTECTION METHODS AND THEIR… · p. 13
- The step size is α = 0.005; PhotoGuard, DeContext, and Diff-PGD use 800 steps, while CaD uses 1000.The sign is positive for loss ascent and negative for loss descent.Source: B PROTECTION METHODS AND THEIR… · p. 13
- The shared token sequence is [T | Z | C], representing text, target-image, and context-image tokens; each image stream has 1024 tokens at 512 × 512.The context tokens are produced by encoding the perturbed image with the model’s VAE encoder.Source: B PROTECTION METHODS AND THEIR… · p. 13
B.1 MODEL-SPECIFIC COMPONENTS
The appendix gives model-specific conditioning details, then describes how PhotoGuard and DeContext instantiate their respective protection objectives. The two models use different image encoders and text-conditioning pipelines.
B.1 MODEL-SPECIFIC COMPONENTS
- FLUX.1 Kontext-dev uses a 16-channel KL autoencoder with 8× spatial downsampling and 2 × 2 latent patch packing, plus pooled CLIP conditioning and 512 T5 tokens.Latents are shifted and scaled by scalar factors.Source: §B.1 · p. 13
- FIBO Edit uses the Wan 2.2 video VAE on a single frame, per-channel latent normalization, and SmolLM3 text conditioning with a joint additive attention mask.Its prompt length varies; each image-instruction pair gets a cached JSON description from the FIBO prompt-to-JSON vision-language model.Source: §B.1 · p. 13
B.2 PHOTOGUARD (ENCODER ATTACK)
- PhotoGuard attacks only the VAE encoder, starting from a randomly perturbed image and descending a loss that pushes its encoding toward the encoding of a mid-gray image.The step size decays linearly from α to α/100; FLUX uses its KL-VAE and FIBO uses the Wan VAE.Source: §B.2 · p. 13
B.3 DECONTEXT (CONTEXT-ATTENTION SUPPRESSION)
- DeContext hooks query/key tensors after QK-normalization in the first 26 single-stream blocks and minimizes the mean proportion of target-query attention assigned to context tokens.For FIBO, padded text tokens are masked before softmax, matching the model’s attention calculation.Source: §B.3 · p. 14
B.4 DIFF-PGD (EDIT-RECONSTRUCTION LOSS)
Diff-PGD protects an image by maximizing the editing model’s flow-matching error on a clean edited target. Its targets are generated from the original image and cached per prompt.
B.4 DIFF-PGD (EDIT-RECONSTRUCTION LOSS)
- Diff-PGD maximizes flow-matching error when the model tries to produce a clean edit from the protected context.For each image and instruction, it first generates and encodes a clean edited target from the original image; targets are generated lazily and cached, with one generation per prompt per image.Source: §B.4 · p. 14
- At each step, Diff-PGD samples an instruction, timestep, and noise, forms a noisy target latent, and ascends the loss using the protected context tokens.For FIBO, the same JSON prompt is used to generate the target and compute the loss, and targets from VLM-generated JSON are cached separately from minimal-prompt targets.Source: §B.4 · p. 14
B.5 OURS (CONTEXT-ATTENTION MAXIMIZATION)
CaD directs target-query attention toward context tokens in selected double-stream blocks. It uses an unconditional prompt and selects different early-block ranges for FLUX and FIBO.
B.5 OURS (CONTEXT-ATTENTION MAXIMIZATION)
- CaD acts on double-stream blocks and drives target-query attention mass onto the reference tokens, using a single empty prompt rather than a prompt pool.It rebuilds the joint [T | Z | C] sequence and computes ordinary softmax attention for target queries.Source: §B.5 · p. 14
- The objective pushes the reference mass of every head and target query toward 1; FLUX selects double blocks 0–9, while FIBO selects blocks 0–3.For FIBO, the attention mask follows the model’s own attention, and a VLM generates one JSON version of the empty instruction per image.Source: §B.5 · p. 14