PaperlayerView exact revision on arXiv

Source-linked AI summary

DeContext as Defense: Safe Image Editing in Diffusion Transformers

Linghui Shen, Mingyue Cui, Xingyi Yang

arXiv:2512.16625v2cs.CV

TL;DR

Personal images can be misused for identity impersonation or misinformation, and existing defenses do not adequately address modern DiT-based in-context editing. DeContext uses targeted perturbations to disrupt multimodal attention carrying context, concentrating them at important denoising steps and transformer blocks. Experiments on FLUX.1-Kontext and Step1X-Edit show DeContext detaches context while preserving high quality; protected images cause over 70% lower face recognition accuracy, with image quality typically within 20% of clean images.

  • Problem

    Personal images can be misused for identity impersonation or misinformation, and existing defenses do not adequately address modern DiT-based in-context editing.

    Source: §1 · p. 2, §2.2 · p. 3

  • Method

    DeContext uses targeted perturbations to disrupt multimodal attention carrying context, concentrating them at important denoising steps and transformer blocks.

    Source: §1 · p. 2, §4.1 · p. 4, §4.2 · p. 4

  • Results

    Experiments on FLUX.1-Kontext and Step1X-Edit show DeContext detaches context while preserving high quality; protected images cause over 70% lower face recognition accuracy, with image quality typically within 20% of clean images.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    The results support attention-based context disruption as a defense for DiT-based in-context image editing that can preserve output quality.

    Source: §1 · p. 2, §6 · p. 8

    Future work aims to improve efficiency on large models and robustness and transferability in black-box settings.

    Source: §6 · p. 8

Abstract

from arXiv · show

In-context diffusion models allow users to modify images with remarkable ease and realism. However, the same power raises serious privacy concerns: personal images can be easily manipulated for identity impersonation, misinformation, or other malicious uses, all without the owner's consent. While prior work has explored input perturbations to protect against misuse in personalized text-to-image generation, the robustness of modern, large-scale in-context DiT-based models remains largely unexamined. In this paper, we propose DeContext, a new method to safeguard input images from unauthorized in-context editing. Our key insight is that contextual information from the source image propagates to the output primarily through multimodal attention layers. By injecting small, targeted perturbations that weaken these cross-attention pathways, DeContext breaks this flow, effectively decouples the link between input and output. This simple defense is both efficient and robust. We further show that early denoising steps and specific transformer blocks dominate context propagation, which allows us to concentrate perturbations where they matter most. Experiments on Flux Kontext and Step1X-Edit show that DeContext consistently blocks unwanted image edits while preserving visual quality. These results highlight the effectiveness of attention-based perturbations as a powerful defense against image manipulation. Code is available at https://github.com/LinghuiiShen/DeContext.

1. Introduction

DeContext addresses privacy risks from unauthorized in-context editing by targeting how DiT models transfer image context. Its attention-aware perturbations are reported to block malicious edits while preserving image quality.

1. Introduction

  • Inference-time editing from a single image can enable identity cloning, style imitation, and reproduction of proprietary content without consent.The paper identifies risks including deepfakes, impersonation, and copyright violation.Source: §1 · p. 2
  • Existing protections were designed mainly for legacy UNets or other settings and can distort outputs without effectively safeguarding DiT-based in-context editing.The paper describes the lack of an effective safeguard tailored to these models.Source: §1 · p. 2
  • DeContext targets multimodal attention, which the paper identifies as the route through which conditional-image information propagates, to inhibit unauthorized identity or style transfer.The method applies localized perturbations between target queries and context keys, without modifying the model.Source: §1 · p. 2
  • Perturbations are concentrated at early denoising steps and influential early-to-middle transformer blocks to improve protection while preserving visual fidelity.The paper reports experiments on FLUX Kontext and Step1X-Edit.Source: §1 · p. 2
  • DeContext outperforms prior defenses on protection and image quality, with over 70% lower face-recognition accuracy and image quality typically within 20% of clean images.The reported protection also foils in-context edits and identity extraction.Source: §1 · p. 2

2. Related Work

The related work describes in-context DiT editing as a privacy challenge that existing defenses do not fully address. It motivates analyzing attention pathways and targeting them selectively.

2.1. Conditional Image Generation

  • Recent DiT-based editors condition generation directly on reference images at inference time, enabling control but exposing personal images to misuse.Potential harms include identity impersonation and misinformation without owner consent.Source: §2.1 · p. 2

2.2. Privacy Protection in Generative Models

  • Prior defenses target training-time personalization or inference-time editing, but existing approaches leave gaps for modern DiT-based image editing and its dual-stream context pathways.The paper notes a lack of systematic analysis of attention mechanisms mediating context-to-output flow.Source: §2.2 · pp. 2–3

2.3. Attention Mechanisms in Diffusion Models

  • Context influence varies across denoising timesteps and transformer depths, motivating targeted cross-attention interventions rather than global model changes.The paper presents its analysis and DeContext as addressing these gaps while maintaining output quality.Source: §2.3 · p. 3

3. Motivation Analysis

The motivation analysis contrasts a conventional attack with direct intervention in attention. It identifies attention-mediated context flow as a more effective target for detaching context from generated images.

3. Motivation Analysis

  • The analysis tests whether standard attacks or direct attention disruption can stop context use, finding that internal attention intervention succeeds where standard attacks fail.This contrast motivates an attack designed specifically to disrupt attention-based conditioning.Source: §3 · p. 3

3.1. Background

  • DiTs jointly process text and image tokens, with concatenated context and target tokens allowing attention to carry context information into the target.The background describes image generation conditioned on a text prompt and a context image.Source: §3.1 · p. 3

3.2. Context Propagates through Attention

  • The standard untargeted PGD attack on the flow-matching loss leaves facial identity and structure clear, showing that this end-to-end attack does not break conditioning.The attack produces mild blur and lighting artifacts while context information remains reflected.Source: §3.2 · pp. 3–4
  • Removing attention links where target images query context images severs context, leaving generation influenced solely by the text prompt.The figure contrasts this outcome with the standard attack, which produces only re-lighting artifacts.Source: §3.2 · p. 4, Figure 2 · p. 4

4. DeContext

DeContext protects DiT in-context editing by perturbing context images to reduce target-to-context attention. It concentrates suppression on early denoising steps and context-heavy transformer blocks.

4.1. Context Detachment via Attention

  • DeContext suppresses attention from target-image queries to context keys, optimizing the context image while leaving model weights frozen.Its objective drives the context proportion of attention toward zero.Source: §4.1 · p. 4
  • Sampling prompts, timesteps, and noise makes single-sample gradient updates an efficient approximation of the expected objective, effective across prompts and seeds.The method samples from a pool of 60 editing commands.Source: §4.1 · p. 4

4.2. Concentrated Context Detachment

  • At early, high-noise timesteps, context-image gradients dominate; DeContext therefore restricts optimization to those influential steps.This also permits approximating the target with random Gaussian noise, avoiding paired images.Source: §4.2 · p. 5, Figure 4 · p. 5
  • Context attention is highest in front-to-middle transformer blocks before attack, and focused suppression reduces it across all blocks.The block-wise strategy targets the context-heavy blocks identified by the analysis.Source: §4.2 · p. 5, Figure 5 · p. 5
  • The defense comparison reports that DeContext removes context identity while preserving visual quality, whereas prior T2I and I2I attacks introduce artifacts.The figure contrasts visual identity removal and output quality across defenses.Source: Figure 6 · p. 6

5. Experiments

Experiments evaluate DeContext on face images using identity-protection and image-quality measures, comparing it with adapted defenses. Results report identity removal with preserved quality on FLUX.1-Kontext, robustness across prompts and ablations, and effectiveness on Step1X-Edit.

5.1. Experimental Setup

  • Experiments use 50-identity subsets of VGGFace2 and CelebA-HQ, adapt Anti-DB, AdvDM, CAAT, and FaceLock defenses, and evaluate identity alteration and image quality using SER-FQA, BRISQUE, and FID.For I2I tasks, one image per identity serves as context, and CLIP-I additionally measures source-to-generated-image semantic similarity. Higher SER-FQA and lower BRISQUE and FID indicate better image quality.Source: §5.1 · pp. 6–7

5.2. Main Results

  • Across four additional editing prompts, ISM and CLIP-I decrease by 73% and 36% on average, while BRISQUE and SER-FIQ vary under 10% in most cases.The authors note that FID can be relatively higher because its real-face embeddings align more closely with clean outputs than perturbed ones.Source: §5.2 · pp. 7–8, Table 2 · p. 7

5.3. Ablation studies

  • A larger attack budget improves identity detachment but adds artifacts, while perturbing early-to-mid single blocks gives the best detachment-quality trade-off and efficiency.The default budget is η = 0.1; the block study compares all double blocks, the first 25 single blocks, and the last 25 single blocks.Source: §5.3 · p. 8, Table 4 · p. 8, Table 5 · p. 8

5.4. Extension Results on Step1X-Edit

  • On Step1X-Edit, DeContext achieves comparable identity-removal and image-quality performance to FLUX.1-Kontext, with average ISM reduction exceeding 80%.Qualitative results show realistic outputs across neutral and appearance-changing instructions.Source: §5.4 · p. 8, Table 3 · p. 8, Figure 7 · p. 8

6. Conclusion and Future Work

The paper concludes that DeContext disrupts context propagation through multimodal attention and reports effective protection with preserved quality on FLUX.1-Kontext and Step1X-Edit. Future work targets greater efficiency on large models and stronger robustness and transferability in black-box settings.

6. Conclusion and Future Work

  • Experiments on FLUX.1-Kontext and Step1X-Edit show DeContext detaching contextual information from the input while preserving high quality.The paper describes DeContext as the first defense for DiT-based in-context image editing.Source: §6 · p. 8
  • Future work aims to improve efficiency on large models and enhance robustness and transferability in black-box settings.Source: §6 · p. 8

Appendix

  • The supplement covers design, implementation, extension experiments, analysis, a user-preference study, limitations, and visual results.Source: Appendix · p. 12

A. Implementation Details

The appendix contrasts a reconstruction-loss attack with DeContext’s attention-aware perturbation strategy. It describes the latter’s concentrated optimization settings and the diverse prompt pool used to promote generalization.

A.1. Diff-PGD Baseline

  • Diff-PGD perturbs the condition image by maximizing flow-matching reconstruction loss under an ℓ∞ constraint, but fails to produce meaningful adversarial effects on DiT-based I2I models.Its objective is to prevent the desired output, yet the reported failure persists even when targeting early denoising stages.Source: §A.1 · p. 12

A.2. DeContext

  • DeContext suppresses context-image contributions to cross-attention; its concentrated setting attacks single blocks 0–25 at timesteps 980–1000.At those high-noise steps, it approximates the target image with random Gaussian noise, avoiding paired target images; the update uses 1 − rctx.Source: §A.2 · p. 12

A.3. Prompt Pool

  • The 60-instruction prompt pool spans six groups, including expressions, accessories, posture, style, scenes, and combined edits.The pool is designed to promote prompt-agnostic generalization during adversarial optimization.Source: §A.3 · p. 12

B. Extension Experiments on Items

Item-level experiments test DeContext beyond human portraits using 50 images from Subject200K and six prompts. The reported similarity reductions indicate contextual divergence, with visual examples described as a reasonable defense against misuse.

C. User Study

A 20-participant study compared DeContext with four protection methods across four dimensions. DeContext received more first-place rankings than all four baselines combined.

C. User Study

  • Twenty participants compared DeContext with Anti-Dreambooth, CAAT, AdvDM, and FaceLock.Each participant was shown eight generated images per method.Source: C. User Study · p. 14
  • Participants ranked outputs for identity detachment, prompt adherence, image quality, and overall protection preference.The study used images generated from the same input prompt and protected image.Source: C. User Study · p. 14
  • DeContext’s total first-place rankings exceeded the combined first-place selections of all four baselines.The study compared generated images on identity detachment, prompt adherence, image quality, and overall protection preference.Source: Figure 8 · p. 14, C. User Study · p. 14

D. Discussion and Future Work

DeContext is effective when the source image strongly guides a simple edit, but its impact diminishes for complex, language-driven scene changes. The paper suggests selectively reducing attention to sensitive elements as future work.

D. Discussion and Future Work

  • DeContext’s effectiveness diminishes when language drives complex scene modifications rather than the source image.In these cases, the model already gives the visual context little attention, so further weakening cross-attention has only minor effects.Source: D. Discussion and Future Work · p. 14
  • Under substantial scene transformations, object identity and overall composition can remain largely intact despite DeContext.The described failure case showed only slight resizing of a person.Source: D. Discussion and Future Work · p. 14
  • Future work could replace uniform context suppression with selective attention reduction focused on sensitive elements.The paper identifies significant scene alterations as a setting where this may better limit those elements’ influence.Source: D. Discussion and Future Work · p. 14

E. Additional Qualitative Results

Additional visualizations show DeContext’s defense on human portraits and item images.

E. Additional Qualitative Results

Loading 2512.16625v2…