PaperlayerView exact revision on arXiv

Source-linked AI summary

LazyDrag: Enabling Stable Drag-Based Editing on Multi-Modal Diffusion Transformers via Explicit Correspondence

Zixin Yin, Xili Dai, Duomin Wang, Xianfang Zeng, Lionel M. Ni, Gang Yu, Heung-Yeung Shum

arXiv:2509.12203v3cs.CV

TL;DR

Drag-based editing relies on implicit attention matching that can be unstable, while prior approaches use test-time optimization or weakened inversion. LazyDrag uses an explicit correspondence map to control attention in MM-DiTs, enabling full-strength inversion without test-time optimization. The authors report that it outperforms existing methods on DragBench and supports precise, text-guided editing.

  • Problem

    Implicit attention-based point matching is unreliable in drag-based editing, and prior methods use test-time optimization or weakened inversion, limiting inpainting and text guidance.

    Source: §1 · p. 1

  • Method

    LazyDrag uses a drag-derived explicit correspondence map to drive attention controls in MM-DiTs under full-strength inversion, without test-time optimization.

    Source: §1 · pp. 2–3

  • Results

    LazyDrag significantly outperforms existing methods on DragBench in quantitative metrics and human preference.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The method supports precise drag edits alongside text-guided inpainting and semantically consistent modifications, including under ambiguous instructions.

    Source: §1 · pp. 2–3

    Edit and generation quality depend heavily on the underlying base model.

    Source: §B.5 · p. 19

Abstract

from arXiv · show

The reliance on implicit point matching via attention has become a core bottleneck in drag-based editing, resulting in a fundamental compromise on weakened inversion strength and costly test-time optimization (TTO). This compromise severely limits the generative capabilities of diffusion models, suppressing high-fidelity inpainting and text-guided creation. In this paper, we introduce LazyDrag, the first drag-based image editing method for Multi-Modal Diffusion Transformers, which directly eliminates the reliance on implicit point matching. In concrete terms, our method generates an explicit correspondence map from user drag inputs as a reliable reference to boost the attention control. This reliable reference opens the potential for a stable full-strength inversion process, which is the first in the drag-based editing task. It obviates the necessity for TTO and unlocks the generative capability of models. Therefore, LazyDrag naturally unifies precise geometric control with text guidance, enabling complex edits that were previously out of reach: opening the mouth of a dog and inpainting its interior, generating new objects like a ``tennis ball'', or for ambiguous drags, making context-aware changes like moving a hand into a pocket. Additionally, LazyDrag supports multi-round workflows with simultaneous move and scale operations. Evaluated on the DragBench, our method outperforms baselines in drag accuracy and perceptual quality, as validated by VIEScore and human evaluation. LazyDrag not only establishes new state-of-the-art performance, but also paves a new way to editing paradigms.

1 INTRODUCTION

LazyDrag replaces fragile implicit attention-based point matching with explicit drag-derived correspondence to stabilize full-strength inversion without test-time optimization. It combines precise drag control with text guidance, and the paper reports stronger DragBench performance than existing methods.

1 INTRODUCTION

  • LazyDrag addresses unstable implicit matching, which favors spatially nearby regions over semantically related ones and degrades edits.Prior methods respond with test-time optimization or weakened inversion, which can impair inpainting and text guidance.Source: §1 · p. 1
  • LazyDrag is built on MM-DiTs, whose tighter vision–text fusion is described as improving inversion robustness and the ceiling for attention control.The method applies control across single-stream attention layers without manually selecting layer indices as in U-Nets.Source: §1 · pp. 1–2
  • Figure 1 illustrates dog-mouth inpainting, prompt-guided results under ambiguous drags, and coherent multi-round editing with simultaneous movement and scaling.Figure 2 shows examples at different inversion strengths using the prompt “a red apple in the mouth”.Source: Figure 1 · p. 2, Figure 2 · p. 3
  • An explicit correspondence map derived from drag instructions drives attention controls, stabilizing full-strength inversion without test-time optimization.The authors describe this as the first use of full-strength inversion in drag-based editing with MM-DiTs.Source: §1 · pp. 2–3
  • Coupling the correspondence map with text guidance resolves ambiguous drags and enables natural inpainting and semantically consistent edits.Examples include opening and inpainting a dog's mouth, generating a tennis ball, and guiding a hand into a pocket.Source: §1 · pp. 2–3
  • LazyDrag significantly outperforms existing methods on DragBench in quantitative metrics and human preference.The method does not require test-time optimization.Source: §1 · p. 3

2 RELATED WORK

Prior work spans text-only editing, drag-based spatial control, and methods for drag-based editing in U-Nets and MM-DiTs. LazyDrag positions explicit correspondence-driven attention control as an alternative to implicit matching and the compromises of earlier approaches.

2 RELATED WORK

  • LazyDrag introduces drag-based editing within MM-DiTs, where prior methods described here addressed text editing or identity preservation rather than drag control.Figure 3 depicts its pipeline: correspondence-map generation, attention input control, and attention-output refinement.Source: §2 · p. 3, Figure 3 · p. 4
  • Text-based editing offers flexible prompt control, but text-only methods are limited in spatial precision.Work in MM-DiTs includes DiTCtrl for long video generation, ColorCtrl for light-consistent color edits, and CharaConsist for character identity preservation.Source: §2 · p. 3
  • Most prior drag-based methods use test-time optimization; existing TTO-free approaches are described as having quality or artifact trade-offs.FastDrag is noted for faster inference but lower quality than TTO-based methods, while Inpaint4Drag uses an inpainting model and can produce warping and boundary artifacts.Source: §2 · pp. 3–4
  • Prior inversion-based approaches use low inversion strength, which the authors say degrades inpainting and limits semantic generation.LazyDrag instead uses full-strength inversion and text-guided attention mechanisms.Source: §2 · p. 4

3 METHOD

LazyDrag replaces implicit attention-based point matching with an explicit map derived from drag instructions, then uses the map to control attention in MM-DiTs. Its method combines displacement-field construction, latent initialization, and attention input and output controls.

3 METHOD

  • LazyDrag builds an explicit correspondence map from drag instructions to control attention in MM-DiTs, targeting stable full-strength inversion without test-time optimization.The map drives a two-part attention mechanism for preserving identity and background.Source: §3 · p. 4, §3.3 · p. 6

3.1.3 LAZYDRAG: BRIDGING THE GAP.

  • FastDrag’s latent initialization loses handle-specific cues, while attention-similarity matching in CharaConsist is unstable under full-strength inversion; LazyDrag replaces both with drag-derived correspondence.FastDrag can trade positioning accuracy against naturalness and can cancel opposing drags; CharaConsist requires additional denoising steps for matching.Source: §3.1.1 · p. 5, §3.1.2 · p. 5, §3.1.3 · p. 5

3.2 GENERATING THE EXPLICIT CORRESPONDENCE MAP

  • Winner-takes-all fusion assigns each editable point to its nearest handle, preserving opposing drag magnitudes for edits such as opening a mouth.Drag mode uses elasticity-based displacement, while move mode uses translation and scaling.Source: §3.2 · p. 5
  • The correspondence map defines a source-point match M(x) and weight A(x), then partitions the latent grid into background, destination, inpainting, and transition regions.The updated latent uses matched source noise at destinations, fresh noise in inpainting regions, and source noise in background and transition regions.Source: §3.2 · pp. 5–6
  • Gaussian noise initializes newly exposed inpainting regions, replacing FastDrag’s interpolation to avoid repetitive artifacts and support text-guided inpainting.The passage contrasts this initialization with uniform noise and states that the Gaussian choice aligns with the diffusion prior.Source: §3.2 · p. 6

3.3.1 ATTENTION INPUT CONTROL VIA TOKEN REPLACEMENT AND CONCATENATION

  • At single-stream attention inputs, LazyDrag hard-replaces background tokens with cached originals and concatenates matched source tokens for destination and transition regions.The correspondence-driven signal is intended to preserve identity while blending boundaries smoothly.Source: §3.3.1 · p. 6

3.3.2 ATTENTION OUTPUT REFINEMENT VIA GATED MERGING (ATTN REFINE)

  • At attention outputs, LazyDrag gates blending with map weights, strengthening control at handle points while allowing surrounding regions to relax naturally.The time-dependent factor decays over denoising, and the gated merge avoids extra denoising steps and multi-step latent optimization.Source: §3.3.2 · p. 7

4 EXPERIMENTS

LazyDrag outperforms the compared methods on DragBench while avoiding test-time optimization. The experiments also examine qualitative edit behavior, user preferences, drag versus move modes, and component and timestep choices.

4.1 SETUP

  • DragBench contains 205 images and 349 handle-target pairs; the evaluation compares eight U-Net baselines using MD and VIEScore, omitting IF as potentially misleading.The method is built on FLUX.1 Krea-dev, and the denoising-step count is fixed at 50 for all methods.Source: §4.1 · p. 7

4.2 QUANTITATIVE EVALUATION

  • LazyDrag outperforms existing approaches across DragBench metrics without per-edit test-time optimization.The paper emphasizes drag accuracy and perceptual quality, and reports greater robustness to masks and point pairs than Inpaint4Drag.Source: Table 1 · p. 8, §4.2 · p. 8

4.3 QUALITATIVE EVALUATION

  • Across five DragBench examples, LazyDrag handles arm lifting, vehicle and sofa edits, pocket insertion, and tiger-head rotation while preserving structure and image quality.Other methods introduce artifacts or unintended background changes in several cases; DragText also succeeds on the tiger rotation.Source: §4.3 · pp. 8–9, Figure 4 · p. 8

4.4 USER STUDY

  • LazyDrag was preferred in 61.88% of comparisons by 32 expert participants evaluating 32 randomly sampled cases.Participants judged edit success, naturalness, and background preservation.Source: §4.4 · p. 9, Table 2 · p. 9

4.5 COMPARISON BETWEEN DRAG AND MOVE MODES

  • Move mode tends to preserve identity better, whereas drag mode supports rotations and extensions but may slightly reduce texture detail; both produce reasonable results.These findings illustrate the flexibility of the correspondence map with correspondence-driven preservation.Source: §4.5 · p. 9, Figure 5 · p. 9

4.6 ABLATION STUDY

  • Removing WTA and latent initialization increases MD and slightly lowers PQ and O; removing background preservation further lowers SC and O, while attention-similarity control sharply degrades results.The ablation uses controlled replacements for the removed components; the full method performs best.Source: §4.6 · p. 9, Figure 6 · p. 10, Table 3 · p. 10
  • Increasing the activation timestep improves destination accuracy but may add warping artifacts; lowering it favors naturalness but may vary identity or motion, so experiments use 40.The comparison varies the activation timestep among 20, 40, and 50.Source: §4.6 · pp. 9–10, Figure 7 · p. 10, Table 4 · p. 10

5 CONCLUSION

The conclusion presents LazyDrag as a training-free MM-DiT editing method that replaces implicit point matching with an explicit correspondence map. It reports stable full-strength inversion without TTO and state-of-the-art performance for precise, text-guided edits.

5 CONCLUSION

  • LazyDrag replaces implicit attention-based point matching with an explicit correspondence map that drives attention controls during generation.The conclusion connects this design to stable edits under full-strength inversion without TTO, identity and background preservation, faithful inpainting, and text-guided resolution of ambiguous drags.Source: §5 · p. 10

ETHICS STATEMENT

The ethics statement identifies potential risks from misleading or harmful edited content and societal biases in pretrained models.

ETHICS STATEMENT

  • The authors identify harmful or misleading content and unintended reproduction of pretrained-model biases as ethical risks.They call for responsible use, transparency, user consent, and future bias detection and mitigation.Source: ETHICS STATEMENT · p. 11

REPRODUCIBILITY STATEMENT

The paper describes its reproducibility materials and inference settings, including baseline exclusions and choices made to improve Inpaint4Drag quality rather than speed.

REPRODUCIBILITY STATEMENT

  • The authors state that inference procedures and evaluation settings are described and that source code will be released upon acceptance.The referenced descriptions are in Sections 3 and 4.1 and Appendix A.Source: REPRODUCIBILITY STATEMENT · p. 11

A.1 INFERENCE SETTINGS

  • Inference uses 50 denoising steps, CFG 1, original DragBench resolutions, and a single NVIDIA H800 GPU.Other images are generated at 1024 × 1024; LazyDrag applies correspondence-driven preservation only to FLUX.1 Krea-dev single-stream attention layers.Source: §A.1 · p. 16
  • EasyDrag and CLIPDrag are excluded because their released implementations were unreliable or unreproducible; Inpaint4Drag settings prioritize editing quality over speed.For Inpaint4Drag, the authors remove its LCM LoRA, use 50 denoising steps, and replace its distilled VAE with the original VAE.Source: §A.1 · p. 16

A.2 IMPLEMENTATION DETAILS OF DISPLACEMENT FIELD CALCULATION

LazyDrag constructs displacement fields from drag vectors, assigns each location to a winning handle to avoid conflicting motions, and extends the model to axis-aligned scaling. The accompanying evaluation details specify VIEScore and user-study procedures.

A.2 IMPLEMENTATION DETAILS OF DISPLACEMENT FIELD CALCULATION

  • Each drag is a source-to-target vector whose influence decays with distance and whose displacement follows the applied force.The construction uses a reference circle around the point set to define a stretch factor.Source: §A.2 · p. 16
  • Winner-takes-all blending assigns each point to its nearest handle, producing sharper spatial separation and avoiding interference between opposing drags.The final displacement comes from the winning instruction rather than averaging competing instructions.Source: §A.2 · p. 16
  • An axis-aligned scaling vector extends the displacement model to resizing as well as moving.Source: §A.2 · p. 16

A.3 EVALUATION DETAILS

  • VIEScore uses the same prompts as GEdit-Bench for PQ and O, while SC follows the instruction shown in Figure 14.Scores are collected and calculated with the official GEdit-Bench codebase.Source: §A.3 · p. 17

A.4 USER STUDY DETAILS

  • For a fair comparison, the study randomly selected 32 results for nine comparison methods on Drag-Bench, shuffled their indices, and invited 32 participants with relevant skills to perform interface-guided tasks.Source: §A.4 · p. 17

B.1 MORE RESULTS ON DRAGBENCH

Additional results describe LazyDrag's qualitative behavior, text-guided edits, and U-Net component ablations. They also examine activation-timestep sensitivity and the effects of winner-takes-all blending and latent initialization.

B.1 MORE RESULTS ON DRAGBENCH

  • LazyDrag produces more natural, accurate edits while better preserving background consistency than the compared baselines in additional DragBench examples.Source: §B.1 · p. 17
  • The activation-timestep analysis progressively increases the timestep, while the failure-case figure presents DragBench examples.The supplied figure captions identify the analyses but do not state their outcomes.Source: Figure 9 · p. 18, Figure 10 · p. 18

B.2 EFFECT OF TEXT GUIDANCE

  • With added text prompts, LazyDrag resolves ambiguities in drag instructions and supports more complex, precise guidance than DragText and CLIPDrag.Source: §B.2 · p. 18

B.3 EFFECT WITH U-NETS

  • On U-Net-based FastDrag, winner-takes-all blending improves target localization in complex multi-instruction cases, while random latent initialization reduces repetitive-pattern artifacts.The reported improvements align with MD and SC gains for blending and PQ and O gains for initialization on 97 multi-drag DragBench cases.Source: §B.3 · p. 18, Table 5 · p. 18

B.4 RUNTIME ANALYSIS

The runtime evaluation compares LazyDrag with TTO-required and TTO-free baselines under default and optimized settings. LazyDrag avoids per-edit optimization, with reported runtimes comparable to FastDrag in the optimized setting and faster than DragText in the default setting.

B.4 RUNTIME ANALYSIS

  • The Default setting uses 50 inference steps, full inversion strength, and bfloat16; the Optimized setting uses 20 sampling steps and inversion strength 0.7.The study also benchmarks DragText, FastDrag, and standard text-to-image inference without editing interventions.Source: §B.4 · p. 18
  • 14.10s vs. 27.88s: LazyDrag is faster than DragText in the Default high-quality setting.In the Optimized setting, LazyDrag takes roughly 4.31 seconds, comparable to FastDrag at 4.21s.Source: §B.4 · pp. 18–19, Table 6 · p. 19
  • The MM-DiT backbone accounts primarily for higher memory use and inference time, while inversion is a one-time cost per image and later edits need only map generation and synthesis.The paper notes possible speedups from GPU parallelization, CPU token caching, quantization, and general techniques such as xDiT.Source: §B.4 · p. 19

B.5 LIMITATIONS

The limitations include artifacts from high activation timesteps in multi-drag cases, positional shifts on very small drags, and dependence on the underlying base model. The paper also describes its LLM use and the instruction for evaluating drag-following edits.

B.5 LIMITATIONS

  • Too-high final activation timesteps can cause unnatural artifacts in multi-drag cases, especially when targets overlap; very small drags may shift because of VAE compression and latent patching.Reducing the timestep can make results more natural while retaining reasonable target positions; fine-grained edits remain possible but may shift slightly.Source: §B.5 · p. 19
  • Edit and generation quality depend heavily on the underlying base model.Source: §B.5 · p. 19

C LLM USAGE

  • The authors used an LLM for grammar refinement, VIEScore evaluation, and drafting the user-study web interface.The evaluation instruction asks whether the edited image follows the drag instruction relative to the original.Source: C LLM USAGE · pp. 20, 22

Related papers

Loading 2509.12203v3…