PaperlayerView exact revision on arXiv

Source-linked AI summary

FlowAnchor: Stabilizing the Editing Signal for Inversion-Free Video Editing

Ze Chen, Lan Chen, Yuanhang Li, Qi Mao

arXiv:2604.22586v1cs.CV

TL;DR

Inversion-free video editing remains unreliable in multi-object scenes and longer videos because its editing signal becomes spatially imprecise and weaker with length. FlowAnchor addresses this with spatial refinement and adaptive magnitude modulation, and reports consistently stronger editing across diverse scenarios.

  • Problem

    Inversion-free editing is efficient and structure-preserving for images, but extending it to videos is challenged by unstable editing signals in high-dimensional latent spaces.

  • Method

    FlowAnchor is a training-free framework that uses Spatial-aware Attention Refinement and Adaptive Magnitude Modulation to anchor editing location and strength.

  • Results

    FlowAnchor consistently outperforms existing methods across diverse editing scenarios, including text alignment, visual fidelity, temporal consistency, and computational efficiency.

  • Takeaways & Limitations

    Explicitly stabilizing the editing signal advances inversion-free flow-based video editing toward more faithful and temporally consistent edits.

  • Takeaways & Limitations

    FlowAnchor still struggles with global style transformations and substantial motion changes inherited from the inversion-free paradigm.

Abstract

from arXiv · show

We propose FlowAnchor, a training-free framework for stable and efficient inversion-free, flow-based video editing. Inversion-free editing methods have recently shown impressive efficiency and structure preservation in images by directly steering the sampling trajectory with an editing signal. However, extending this paradigm to videos remains challenging, often failing in multi-object scenes or with increased frame counts. We identify the root cause as the instability of the editing signal in high-dimensional video latent spaces, which arises from imprecise spatial localization and length-induced magnitude attenuation. To overcome this challenge, FlowAnchor explicitly anchors both where to edit and how strongly to edit. It introduces Spatial-aware Attention Refinement, which enforces consistent alignment between textual guidance and spatial regions, and Adaptive Magnitude Modulation, which adaptively preserves sufficient editing strength. Together, these mechanisms stabilize the editing signal and guide the flow-based evolution toward the desired target distribution. Extensive experiments demonstrate that FlowAnchor achieves more faithful, temporally coherent, and computationally efficient video editing across challenging multi-object and fast-motion scenarios. The project page is available at https://cuc-mipg.github.io/FlowAnchor.github.io/.

1 Introduction

FlowAnchor addresses instability that causes inversion-free video editing to fail in multi-object scenes and longer videos. It anchors both spatial localization and editing strength to stabilize the editing trajectory.

  • Inversion-based video editing is computationally expensive and can accumulate reconstruction errors that cause temporal drift.
  • Inversion-free video editing preserves speed and structure in images but degrades when videos contain multiple objects or more frames.Naive video adaptations treat frames as image batches, limiting reliable editing in these settings.
  • Editing-signal instability has two failure modes: imprecise localization and length-induced magnitude attenuation.The signal is defined as the difference between source- and target-conditioned velocity fields and steers the trajectory toward the target distribution.
  • FlowAnchor uses Spatial-aware Attention Refinement to align textual guidance with intended regions and Adaptive Magnitude Modulation to preserve editing strength.Together, these mechanisms explicitly anchor where and how strongly to edit without training.
  • FlowAnchor is evaluated on FiVE-Bench and Anchor-Bench, with reported gains in text alignment, fidelity, temporal consistency, and computational efficiency.Anchor-Bench emphasizes multi-object and fast-motion localized video editing.

2 Related Work

Video diffusion models have advanced temporal modeling, while inversion-free editing avoids costly inversion but remains unstable when extended from images to videos. FlowAnchor targets the editing signal as the primary bottleneck.

  • Early text-to-video models inflate 2D U-Nets but struggle with long-range temporal coherence because of limited temporal awareness.
  • Recent transformer-based video diffusion models use full 3D spatiotemporal attention to jointly model appearance, motion, and scene dynamics.
  • Inversion-free image editing constructs a direct source-to-target trajectory, but naive video extensions often produce misaligned results.
  • Existing concurrent video-editing methods either overlook editing-signal instability or rely on costly auxiliary conditions.

3 Method

FlowAnchor stabilizes inversion-free flow-based video editing by anchoring both the spatial location and magnitude of the editing signal. It combines attention refinement for semantic localization with adaptive amplification for length-induced attenuation.

  • Rectified Flow and Inversion-Free Editing: Rectified Flow defines text-conditioned velocity fields and enables inversion-free editing through a velocity-difference signal guiding the trajectory toward the target distribution.FlowEdit constructs a direct source-to-target path using source- and target-conditioned states.
  • Motivation: Naive video adaptation destabilizes the editing signal through spatial misalignment and magnitude attenuation as frame count increases.Localization errors cause semantic leakage, while weakened magnitude can fail to drive the intended edit in longer sequences.
  • Spatial-aware Attention Refinement: The refined attention produces a localized editing signal that captures semantic variation in the target region and supports subsequent magnitude anchoring.Joint modulation provides an explicit anchor for where to edit.
  • Spatial-aware Attention Refinement: Spatial-aware Attention Refinement modulates cross-attention using target text tokens and editing masks to sharpen semantic focus and enforce spatio-temporal consistency.Text-token modulation separates target from non-target responses, while spatio-temporal modulation regulates target-token attention across the video.
  • Evaluation: The framework is evaluated against baselines on two benchmarks, with qualitative comparisons addressing localization, effect quality, and temporal consistency.Table 1 reports quantitative comparisons, while the qualitative figure compares editing outcomes across baselines.
  • Adaptive Magnitude Modulation: Adaptive Magnitude Modulation increases reinforcement with video length and selectively amplifies high-contrast signal positions.The frame-adaptive factor is anchored at the model’s default length, while the contrast map acts as a soft importance mask for meaningful editing regions.

4 Experiments

Experiments evaluate FlowAnchor on FiVE-Bench and Anchor-Bench against diverse baselines using automatic metrics, human preferences, ablations, efficiency comparisons, and robustness tests. The results show strong localized alignment, fidelity, temporal coherence, efficiency, and tolerance to imprecise masks, while global style and substantial motion edits remain challenging.

  • Experimental Setup: FlowAnchor uses Wan2.1-T2V-1.3B with 25 inference steps, skips the first two steps, applies SAR early, and applies AMM at every denoising step.The default settings are τ=0.6T, β1=β2=0.3, γ=1.0, and F0=21.
  • Experimental Setup: Experiments compare FlowAnchor with seven state-of-the-art methods across T2I-based, inversion-based flow, and inversion-free flow categories.Wan-Edit+Mask is additionally implemented by integrating masks into Wan-Edit.
  • Quantitative Results: FlowAnchor achieves the highest L.CLIP-T on both benchmarks while also obtaining the best M.PSNR, L.DINO, and CLIP-F scores.These results indicate strong localized alignment, source fidelity, and temporal coherence.
  • Human and Qualitative Evaluation: FlowAnchor is consistently preferred by 20 participants across text alignment, fidelity, temporal consistency, and overall preference.Qualitative comparisons also show accurate alignment, preserved background fidelity, and temporal consistency under fast and large motions.
  • Ablation Study: Removing either SAR constraint degrades localization and CLIP-T, while removing AMM substantially reduces signal magnitude and editing effects.The selected modulation strengths balance editing guidance against fidelity and structural distortion.
  • Efficiency and Robustness: FlowAnchor achieves the lowest inference time while maintaining competitive GPU memory usage, and remains visually consistent across tight masks, scribbles, and coarse boxes.Its mask tolerance follows from using masks as early spatial anchors rather than for later detail generation.
  • Limitations: The method still struggles with global style transformations and substantial motion changes inherited from the inversion-free paradigm.These limitations are identified as directions for future work.

5 Conclusion

The conclusion presents FlowAnchor as a training-free framework that stabilizes editing signals for inversion-free flow-based video editing. It attributes improved editing to spatial anchoring and adaptive strengthening, supported by qualitative and quantitative results across diverse scenarios.

  • Conclusion: FlowAnchor addresses editing-signal instability caused by imprecise localization and weakened magnitude in inversion-free video editing.These issues can produce distorted editing trajectories and degraded results.
  • Conclusion: Spatial-aware Attention Refinement and Adaptive Magnitude Modulation jointly spatially anchor and adaptively strengthen the editing signal.Together, they enable a stable editing trajectory.
  • Conclusion: Qualitative and quantitative experiments show that FlowAnchor consistently outperforms existing methods across diverse editing scenarios.The conclusion frames this as an advance for inversion-free video editing.

B.1 SAR Implementation

The implementation describes SAR as early-stage cross-attention refinement and AMM as frame-adaptive signal rescaling. Both mechanisms use bounded operations to preserve numerical stability while anchoring localization and compensating for temporal attenuation.

  • SAR: SAR is applied to all 30 cross-attention layers during early denoising, with β1=β2=0.3 and τ=0.6T.It modulates cross-attention logits before softmax.
  • SAR: SAR first strengthens target-token responses inside the spatial mask and then regulates target-token responses across spatio-temporal positions.The refined logits are passed through softmax to produce the final normalized attention map.
  • SAR: SAR uses convex interpolations that reshape relative logit contrast without introducing values outside the original range.Softmax normalization therefore remains a valid probability distribution.
  • AMM: AMM constructs a sample-wise normalized dynamic contrast map by channel averaging and min-max normalization over spatio-temporal positions.The map lies in [0,1] and is broadcast across channels.
  • AMM: AMM rescales the editing signal within a controlled range rather than causing unbounded amplification, using γ=1.0.The final modulation uses element-wise multiplication with channel broadcasting.

C.1 Dataset

Anchor-Bench is a localized video-editing benchmark designed for realistic multi-object scenes, cluttered backgrounds, and fast motion. It provides prompts, propagated edit masks, and full-sequence metrics covering alignment, fidelity, and temporal consistency.

  • Benchmark Design: Anchor-Bench contains 74 text-video editing pairs collected from real-world videos with multiple objects, cluttered backgrounds, and fast motion.Videos contain up to 81 frames at 480p resolution.
  • Editing Categories: The benchmark covers color editing, material editing, and object replacement, including rigid and non-rigid objects.Each source video has one source prompt and multiple target prompts for localized editing instructions.
  • Annotations: Each target prompt includes an edit-mask sequence annotated on the first frame and propagated through subsequent frames using optical flow.The masks support localized evaluation of the intended editing region.
  • Evaluation Metrics: L.CLIP-T evaluates the cropped masked region against a local target phrase containing only the edited semantics.Global CLIP-T instead measures alignment with the full target prompt.
  • Evaluation Metrics: L.DINO measures local structure preservation, M.PSNR evaluates reconstruction quality in unedited regions, and CLIP-F and Warp-Err assess temporal consistency.Lower Warp-Err indicates better temporal consistency.
  • Evaluation Protocol: Unlike FiVE-Bench, Anchor-Bench evaluates frame-wise metrics on the full video sequence rather than sparsely sampled frames.This provides a more complete assessment of editing quality and temporal consistency throughout each video.

D Baseline Implementation Details

The evaluation compares FlowAnchor with several video-editing baselines under shared implementation and mask settings. The experiments also examine localization limitations, hyperparameter effects, and comparisons with VACE.

  • Baseline comparison: FlowAnchor is compared with TokenFlow, VideoGrain, RF-Solver-Edit, UniEdit-Flow, Wan-Edit, and FlowDirector using official implementations and default settings.All quantitative results use the same evaluation protocol and hardware setting.
  • Mask-guided baseline: Wan-Edit+Mask adds latent blending between edited and source latents after each editing update to provide a mask-guided baseline.The benchmark mask is resized to the latent resolution at each timestep.
  • Localization limitations: Attention-based localization can be inaccurate in complex videos, especially multi-object scenes and fast motion, producing ambiguous or drifting editing regions.This limitation is reported for FlowDirector and Wan-Edit-style attention maps.
  • FlowAnchor design: FlowAnchor operates on internal representations through SAR and AMM, rather than relying solely on explicit masks, to improve localization and editing-signal strength.The comparison with VACE describes FlowAnchor as training-free and adaptable to diverse editing types while maintaining structural stability.

E.1 Hyperparameter Analysis of SAR and AMM

The hyperparameter analysis identifies balanced settings for SAR and AMM and shows that SAR is most useful during an early denoising interval. Extending SAR too far offers no clear benefit and may interfere with detail generation.

  • SAR strength: β1 = β2 = 0.3 balances SAR attention modulation, avoiding insufficient localization at smaller strengths and instability at larger strengths.Increasing SAR strength improves semantic focus, but values beyond the default provide only marginal localization gains while making editing more aggressive.
  • AMM strength: γ = 1.0 provides the best trade-off between AMM editing strength and structural fidelity.Smaller values under-edit, whereas larger values cause over-editing and structural distortion.
  • SAR application range: τ = 0.6T yields the most reliable SAR results, while τ = 0.8T provides insufficient guidance and τ = 0.4T adds no clear gains.SAR mainly anchors the editing region early, leaving later denoising steps to preserve appearance and structural details.

F Robustness to Mask Granularity

FlowAnchor remains effective with hand-drawn, coarse bounding-box, and tight masks, showing robustness to imperfect spatial annotations. Tight masks nevertheless provide the strongest local fidelity, structure preservation, and temporal consistency.

  • Robustness across mask forms: FlowAnchor remains effective across hand-drawn, coarse bounding-box, and tight masks, with only moderate variation under coarser masks.Hand-drawn masks and bounding boxes still achieve competitive CLIP-T and L.CLIP-T scores, indicating that pixel-accurate masks are not required.
  • Effect of mask precision: Tight masks achieve higher L.DINO and M.PSNR than coarse masks and the best CLIP-F score.These results indicate stronger local fidelity, structure preservation, and temporal consistency for tight masks.

G Additional Comparisons with FlowDirector

FlowAnchor avoids FlowDirector’s attention-mask gating by refining attention and adaptively modulating the editing signal, improving localization, background preservation, object structure, and temporal consistency.

  • FlowDirector gates corrected editing flow with a softened mask derived from source and target cross-attention maps.The mask restricts editing to attention-selected regions.
  • FlowDirector’s attention-derived localization can be spatially ambiguous and temporally unstable, causing leakage into backgrounds and damage to source fidelity.These issues worsen in multi-object videos and under fast motion.
  • FlowAnchor refines attention for semantic alignment and localization, then modulates the editing signal content-adaptively instead of using hard spatial gating.The editing signal encodes the semantic difference between target and source conditions.
  • AMM strengthens semantic residuals already encoded in the editing signal, giving larger corrections to high-contrast regions without blindly amplifying weak or irrelevant responses.This reduces background-noise propagation and stabilizes the editing trajectory.
  • Compared with FlowDirector, FlowAnchor achieves more accurate localization, better background and object-structure preservation, and stronger temporal consistency in multi-object and fast-motion scenarios.Additional FiVE-Bench results demonstrate effectiveness across diverse benchmark scenarios.
  • FlowAnchor is also compared with VACE, a training-based inpainting framework that can under-edit text-based localized edits even when the mask covers the entire object.

I Additional Results of FlowAnchor

FlowAnchor supports diverse localized video-editing tasks, remains robust in cluttered multi-object scenes, and preserves temporal coherence during fast and complex motion.

  • FlowAnchor supports localized style transfer, non-rigid object addition, and non-rigid shape editing with fine-grained control.Examples include transforming a dog into a plush dog and adding sunglasses that adapt to motion and geometry.
  • In complex multi-object videos, FlowAnchor selectively edits target regions while preserving other objects, temporal coherence, and structural consistency.
  • Under rapid and complicated motion, FlowAnchor achieves accurate localized editing while preserving temporal coherence.The breakdance example was previously identified as a failure case for IF-V2V.
  • Figure 17 compares FlowAnchor with FlowDirector, while Figures 18–21 present qualitative results across editing types and benchmark scenarios.
Loading 2604.22586v1…