PaperlayerView exact revision on arXiv

Source-linked AI summary

TripleFlow: Training-Free Video Object Removal by Bridging Residual Editing and Native Generation

Songhe Wang, Lifu Wei, Shuolin Xu, Charles A. Kamhoua, David Miller

arXiv:2609.39157v2cs.CV

TL;DR

Video object removal must infer a specific occluded background from context, but existing editing can suppress objects without fully reconstructing what lies behind them. TRIPLEFLOW coordinates source, residual, and synthesis flows, feeding generated background into the editing trajectory throughout sampling. Across five benchmarks, it is reported to outperform baselines in reconstruction fidelity, temporal consistency, and removal of associated physical effects.

  • Problem

    Removal prompts specify what to erase but not what to generate, leaving the model to infer the specific hidden background from surrounding video context.

    Source: §1 · pp. 2–3

  • Method

    TRIPLEFLOW coordinates source, residual, and synthesis flows, reusing a target prediction and feeding synthesized background into the editing trajectory at every step.

    Source: §1 · pp. 2–3

  • Results

    Across five benchmarks, TRIPLEFLOW significantly outperforms baselines in reconstruction fidelity, temporal consistency, and natural elimination of associated physical effects.

    Source: §5 · p. 9

  • Takeaways & Limitations

    Ablations indicate that joint flow coordination is essential for eliminating ghosting and robustly reconstructing the occluded background.

    Source: §1 · p. 3

Abstract

from arXiv · show

Video object removal presents a uniquely difficult editing challenge. Because a removal prompt specifies only what to erase rather than what to generate, the model must infer and reconstruct a highly specific occluded background entirely from the surrounding context. Existing training-free methods struggle with this because their editing mechanisms act primarily as localized erasers. They fail to actively synthesize the missing background details and often leave behind ghosting artifacts. To solve this, we propose TripleFlow, a training-free framework that tightly couples erasure and generation. It coordinates a source flow, a residual flow, and a synthesis flow throughout the entire process. By reusing a single target prediction, the residual flow isolates and suppresses the object, while the synthesis flow independently reconstructs the occluded background. Crucially, TripleFlow injects this newly synthesized background back into the editing trajectory at every step. This continuous feedback loop ensures that the generated structures actively guide the removal process, achieving seamless completion that is spatiotemporally consistent with the unedited scene. Extensive evaluations across five challenging benchmarks demonstrate that TripleFlow establishes a new state-of-the-art, significantly outperforming existing baselines in both reconstruction fidelity and temporal consistency.

1 INTRODUCTION

Video removal must reconstruct a specific hidden background from surrounding context, unlike replacement, which has an explicit prompt for what to generate. TRIPLEFLOW couples erasure and synthesis through continuous feedback, and is reported to outperform existing training-free baselines across five benchmarks.

1 INTRODUCTION

  • Unlike semantic replacement, removal provides no description of the missing surface, so the model must infer the specific background from spatial and temporal scene context.Unsuccessful reconstruction exposes errors in geometry, texture, and shading.Source: §1 · pp. 2–3
  • Existing residual editing can suppress the foreground but may leave ghosting or blur because velocity subtraction does not continuously generate the hidden background.A final-step switch to target-only generation disconnects synthesis from the editing trajectory.Source: §1 · p. 2
  • TRIPLEFLOW coordinates source, residual, and synthesis flows, reusing one target-velocity prediction and feeding synthesized background into the residual trajectory at every noise step.This feedback makes the generated background guide the ongoing edit while the source flow anchors it to the original video.Source: §1 · pp. 2–3
  • Across five challenging benchmarks, TRIPLEFLOW achieves state-of-the-art performance and outperforms existing training-free baselines; ablations support complementary roles for its flows.The framework also removes associated physical effects such as shadows and reflections.Source: §1 · p. 3

2 RELATED WORK

Related work spans video inpainting, training-free object removal, and flow-based generative editing. These approaches reconstruct masked regions, suppress object effects, or modify generative trajectories for editing.

Video inpainting and object removal

  • Video inpainting propagates spatial and temporal features, while recent supervised methods use diffusion priors for fine-grained reconstruction and paired training to remove shadows and reflections.These methods emphasize faithful background reconstruction alongside removal of the target's visual influence.Source: §2 · p. 3

Training-free video object removal

  • Training-free removal methods use pretrained generators with tracking, attention, inversion, or feature-retention controls to suppress object identity and associated effects.Examples include OmnimatteZero and ObjectWIPER.Source: §2 · p. 3

Flow-based generative editing

  • Flow-based editing methods use velocity predictions for inversion-free source-relative edits, while regional attention, soft localization, and temporal mask unions restrict edit regions.The cited flow-based methods are described as excelling at semantic replacement.Source: §2 · p. 3

3 METHOD

The method frames removal as preserving visible scene content and motion while reconstructing a plausible background, then couples source-referenced editing with a synthesis flow that feeds generated content back into later steps.

3.1 PROBLEM SETUP

  • Removal targets masked objects and effects while preserving the visible scene and motion; a clean first-frame reference supplies the desired background appearance.The reference can come from an unoccluded frame or an off-the-shelf image model.Source: §3.1 · p. 4
  • TripleFlow edits at inference time with a frozen generator and VAE, using a fixed-noise analytic source path rather than numerical inversion.The source path provides a reference at each noise level for comparing source and target velocities.Source: §3.1 · p. 4, §3.2 · p. 4

3.2 TRIPLEFLOW DYNAMICS

  • A shared target velocity drives source-relative residual editing and a synthesis state that independently accumulates background content.The source velocity and soft edit weight help preserve visible structure and motion while limiting the residual update near the removal boundary.Source: Figure 3 · p. 5, §3.2 · p. 5
  • Each step blends residual and synthesis estimates, increasing synthesis weight toward the clean endpoint so the combined state guides the next target prediction.Early steps favor the residual estimate; later steps give more weight to background synthesis.Source: Figure 3 · p. 5, §3.2 · p. 6
  • Frame-wise object masks define a hard edit support and a soft boundary weight, while a clean first-frame reference guides target-velocity evaluations.The reference is made from the source first frame and retrieved frames that show the hidden background.Source: §3.2 · p. 6
  • Target-attention control scales masked keys and biases masked query-key pairs to limit object-feature retrieval during early sampling.The same controlled target prediction is shared by the residual and synthesis updates.Source: §3.2 · pp. 6–7

4 EXPERIMENTS

Experiments compare TRIPLEFLOW with training-free editors and a frame-wise trained remover across four benchmarks, including paired and in-the-wild data. Results show stronger reconstruction and removal, with qualitative comparisons and ablations examining scene preservation and flow roles.

Datasets and Baselines

  • The evaluation spans DAVIS, WIPER-Bench, ROSE, and PROVE, including paired-ground-truth subsets and in-the-wild videos without paired references.Comparisons include three training-free video editors and a trained image remover applied frame by frame.Source: §4 · p. 7

Qualitative Evaluation

  • TRIPLEFLOW removes object-associated reflections without effect-specific training, completing regions consistently with the bare floor or empty water.The paper attributes this to separate editing and synthesis flows, with surrounding residual content guiding background propagation.Source: §4 · p. 7

Quantitative Evaluation

  • TRIPLEFLOW achieves the best mean CORE across five benchmarks and the best PSNR, SSIM, and LPIPS on both paired benchmarks.Its PSNR exceeds the strongest competing results by nearly 4 dB on PROVE-M and 2 dB on ROSE.Source: §4 · p. 7, Table 1 · p. 8, Table 2 · p. 8
  • On ten showcase videos, TRIPLEFLOW receives a 56.7% first-place preference rate, versus 23.3% for runner-up OmnimatteZero.On pooled PROVE-M and ROSE results, it has the lowest masked-region LPIPS and highest outside-mask PSNR, and the best mean tLP and tOF scores.Source: §4 · p. 7, Figure 4 · p. 8

Qualitative Evaluation

  • TRIPLEFLOW reconstructs backgrounds revealed later in videos while preserving their placement under camera motion, unlike baselines that alter or fabricate scene structures.Examples include a gallery painting, bookcase contents, and warehouse shelving.Source: §4 · p. 7, Figure 5 · p. 8, Figure 6 · p. 9

Ablation Study

  • Ablating source flow, residual flow, synthesis flow, or editing control produces distinct artifacts or less reliable completion than the full model.The full model consistently outperforms all four ablations in the reported quantitative and qualitative results.Source: §4 · p. 9, Figure 7 · p. 9

5 CONCLUSION

TRIPLEFLOW coordinates source, residual, and synthesis flows in a training-free framework, injecting synthesized background into the residual trajectory. Across five benchmarks, it outperforms baselines in reconstruction fidelity, temporal consistency, and removal of associated physical effects.

Conclusion

  • TRIPLEFLOW coordinates source, residual, and synthesis flows within a single diffusion backbone to couple target erasure with active scene reconstruction.Synthesized background is injected into the residual trajectory.Source: §5 · p. 9
  • Evaluations across five benchmarks show TRIPLEFLOW significantly outperforms baselines in reconstruction fidelity, temporal consistency, and natural elimination of associated physical effects.Source: §5 · p. 9

Reproducibility

  • The paper provides its core implementation, inference configuration, runnable example, and video results to facilitate reproduction and inspection of temporal behavior.Source: REPRODUCIBILITY STATEMENT · p. 10

A.1 BACKBONE AND SAMPLING

The implementation uses a frozen Wan2.2-TI2V-5B backbone and jointly processes each video. Appearance and spatial conditioning use a generated clean first frame, prepared masks, and localized attention control.

A.1 BACKBONE AND SAMPLING

  • Inference uses frozen Wan2.2-TI2V-5B components with 40 sampling steps, timestep shift 12, and source and target guidance scales of 3.5 and 5.0.A single shared Gaussian noise realization defines the source path; numerical inversion and multi-noise averaging are not used.Source: §A.1 · p. 14
  • Each step computes two guided velocities, while synthesis reuses the target velocity without additional backbone evaluations.The implementation requires four transformer forward passes for conditional and negative-prompt evaluations; synthesis influences the next query through Zk+1.Source: §A.1 · p. 14
  • Videos are processed jointly, with padding added only as required by the causal VAE and discarded after decoding; original frames and timing are retained.Spatial dimensions are aligned to backbone latent and patch grids, then outputs are restored to source dimensions.Source: §A.1 · p. 14

A.2 APPEARANCE AND SPATIAL CONDITIONING

  • The clean first frame is generated from the original frame, its target mask, and later views; masks include targets and associated effects, with manual correction when needed.The generated image alone conditions the video model; latent masks are temporally unioned, dilated for support, and used to preserve the source outside the edit region. Target-branch regional attention control is active only during the first 10 sampling steps.Source: §A.2 · pp. 14–15

A.3 BASELINE IMPLEMENTATIONS

The baseline implementations use different backbones, conditioning, and inference configurations. Runtime measurements combine heterogeneous hardware and timing procedures, so they are not controlled end-to-end comparisons.

OmnimatteZero

  • OmnimatteZero uses released LTX-Video-0.9.7 with 30 inference steps and receives source video and masks, but not generated clean first frames.Source: §A.3 · p. 15

Object-WIPER

  • Object-WIPER uses HunyuanVideo with rectified-flow inversion and denoising, source and edited-scene captions, and a prepared union mask.Its configuration retains effect localization, foreground noise reinitialization, and background value-feature reuse.Source: §A.3 · p. 15

ContextFlow

  • ContextFlow quality results use Wan2.1-I2V-14B with inversion and editing intervals, while its clean first frame is generated independently with MagicQuill.It receives neither the TRIPLEFLOW-generated first frame nor later reference images; the subsequent video stage has no explicit object mask.Source: §A.3 · p. 15

OmniEraser

  • OmniEraser applies trained FLUX.1-dev removal weights independently to each frame, without shared temporal state or a generated first-frame condition.Its runs use original benchmark masks without manual repairs, effect expansion, or dilation.Source: §A.3 · p. 15

Execution and runtime configurations

  • The runtime plot compares recorded implementations rather than controlled, same-hardware end-to-end runtimes.Hardware, model-loading and export inclusion, and timing probes differ across methods; first-frame and mask preparation are excluded from several reported elapsed times.Source: §A.3 · p. 15

A.4 HUMAN EVALUATION

The human evaluation compares five edited results across ten selected showcase videos, asking viewers to rank them by removal, background preservation, and temporal coherence. Its preference rates describe this selected set, not a random sample of public benchmarks.

A.4 HUMAN EVALUATION

  • The study used ten selected synthetic showcase videos, each with the source and five edited results, for 50 edited videos total.Displayed clips contained 41 frames at 16 fps and 832 × 480 resolution, including the first frame.Source: §A.4 · p. 15
  • The preference results apply only to the selected showcase cases, which were not a random sample from the public benchmarks.The reported snapshot contains nine completed anonymous questionnaires; respondent identities were not verified.Source: §A.4 · p. 16
  • Viewers ranked five anonymized methods from best to worst after watching complete clips, jointly considering removal completeness, background naturalness and preservation, and temporal coherence.Case order and method-to-label assignments were randomized; the instructions also specified the target and any effects to remove and the content and camera motion to preserve.Source: §A.4 · p. 16
  • 56.7% of case-level rankings placed TRIPLEFLOW first, versus 23.3% for OmnimatteZero, corresponding to 51 and 21 first-place rankings.The 200 rankings include repeated judgments from the same questionnaire and are not 200 independent respondents.Source: §A.4 · p. 16

B ADDITIONAL ABLATION RESULTS

The additional ablation materials describe fixed-subset evaluation and qualitative comparisons of the full model against variants that omit individual flows or editing control. The supplied passages specify the evaluation design but do not report ablation outcomes.

B ADDITIONAL ABLATION RESULTS

  • Table 3 reports N7 ablations on the same 16 PROVE-M and 12 ROSE cases, with per-video frame averages then equal averaging across cases.It lists PSNR, SSIM, LPIPS, hole-region PSNR, and paired CORE object-removal and aftereffect scores; higher is better except for LPIPS.Source: Table 3 · p. 17
  • Figure 8 compares input, full model, and variants without the source flow, residual flow, synthesis flow, or editing control, with red boxes marking removal targets.The qualitative ablations are on PROVE-M and ROSE.Source: Figure 8 · p. 18

C ADDITIONAL QUALITATIVE RESULTS

The additional qualitative results present requested object-removal examples and frame sequences, followed by selected five-method comparisons. The passages describe the examples and their presentation, not aggregate performance findings.

C ADDITIONAL QUALITATIVE RESULTS

  • The examples show input video above the TRIPLEFLOW result, with six corresponding frames ordered from the clip’s first to last frame.The ten examples span synthetic scenes, captured videos, and existing benchmarks.Source: C ADDITIONAL QUALITATIVE RESULTS · p. 19
  • The additional examples include removing an egret, shoes in front of a mirror, a coffee machine, a cyclist and bicycle, a visitor, and a horse.The figure captions identify the requested targets but do not state comparative outcomes.Source: Figure 10 · p. 19, Figure 11 · p. 19, Figure 12 · p. 20

D ADDITIONAL QUALITATIVE COMPARISONS

  • The selected comparisons show five methods across six corresponding frames per video, including the first and last frames; they are qualitative examples rather than an aggregate evaluation.The ContextFlow rows use stored 5B runs with independently generated MagicQuill first frames, and OmniEraser uses 512 × 512 inputs.Source: D ADDITIONAL QUALITATIVE COMPARISONS · p. 22

Related papers

Loading 2609.39157v2…