UniVOE: A Removal-Driven Framework for Interaction-Aware Universal Video Object Editing

Introduction

• Video editing must address dynamic object imprints such as shadows, reflections, and contact effects, since leaving or incorrectly synthesizing them causes perceptible artifacts.The paper notes video editing must handle these effects alongside the target object.Source: Introduction · p. 2
• Realistic injection requires predicting absent foreground, plausible motion, and interaction effects, but matched real-world pairs are hard to capture and synthetic data lacks diversity and realism.Ideal pairs share the same scene, lighting, and background dynamics in object-present and object-absent states.Source: Introduction · p. 2
• Removal and injection share paired object-present/object-absent data structure but differ in difficulty: inpainting can bootstrap removal, while injection needs cleaner supervision for appearance, motion, and interaction.This asymmetry motivates strengthening removal and reusing it to derive injection supervision.Source: Introduction · p. 2
• UniVOE transfers inpainting priors to an effect-aware remover, then uses removal-derived video pairs to train injection and reward-based post-training to improve realism.The injection model is trained with flow-matching; its reward model ranks motion quality and naturalness.Source: Introduction · p. 2, Figure 2 · p. 2
• UniVOE’s contributions include adapting removal with paired data to erase interaction effects and using GRAR to align injection results with human preference.GRAR is introduced because pixel-level reconstruction alone does not fully capture perceptual realism.Source: Introduction · p. 2
• UniVOE achieves state-of-the-art performance on object removal and injection, especially in challenging interaction-effect erasing and prediction.It is competitive with strong open-source and commercial systems, and particularly stronger in interaction-effect cases.Source: Introduction · p. 3

Video Object Removal

• Generic inpainters fill masked regions but are not designed to remove object-induced effects outside the mask; effect-aware methods target those effects, while UniVOE reuses removal to supervise injection.EffectErase treats insertion as an auxiliary inverse task to regularize removal.Source: Video Object Removal · p. 3

Video Object Injection

• Video injection must maintain temporal consistency and coherent motion, while many prior methods require users to specify identity, motion tracks, or other controls; UniVOE uses text by default with optional reference images.The passage describes earlier video methods using first-frame editing, reference preservation, or compositing (Ye et al. 2025).Source: Video Object Injection · p. 3

Post-training for Preference Alignment

• Offline preference optimization uses fixed human-labeled preferences, whereas online post-training scores current-policy samples but standard video-editing approaches face a memory wall from reference policies and long-video rollouts.UniVOE’s GRAR follows single-pass reward-weighted regression and discards the reference policy.Source: Post-training for Preference Alignment · p. 3

Method

• The remover and injector share a structure but are trained for opposite object-present/absent transformations.Both are built on Wan2.1-14B.Source: Method · p. 3

Stage 1: Effect-Aware Video Object Removal

• Removal is bootstrapped from video inpainting, then fine-tuned on paired videos to erase object-induced effects beyond the object mask.Inflated masks leave effects such as shadows and reflections outside the masked region, encouraging recovery of the complete object-absent scene and robustness to inaccurate inference masks.Source: Stage 1: Effect-Aware Video Object… · pp. 3–4
• Reference-guided removal prepends an edited first frame as a static object-absent cue, aligned with the video, noise, and mask sequences.The reference image is optional and guides consistent removal throughout the video.Source: Stage 1: Effect-Aware Video Object… · p. 4

Inference Acceleration

• Distilling the remover from 50 sampling steps to 12 yields substantial speed-up with imperceptible quality degradation, enabling large-scale paired-data generation.The distilled 12-step model uses a fixed sigma shift of 5.0.Source: Stage 1: Effect-Aware Video Object… · p. 4

Stage 2: Removal-Driven Injection Pre-training

• The remover produces 69K paired samples, combined with 68K open-source pairs to train injection on 137K video pairs.The injector learns to add objects together with realistic interaction effects such as shadows, reflections, and contact.Source: Stage 2: Removal-Driven Injection… · p. 5

Stage 3: Post-Training with GRAR

• GRAR post-trains injection because pixel-wise supervision misses perceptual quality differences among plausible appearances, dynamics, and interaction patterns.The reward model evaluates object motion and appearance quality and is fine-tuned from Qwen3-VL using professionally annotated rankings.Source: Stage 3: Post-Training with GRAR · p. 5
• GRAR uses signed, group-relative advantages to pull the model toward high-reward rollouts and away from low-reward ones using a single trainable policy.This avoids a separate reference model; the method is motivated by the VRAM burden of full-sequence editing with a 23B-parameter injector.Source: Stage 3: Post-Training with GRAR · p. 5

Comparison on Object Removal

• For glass and oven removals, UniVOE recovers flowing milk and a sunlit pizza with its shadow, while comparison methods fail to recover these effects.Source: Comparison on Object Removal · pp. 6–7, Figure 4 · p. 6
• UniVOE achieves the best objective metrics on both ROSE and VOR for removal.Source: Table 1 · p. 6, Comparison on Object Removal · p. 7

Comparison on Object Injection

• UniVOE outperforms all compared injection algorithms across benchmarks on LPIPS, FVD, DINO-I, and CLIP-P.Source: Table 2 · p. 6, Comparison on Object Injection · p. 7

Scaling up removal data through inpainting

• Inpainting-trained UniVOE- outperforms “w/o inpainting” across all metrics despite using the same total data scale and fewer training pairs.The no-inpainting model may shift overall tone; inpainting data improves background preservation through strict pixel-level correspondence.Source: Ablation Study · p. 7, Table 3 · p. 7, Figure 6 · p. 7

Ablation on post-training stage

• UniVOE stably outperforms the pre-training-only “w/o GRAR” injector across all metrics.Source: Ablation Study · p. 7, Table 4 · p. 7

• UniVOE combines an inpainting-bootstrapped effect-aware remover, removal-derived injection supervision, and reward-based GRAR post-training to achieve state-of-the-art object editing, especially on challenging interaction effects.Source: Conclusion · p. 7

• The gate starts from remover outputs on real object-present videos, where residuals or background corruption can undermine injection supervision.Source: A Video Filtering Gate for Removal… · p. 10
• A dilated object mask separates the target and nearby interaction-effect region from the background fidelity-check region.The target is localized in the first frame, propagated through the clip, and softly dilated.Source: A Video Filtering Gate for Removal… · p. 10
• Pairs pass only when background fidelity outside the mask and removal completeness inside it both meet the gate's criteria.Large unrelated background changes and any surviving target trace cause rejection.Source: A Video Filtering Gate for Removal… · p. 10
• A Qwen3-VL judge checks videos and a focused grid of eight sampled frames; retained removal-derived pairs join filtered open-source pairs in the injection pretraining set.Uncertainty about complete removal leads to rejection.Source: A Video Filtering Gate for Removal… · p. 10

• Each iteration rolls out K candidates per context from the frozen injection rollout policy and scores them with the human-aligned reward model.Source: B Pseudo Algorithm for GRAR · p. 10
• Candidates receive a signed advantage according to whether their score is above or below the group median, bounding updates independently of absolute reward scale.Source: B Pseudo Algorithm for GRAR · p. 10
• The trainable policy is regressed toward or away from each rollout's velocity target, without storing or forwarding a reference policy.This allows full-sequence 97-frame, 480p post-training of the 23B injector within an 80 GB VRAM budget.Source: B Pseudo Algorithm for GRAR · p. 10

C Compare to DiffusionNFT

• GRAR’s post-training uses signed, median-relative rollout advantages to weight flow-matching regression while forwarding only the injector.The group advantages are bounded at ±1 and have zero mean when rewards are i.i.d. from an absolutely continuous distribution.Source: C Compare to DiffusionNFT · p. 11
• A descent step on LGRAR is, to first order, an ascent step on expected reward J.The result assumes a continuously differentiable velocity model and bounded reward; the derivation includes off-policy bias and a variational residual.Source: C Compare to DiffusionNFT · p. 11
• At 1.3B, GRAR and strict DiffusionNFT obtain closely matching quantitative scores and reward curves, with qualitatively indistinguishable injections.The comparison uses matched rollouts, reward model, and schedule.Source: Table 5 · p. 12, C Compare to DiffusionNFT · p. 12
• GRAR’s gradient coincides to first order with DiffusionNFT’s negative-anchored update while dropping its frozen-reference forward pass.The equivalence relies on small steps and synchronizing the rollout policy after each iteration.Source: C Compare to DiffusionNFT · pp. 11–12
• At 23B with full-sequence 97-frame 480p inputs, DiffusionNFT exceeds the 80 GB budget, whereas GRAR remains feasible.DiffusionNFT keeps a frozen reference alongside the trainable model, roughly doubling resident model memory.Source: C Compare to DiffusionNFT · p. 12

D Ablation on Post-training Stage

• Reward-based post-training resolves common color mutation and unrealistic distortion issues in most common scenes.The comparison is between UniVOE and the pre-trained-only “w/o RL” injector.Source: D Ablation on Post-training Stage · p. 12

E More Results

• Additional UniVOE examples cover both object removal and injection and showcase the authors’ reported superior performance and generalization.Corresponding video demos are in the media supplement.Source: E More Results · p. 12

F Limitations and Future Work

• Injection quality is bounded by remover coverage and retained pairs because filtering cannot create supervision the remover never produces.The strict-by-default gate removes pairs with residual objects or corrupted backgrounds.Source: F Limitations and Future Work · p. 12
• GRAR’s post-training ceiling depends on reward-model fidelity, although within-group ranking makes it robust to reward scale and occasional mis-scored rollouts.The authors identify more diverse, preference-aligned reward models as a route to improving reward-guided post-training.Source: F Limitations and Future Work · p. 12
• The study targets object removal and injection on a single 23B video backbone at 97-frame 480p, leaving broader settings for future work.The authors state that neither the removal-driven recipe nor GRAR loss is specific to injection.Source: F Limitations and Future Work · p. 12
• UniVOE is positioned as a paired-data operator for foundation-model SFT, since specialist editors can emit large volumes of object-present/absent and injection pairs.The passage argues manual annotation alone cannot feasibly provide the millions of paired clips needed for native editing.Source: F Limitations and Future Work · p. 13
• Post-training addresses common injection errors including color mutation, subject disappearance or distortion, and semantic inconsistency.The comparison is between results without RL and UniVOE post-training.Source: Figure 9 · p. 14