PaperlayerView exact revision on arXiv

Source-linked AI summary

GeoRemover: Removing Objects and Their Causal Visual Artifacts

Zixin Zhu, Haoxiang Li, Xuelu Feng, He Wu, Chunming Qiao, Junsong Yuan

arXiv:2509.18538v2cs.CV

TL;DR

Strict masks can leave shadows and reflections untouched, while loosely aligned editing can be difficult to control. GeoRemover removes objects from geometry before rendering appearance, and reports improved removal quality across benchmarks, including higher artifact-residue IoU than OmniEraser.

  • Problem

    Strict masks can leave shadows and reflections outside the edit region untouched, while loosely aligned editing can be difficult to control and alter unrelated scene content.

    Source: §1 · p. 1

  • Method

    GeoRemover removes the target from depth under mask-aligned supervision, then renders appearance from updated geometry, using preference-guided learning to discourage inserted structures.

    Source: §1 · p. 3

  • Results

    73.76% artifact-residue IoU versus OmniEraser’s 68.29% [9]; the authors also report improved removal quality on two benchmark datasets.

    Source: §1 · p. 3, §3.2 · p. 8

  • Takeaways & Limitations

    The authors report precise structural editing and seamless restoration, especially in challenging cases involving shadows and reflections.

    Source: §5 · p. 9

    Stage 2 may not trigger removal when Stage 1 leaves masked depth nearly unchanged, including with fast motion, translucency, reflective surfaces, occlusions, or low texture.

    Source: §4 · p. 8

Abstract

from arXiv · show

Towards intelligent image editing, object removal should eliminate both the target object and its causal visual artifacts, such as shadows and reflections. However, existing image appearance-based methods either follow strictly mask-aligned training and fail to remove these causal effects which are not explicitly masked, or adopt loosely mask-aligned strategies that lack controllability and may unintentionally over-erase other objects. We identify that these limitations stem from ignoring the causal relationship between an object's geometry presence and its visual effects. To address this limitation, we propose a geometry-aware two-stage framework that decouples object removal into (1) geometry removal and (2) appearance rendering. In the first stage, we remove the object directly from the geometry (e.g., depth) using strictly mask-aligned supervision, enabling structure-aware editing with strong geometric constraints. In the second stage, we render a photorealistic RGB image conditioned on the updated geometry, where causal visual effects are considered implicitly as a result of the modified 3D geometry. To guide learning in the geometry removal stage, we introduce a preference-driven objective based on positive and negative sample pairs, encouraging the model to remove objects as well as their causal visual artifacts while avoiding new structural insertions. Extensive experiments demonstrate that our method achieves state-of-the-art performance in removing both objects and their associated artifacts on two popular benchmarks. The code is available at https://github.com/buxiangzhiren/GeoRemover.

1 Introduction

Strictly mask-aligned removal can leave causal artifacts outside the mask, while loosely aligned editing can be difficult to control. GeoRemover instead removes objects from scene geometry before rendering appearance, so their associated artifacts can be removed without broadening the edit mask.

1 Introduction

  • Strictly mask-aligned methods leave shadows and reflections outside the object mask unaddressed, requiring users to annotate subtle artifacts separately.The paper notes this annotation burden makes the approach neither scalable nor user-friendly.Source: §1 · p. 1
  • Loosely mask-aligned methods lack clear edit boundaries and can remove unrelated content, such as a nearby adult.These methods rely heavily on paired training data, which the paper says hinders controllability.Source: §1 · pp. 1–2
  • Shadows and reflections are consequences of object geometry under specific lighting, motivating removal of the object from scene geometry first.The paper frames geometry presence as the cause and associated visual effects as its consequences.Source: §1 · p. 2
  • GeoRemover first edits scene geometry under strictly mask-aligned supervision, then renders appearance from updated geometry so associated artifacts are removed.The rendering stage is trained on image pairs with and without the object and its causal effects, while preserving nearby unmasked content.Source: §1 · pp. 2–3
  • A preference-guided loss discourages unexpected structural insertions, and experiments report improved removal quality over loosely aligned methods on two benchmark datasets.The framework is presented as addressing the controllability and unintended-alteration issues associated with loosely aligned strategies.Source: §1 · p. 3

2 Method

GeoRemover separates object removal into geometry editing and appearance rendering, aiming to control structural changes while implicitly correcting causal visual artifacts. Geometry removal uses mask-aligned depth editing and preference guidance; geometry-conditioned rendering then synthesizes the output image.

2.1 Problem formulation

  • The framework separates geometry removal from appearance rendering to edit scene structure and synthesize an RGB image consistent with updated geometry.The paper states that geometry-aware rendering can implicitly correct causal visual artifacts.Source: §2.1 · p. 3

2.2 Stage 1: geometry removal

  • Stage 1 edits depth under strictly mask-aligned supervision, since shadows and reflections do not appear in the geometry domain.The masked depth map and object mask condition a diffusion model that predicts edited geometry.Source: §2.2 · pp. 3–4, Figure 2 · p. 4
  • Preference-guided training favors object-free geometry with low depth flow over geometry retaining the object, whose depth flow has sharp discontinuities.The method uses a reward based on flow difference and combines the preference-guided objective with standard diffusion loss.Source: §2.2 · pp. 4–5

2.3 Stage 2: appearance rendering

  • Stage 2 conditions image translation on input and target depth maps, whose difference specifies the structural transformation for removal or insertion.The model takes a masked RGB image and geometry-aware conditions represented as depth maps.Source: §2.3 · pp. 5–6

3 Experiments

Across two benchmarks, GeoRemover achieves the best reported scores on four image-quality and structure metrics. Ablations support the two-stage design and preference supervision, while artifact-region evaluation and qualitative comparisons examine removal of causal effects.

3 Experiments

  • Evaluation uses RORD-Val and RemovalBench, with training data from the 516,705-image RORD dataset.The study reports FID, CMMD, Aesthetic Score, LPIPS, and PSNR.Source: §3 · p. 6

3.1 Comparison with SOTA methods

  • GeoRemover consistently achieves the best FID, CMMD, LPIPS, and PSNR scores across RemovalBench and RORD-Val.The comparison includes strictly and loosely mask-aligned methods.Source: §3.1 · pp. 6–7

3.2 Ablation study and discussion

  • With the same RGB and depth inputs, the two-stage model outperforms the one-stage variant across multiple metrics, though it increases runtime.The one-stage model often produces ambiguous or distorted edits; the authors attribute the gap to explicit geometry-appearance decoupling.Source: §3.2 · p. 7, Figure 4 · p. 8
  • DPO reduces the Insert. rate from 5.08% to 1.48%, indicating fewer wrongly inserted objects in the removal region.The table defines Insert. as the percentage of cases with a new object wrongly inserted into that region.Source: §3.2 · p. 7, Table 2 · p. 7
  • Stage 1 reduces depth error in the masked region, and a Fill-in strategy restores geometric contrast when motion blur causes depth errors and removal failure.The study reports that DPO further improves removal precision, without giving values in the supplied passage.Source: §3.2 · p. 7, Figure 5 · p. 8
  • On CausRem, GeoRemover reaches 73.76% IoU for artifact removal, above OmniEraser’s 68.29%.CausRem evaluates predicted residue regions against annotated artifact masks.Source: §3.2 · pp. 7–8, Table 4 · p. 7
  • Bidirectional rendering improves Stage 2 alignment between refined geometry and synthesized images, with better visual quality and fewer final-output artifacts.The two directions enforce consistency in removal and insertion rendering.Source: Table 5 · p. 7, §3.2 · p. 8

3.3 Qualitative comparison on CausRem

  • On CausRem examples, GeoRemover removes objects and effects while preserving background structure; prior methods leave shadows, blur backgrounds, or distort nearby textures.The comparison names Attentive-Eraser, CLIPAway, and OmniEraser.Source: §3.3 · p. 8

4 Failure case

Stage 2 depends on a detectable depth change in the masked region; unreliable or unchanged depth can prevent removal from being triggered.

4 Failure case

  • When edited depth is nearly indistinguishable from input depth, Stage 2 lacks the geometric contrast needed to trigger removal.Stage 2 identifies targets by differencing the two depth maps.Source: §4 · p. 8
  • Fast motion, translucency, specular or reflective surfaces, occlusions, and low texture can make Stage 1 depth unreliable or unchanged.Source: §4 · p. 8
  • Local Max Depth Fill-in propagates maximum depth from a small neighborhood, such as 10×10 pixels, to restore boundary contrast and support coherent background preservation.Source: §4 · p. 8

5 Conclusion

The conclusion presents a geometry-aware two-stage approach for removing objects and causal artifacts, while the related work positions it between precise structure editing and artifact removal.

5 Conclusion

  • The framework decouples geometry removal from appearance rendering and is reported to outperform existing methods quantitatively and visually, especially for shadows and reflections.Source: §5 · p. 9

A Related work

  • Unlike strictly mask-aligned methods that can leave shadows and reflections, loosely aligned methods expand edits but risk reduced controllability and over-editing.Source: A Related work · p. 14
  • Prior geometry-aware generation often uses depth as auxiliary input in single-stage generation, whereas this method edits depth and renders RGB in separate stages.Source: A Related work · p. 14
  • The geometry stage uses a DPO-style preference strategy based on geometric flow smoothness to encourage plausible completion and suppress spurious insertions.Source: A Related work · p. 14

B Additional detals about stage 2: appearance rendering

The appearance model renders RGB from an input image and geometry maps before and after editing, using bidirectional training to learn appearance synthesis conditioned on geometry changes.

B Additional detals about stage 2: appearance rendering

C Additional qualitative comparison

Additional CausRem comparisons report removal of objects and their shadows or reflections, with coherent appearances and preservation of unmasked content.

C Additional qualitative comparison

  • The method removes objects and associated shadows in CausRem scenes, while other methods retain shadow residues or distort unmasked regions.Source: C Additional qualitative comparison · p. 15
  • For reflective surfaces, baseline methods often leave reflections or generate artifacts, while geometry-guided rendering produces coherent appearances without explicit reflection modeling.Source: C Additional qualitative comparison · p. 15
  • The comparisons support the paper's account that editing object structure and rendering from updated geometry can implicitly eliminate causal artifacts while maintaining visual consistency.Source: C Additional qualitative comparison · p. 15

D Additional perceptual metrics

Additional perceptual metrics are reported in Table 6 for RemovalBench and RORD-Val.

E Additional failure and challenging cases

The challenging-case analysis describes residual effects in reflective and self-emitting scenes, and incomplete masks that can lead to hallucinated geometry. A pseudo-depth cue also lets the framework remove a scene-wide watermark.

F CausRem

CausRem contains 200 manually annotated images: 100 with reflections and 100 with shadows. Its annotations preserve object-to-effect pairings, with different annotation counts for the two subsets.

  • CausRem comprises 200 high-quality Pexels images, split evenly between reflection and shadow examples, with objects and their causal effects manually annotated.Source: F CausRem · p. 18
  • Shadow images can include two to three annotated objects with distinct IDs linking each object to its shadow; reflection images have one object-mask pair.Source: F CausRem · p. 18

G Broader impacts

The authors identify potential benefits in several editing and restoration applications while noting risks from more controllable visual manipulation.

  • The authors cite autonomous driving, AR/VR content editing, and photo restoration as potential beneficiaries, while flagging deepfakes as a risk requiring responsible deployment, provenance tracking, and usage guidelines.Source: G Broader impacts · p. 18

Related papers

Loading 2509.18538v2…