Source-linked AI summary
ClickRemoval: An Interactive Open-Source Tool for Object Removal in Diffusion Models
Ledun Zhang, Yatu Ji, Xufei Zhuang, Xinying Yao
TL;DR
Object removal tools often require masks, prompts, or specialized training, increasing interaction costs for non-experts. ClickRemoval uses clicks to guide attention in pretrained Stable Diffusion models without additional training; experiments report competitive restoration quality and first-place user preference at both tested resolutions.
Problem
Object removal methods often rely on fine-grained masks, text prompts, or specialized training pipelines, creating high interaction costs and limiting usability for non-experts.
Source: §1 · p. 1Method
Results
ClickRemoval achieves competitive restoration quality without additional training and ranks first in user preference at 512 and 1024 resolutions.
Source: §3.3 · p. 4Takeaways & Limitations
The released code, model download scripts, Docker environment, and documentation are intended to support reproducible research and practical content editing.
Source: §4 · p. 4
Abstract
from arXiv · showhide
Existing object removal tools often rely on manual masks or text prompts, making precise removal difficult for non-expert users in complex scenes and often leading to incomplete removal or unnatural background completion. To address this issue, we present ClickRemoval, an open-source interactive object removal tool built on pretrained Stable Diffusion models and driven solely by user clicks. Without additional training, hand-drawn masks, or text descriptions, ClickRemoval localizes target objects and restores the background through self-attention modulation during denoising. Experiments show that ClickRemoval achieves competitive results across quantitative metrics and user studies. We release a complete software package at https://github.com/zld-make/ClickRemoval under the Apache-2.0 license.
1 Introduction
ClickRemoval addresses the interaction burden of object removal with a click-driven tool that localizes targets and restores backgrounds. Its attention-redirection framework spans multiple Stable Diffusion backbones, and the authors release a complete software package.
1 Introduction
- Existing removal methods often require fine-grained masks, text prompts, or specialized training, raising interaction costs and limiting usability for non-experts.Source: §1 · p. 1
- ClickRemoval uses positive and negative clicks without masks, text descriptions, or additional training to localize and remove targets and restore backgrounds.Source: §1 · p. 1
- The click-driven framework combines M2N2, SGAR, SGAS, and ARG and can be implemented with SD1.5, SD2.1, or SDXL1.0.Source: §1 · p. 1
- The released software package includes source code, configuration files, a Docker environment, documentation, a demo interface, and evaluation scripts.Source: §1 · p. 1
2 ClickRemoval: Design and Implementation
ClickRemoval converts clicks into semantic maps and uses scheduled self-attention redirection during denoising to suppress targets and guide background restoration. Adaptive restoration guidance then adjusts the final removal strength.
2 ClickRemoval: Design and Implementation
- ClickRemoval converts clicks into soft semantic maps, redirects Stable Diffusion self-attention during denoising, and blends original and modulated predictions to control removal strength.Source: §2 · p. 1, Figure 1 · p. 2
2.1 M2N2 Semantic Distance Map Extraction
- M2N2 propagates a one-hot click distribution through aggregated self-attention maps to derive semantic distances, then forms an object map and a complementary background-reference map.Flood Fill suppresses local minima for the object map; the unfilled normalized map supplies background guidance.Source: §2.1 · p. 2
2.2 Self-Guided Attention Redirection and Scheduling
- SGAR suppresses object-related attention keys and reweights background keys, while SGAS stages that guidance across denoising to preserve natural background completion.The early 20% of steps are unchanged; background guidance is typically used for about 5–10 early-middle steps.Source: §2.2 · p. 2
2.3 Attention Redirection Guidance
- ARG linearly combines original and modulated noise predictions using a user-specified strength coefficient, with smaller values keeping results closer to the original model.Source: §2.3 · p. 2
3 Experiments
Experiments compare ClickRemoval with inpainting and object-removal baselines using a target-free test set, quantitative metrics, visual comparisons, and user and GPT-assisted evaluations. ClickRemoval shows competitive restoration quality, while click settings support progressive removal and preservation of nontarget regions.
3.1 Experimental Setup
- The test set contains approximately 5,000 manually cleaned and annotated samples from Pico-Banana-400K, using edited target-free images as references for removal evaluation.The authors release the evaluation protocol, annotation schema, and scripts, but not benchmark images because of dataset license restrictions.Source: §3.1 · p. 3
- The comparison includes open-source inpainting and object-removal baselines, and reports FID, KID, and Local-FID for global and local restoration quality.Local-FID uses a square crop centered on the mask bounding box, with side length max(bounding-box longer side, 299).Source: §3.1 · p. 3
3.2 Comparison Experiments and Restoration Effect Visualization
- At 512 resolution, ClickRemoval’s SD1.5 variant achieves the best overall performance among same-resolution methods: FID 9.35, KID 0.899, and Local-FID 17.27.At 1024 resolution, its SDXL1.0 variant reaches FID 8.05 and Local-FID 15.56, comparable to AttentiveEraser (7.98 / 15.74).Source: §3.2 · p. 3
- Representative comparisons show more complete object removal and more natural background textures and structures, while vanilla SD backbones with ARG restore targets that SD1.5-Inp struggles to remove thoroughly.The ablation comparison concerns challenging removal cases.Source: §3.2 · pp. 3–4, Figure 3 · p. 4
- More positive clicks generally improve removal completeness, while negative clicks help preserve nontarget regions in occluded-object and multiobject cases.The study also includes large-object removal and complex-background restoration.Source: §3.2 · p. 4, Figure 4 · p. 4
3.3 User Study and LLM-Assisted Validation
- In the user study, ClickRemoval ranks first at both resolutions, receiving 30.48% of votes at 512 and 41.89% at 1024; GPT-assisted validation ranks it first at 512 and second at 1024.At 1024, its GPT-assisted score is 39.21%, 1.97 percentage points behind AttentiveEraser (41.18%).Source: §3.3 · p. 4
4 Conclusion
ClickRemoval combines click-based semantic maps, scheduled self-attention redirection, and adaptive restoration guidance for training-free object removal. The paper reports competitive restoration quality and strong user preference across resolutions.
4 Conclusion
- ClickRemoval removes target objects without manual masks or text prompts by combining click-based semantic maps, scheduled self-attention redirection, and adaptive restoration guidance.The released code, model download scripts, Docker environment, and documentation are intended to support reproducible research and practical content editing.Source: §4 · p. 4