Source-linked AI summary
RUN: Reversible Unfolding Network for Concealed Object Segmentation
Chunming He, Rihan Zhang, Fengyang Xiao, Chengyu Fang, Longxiang Tang, Yulun Zhang, Linghe Kong, Deng-Ping Fan, Kai Li, Sina Farsiu
TL;DR
COS methods often restrict reversible modeling to masks, leaving RGB-domain information underused for separating concealed objects from similar backgrounds. RUN addresses this gap by unfolding a residual-sparsity-based foreground-background model into stages combining mask-level SOFS and RGB-level ROBE, with reported superiority across COS and salient object detection tasks. The framework also shows potential for adaptable unfolding-based solutions, although quantitative analysis is limited for methods that are not publicly open-sourced.
Problem
Existing COS methods typically apply reversible strategies only in the mask domain, leaving potentially useful RGB-domain information underexplored.
Method
RUN adds a residual sparsity constraint and unfolds its optimization into multistage SOFS and ROBE modules that reversibly model foreground and background in mask and RGB domains.
Results
Experiments validate RUN’s superiority across five COS tasks and salient object detection, while supporting the effectiveness and adaptability of unfolding-based frameworks.
Takeaways & Limitations
RUN demonstrates the potential of deep unfolding for COS and other high-level vision tasks through a plug-and-play framework.
Takeaways & Limitations
Quantitative analysis is limited for methods that are not publicly open-sourced.
Abstract
from arXiv · showhide
Existing concealed object segmentation (COS) methods frequently utilize reversible strategies to address uncertain regions. However, these approaches are typically restricted to the mask domain, leaving the potential of the RGB domain underexplored. To address this, we propose the Reversible Unfolding Network (RUN), which applies reversible strategies across both mask and RGB domains through a theoretically grounded framework, enabling accurate segmentation. RUN first formulates a novel COS model by incorporating an extra residual sparsity constraint to minimize segmentation uncertainties. The iterative optimization steps of the proposed model are then unfolded into a multistage network, with each step corresponding to a stage. Each stage of RUN consists of two reversible modules: the Segmentation-Oriented Foreground Separation (SOFS) module and the Reconstruction-Oriented Background Extraction (ROBE) module. SOFS applies the reversible strategy at the mask level and introduces Reversible State Space to capture non-local information. ROBE extends this to the RGB domain, employing a reconstruction network to address conflicting foreground and background regions identified as distortion-prone areas, which arise from their separate estimation by independent modules. As the stages progress, RUN gradually facilitates reversible modeling of foreground and background in both the mask and RGB domains, directing the network's attention to uncertain regions and mitigating false-positive and false-negative results. Extensive experiments demonstrate the superior performance of RUN and highlight the potential of unfolding-based frameworks for COS and other high-level vision tasks. We will release the code and models.
1. Introduction
COS is difficult because concealed objects resemble their surroundings, while existing reversible methods mainly operate in the mask domain. RUN unfolds a foreground-background separation model across mask and RGB domains to reduce uncertainty and improve segmentation.
- Motivation: COS is challenging because objects are intrinsically similar to their backgrounds, making subtle discriminative cues difficult to extract.Earlier methods often emphasized foreground regions or relied on handcrafted features, while recent reversible methods modeled both regions only at the mask level.
- Motivation: Existing reversible methods leave the RGB domain underexplored, although mask uncertainty appears as color distortion during foreground-background separation.Addressing these RGB distortions supports more precise foreground and background separation.
- RUN framework: RUN formulates COS as foreground-background separation with a residual sparsity constraint, then unfolds its iterative optimization into a multistage network.Each optimization step corresponds to one network stage, integrating model-based solutions with deep networks.
- RUN framework: Each RUN stage combines SOFS for reversible mask-level foreground separation with ROBE for reversible RGB-level background extraction.SOFS uses RSS to extract non-local information, while ROBE reconstructs the image and refines the background.
- Results and implications: Across stages, RUN focuses on uncertain regions and reduces false-positive and false-negative outcomes through reversible modeling in both domains.The refined background is passed to later stages, and the framework can be integrated with existing methods.
- Results and implications: Experiments across five COS tasks and salient object detection validate RUN’s superiority and the adaptability of unfolding-based frameworks.The authors also describe the framework as plug-and-play for COS and other high-level vision tasks.
2. Related Works
Related COS methods increasingly use reversible foreground-background reasoning, but these methods remain focused on mask-level processing. Deep unfolding offers interpretability by combining model-based and learning-based approaches, yet its use in high-level vision is underexplored.
- Concealed object segmentation: Recent COS methods use reversible attention, foreground-background masks, or reconstruction to improve segmentation, but remain focused on the mask level.Examples include PraNet, FEDER, and BiRefNet.
- Deep unfolding network: Deep unfolding integrates model-based and learning-based approaches and generally offers greater interpretability than purely learning-based methods.Its application to high-level vision remains limited because explicit intrinsic models are often unavailable.
3. Methodology
RUN formulates concealed object segmentation as foreground–background separation with a residual sparsity constraint, then unfolds the optimization into stages that reversibly model mask and RGB information. Its SOFS and ROBE modules refine segmentation and reconstruction jointly while balancing interpretability with generalizability.
- COS model: RUN decomposes a concealed image into foreground and background and optimizes the foreground mask using a learned objective with regularization terms.The foreground is represented as C • M, where M is the segmentation mask.
- COS model: An extra residual sparsity constraint refines the mask and minimizes segmentation uncertainty through an attention-weighted ℓ1-norm.The constraint uses α as its weight, f_M as the uncertainty-refined mask, and w as the attention map.
- COS model: Pixels in the ambiguous mask range [0.4, 0.6] are excluded, while extreme refined-mask values are set to 0.1 and 0.9 to preserve optimization flexibility.This design encourages masks with higher certainty.
- Deep unfolding mechanism: RUN unfolds alternating mask and background optimization updates into a multistage network, with each optimization step corresponding to one stage.The mask and background updates use gradient, proximal, and closed-form solutions derived from the model.
- Deep unfolding mechanism: Each stage combines SOFS and ROBE: SOFS refines masks with reversible state-space processing, while ROBE dynamically fuses background estimates and reconstructs the image.SOFS uses distinct local and global perception fields; ROBE addresses distortion-prone regions caused by conflicting foreground and background interpretations.
- Deep unfolding mechanism: Joint segmentation and reconstruction align the reconstructed image with the concealed input, mitigating distortions and directing later stages toward uncertain regions.The training loss combines segmentation losses with a mean square reconstruction loss across stages.
4. Experiments
Experiments evaluate RUN across multiple concealed-object tasks, ablations, degraded scenarios, and integrations with existing methods. Results show strong cross-task performance, robustness, and gains from key design choices.
- Comparative Evaluation: RUN achieves state-of-the-art performance across camouflaged object detection settings, backbones, and four datasets.The strongest results are reported particularly on COD10K and NC4K, with more complete and accurate segmentation maps.
- Comparative Evaluation: RUN achieves top performance on medical concealed-object tasks, including polyp segmentation and medical tubular object segmentation.Visual results also support segmentation of small polyps and fine vessels and nerves.
- Comparative Evaluation: RUN surpasses existing methods on transparent object detection across two datasets, producing more precise object segmentation.The results indicate potential for autonomous-driving applications.
- Ablation Study: Removing deep features, reversible priors, state-space modules, or auxiliary edge outputs reduces performance, supporting their inclusion in RUN.The ablation study is conducted on COD10K and identifies deep features as particularly important within the unfolding framework.
- Ablation Study: A simple reconstruction network is sufficient for background extraction, while removing reconstructed outputs produces suboptimal results.Replacing it with larger CNN- or Transformer-based networks yields no significant gains.
- Ablation Study: RUN outperforms traditional and learning-based unfolding alternatives, and four stages are selected to balance performance and computational efficiency.The stage analysis reports that RUN already surpasses most compared methods at K = 2.
- Further Analysis: RUN consistently outperforms competing methods on images containing small or multiple concealed objects despite performance degradation for all methods.The evaluated subsets contain 1,084 small-object images and 186 multi-object images.
- Further Analysis: RUN remains more resilient as haze increases, and replacing its reconstruction network with a pretrained dehazing model further improves robustness.The enhanced model is called RUN+ and uses the more complex B3(•) network.
5. Conclusions
RUN formulates concealed object segmentation as foreground-background separation and unfolds its optimization into a multistage network with reversible mask- and RGB-domain modules.
- 5. Conclusions: RUN unfolds an optimized foreground-background separation model into a multistage network, with SOFS and ROBE in each stage.SOFS operates at the mask level with RSS for non-local information extraction, while ROBE reconstructs the image to address conflicting RGB-domain regions.
- 5. Conclusions: SOFS applies reversible modeling to segmentation masks and introduces RSS to extract non-local information.
- 5. Conclusions: ROBE uses a reconstruction network to address conflicting foreground and background regions in the RGB domain.
- 5. Conclusions: Extensive experiments verify RUN's superiority, supporting its potential for concealed object segmentation and other high-level vision tasks.
A.1. Datasets and metrics
The experiments cover concealed object, medical, transparent, defect, and salient object tasks using established datasets and task-specific metrics; supplementary tables and figures report additional comparisons.
- Datasets and metrics: Polyp image segmentation uses CVC-ColonDB and ETIS, with mDice, mIoU, and Sα as evaluation metrics.Higher values indicate better performance for all three metrics.
- Datasets and metrics: Transparent object detection uses GDD and GSD with PVT V2 as the default backbone and evaluates mIoU and maximum F-measure.
- Datasets and metrics: Concealed defect detection evaluates a COD-trained model on CDS2K using eight metrics, with lower MAE and higher values for the remaining metrics indicating better performance.
- Datasets and metrics: Salient object detection uses DUT-OMRON, DUTS-test, ECSSD, HKU-IS, and PASCAL-S benchmark datasets.
A.2. Generalization on concealed defect detection
RUN generalizes to concealed defect detection by applying a model trained on the camouflaged object detection task directly to CDS2K, where it outperforms existing state-of-the-art approaches.
- A.2. Generalization on concealed defect detection: RUN achieves superior performance over existing state-of-the-art approaches on concealed defect detection.The comparison is reported in Table S1.
- A.2. Generalization on concealed defect detection: The concealed defect evaluation directly applies the model trained on the COD task to segment objects in CDS2K.
A.3. Generalization on salient object detection
RUN generalizes to salient object detection and outperforms existing state-of-the-art approaches, indicating the potential of unfolding-based frameworks for broader high-level vision tasks.
- A.3. Generalization on salient object detection: RUN outperforms existing state-of-the-art approaches on salient object detection and achieves a leading position.The results are reported in Table S2.
- A.3. Generalization on salient object detection: The salient object detection results underscore the potential of unfolding-based frameworks for high-level vision tasks.
B. Limitations and Future Work
RUN remains unstable in degraded scenarios, and future work targets broader degradation types and the computational demands of larger unfolding-based models. The authors also identify further development of degradation-resistant integrations and high-resolution COS as open directions.
- Limitations: RUN, like other advanced methods, exhibits instability in degraded scenarios, although RUN+ is robust to haze degradation.The authors attribute the instability to difficulty extracting subtle discriminative information under environmental degradation.
- Future Work: Future work will integrate RUN with advanced low-level vision algorithms for scenarios involving low light, blur, and noise.
- Future Work: Integrating large-scale algorithms into the unfolding-based multistage framework introduces significant computational and storage demands.Developing effective strategies for integrating degradation-resistant models remains an important research direction.
- Future Work: The authors aim to establish an unfolding-based high-resolution COS method and encourage further DUN-based algorithms for high-level vision.These directions are presented as ways to further balance interpretability and generalizability.