PaperlayerView exact revision on arXiv

Source-linked AI summary

Suppress and Balance: A Simple Gated Network for Salient Object Detection

Xiaoqi Zhao, Youwei Pang, Lihe Zhang, Huchuan Lu, Lei Zhang

arXiv:2007.08074v3cs.CV

TL;DR

Salient object detectors often lack interference control during encoder–decoder exchange and overlook unequal contributions from encoder blocks, while large-dilation context extraction can weaken local correlations. GateNet uses multilevel gates, a gated dual branch, and Fold-ASPP to regulate feature transmission, restore details, and capture multiscale cues. It performs favorably against most state-of-the-art methods on five challenging datasets and runs at 30 fps.

  • Problem

    Salient object detection methods lack interference control in encoder–decoder exchange and do not account for unequal contributions from different encoder blocks; large-dilation ASPP can also weaken local feature correlations.

  • Method

    GateNet combines multilevel gate units, a gated dual-branch architecture, and Fold-ASPP to regulate encoder information, supplement details, and gather multiscale saliency cues.

  • Results

    GateNet performs favorably against most state-of-the-art methods on five challenging datasets and achieves 30 fps.

  • Takeaways & Limitations

    The gated FPN design provides a baseline for dense prediction tasks while Fold-ASPP retains local correlations during multiscale context extraction.

Abstract

from arXiv · show

Most salient object detection approaches use U-Net or feature pyramid networks (FPN) as their basic structures. These methods ignore two key problems when the encoder exchanges information with the decoder: one is the lack of interference control between them, the other is without considering the disparity of the contributions of different encoder blocks. In this work, we propose a simple gated network (GateNet) to solve both issues at once. With the help of multilevel gate units, the valuable context information from the encoder can be optimally transmitted to the decoder. We design a novel gated dual branch structure to build the cooperation among different levels of features and improve the discriminability of the whole network. Through the dual branch design, more details of the saliency map can be further restored. In addition, we adopt the atrous spatial pyramid pooling based on the proposed "Fold" operation (Fold-ASPP) to accurately localize salient objects of various scales. Extensive experiments on five challenging datasets demonstrate that the proposed model performs favorably against most state-of-the-art methods under different evaluation metrics.

1 Introduction

GateNet addresses uncontrolled encoder–decoder information exchange and unequal encoder-block contributions with gated feature transmission, Fold-ASPP, and a dual-branch design. It reports stronger results than competitors across five datasets and 30 fps inference.

  • U-shaped saliency networks often use encoder features through an all-pass skip layer, without controlling interference or accounting for unequal encoder-block contributions.
  • Large-dilation ASPP can weaken correlations among sampled points, harming discrimination of subtle structures in size-varying objects.
  • GateNet uses multilevel gate units to balance encoder-block contributions, suppress non-salient features, and transmit more useful context to the decoder.
  • Fold-ASPP gathers multiscale high-level cues by applying atrous convolution over folded local neighborhoods, producing more stable features and finer structures.
  • The dual-branch architecture combines FPN outputs with gated encoder features, supplementing complementary residual information for the final saliency map.
  • 30 fps is achieved while comparing GateNet with seventeen state-of-the-art methods on five challenging datasets.

2 Related Work

Related work emphasizes multiscale context, gated information flow, and encoder–decoder architectures for salient object detection. GateNet combines these themes in an architecture with gated feature transmission and Fold-ASPP.

  • 2.1 Salient Object Detection: Traditional salient object detection methods rely on low-level handcrafted features and heuristic priors such as color contrast, background, and center priors.
  • 2.1 Salient Object Detection: Deep salient object detectors commonly use fully convolutional or U-shaped architectures, but single-scale features inadequately characterize varied objects and image contexts.
  • 2.2 Multiscale Feature Extraction: ASPP enlarges receptive fields without increasing computational cost, yet repeated downsampling and larger dilation rates can degrade fine-detail correlations.
  • 2.2 Multiscale Feature Extraction: Fold-ASPP addresses these issues by using folded atrous convolution to achieve a local-in-local effect for more stable multiscale features.
  • 2.3 Gated Mechanisms: Gated mechanisms filter information during feature fusion and can function as a specialized attention mechanism for controlling multilevel message passing.
  • 2.3 Gated Mechanisms: The GateNet architecture combines a VGG-16 encoder, transition layers, gate units, decoder blocks, and Fold-ASPP with dual supervision.

3 Proposed Method

GateNet uses multilevel gates to regulate encoder-to-decoder information and a dual-branch decoder to combine complementary feature pathways. Fold-ASPP adds multiscale context while preserving local feature correlations.

  • Network Overview: GateNet combines encoder and decoder feature maps through multilevel gate units that weight transition-layer features before decoder fusion.The gate values are computed from integrated encoder and previous-decoder features, then applied across FPN and parallel branches.
  • Gated Dual Branch: The gate vector contains two values that separately weight features in the FPN and parallel branches.The FPN branch progressively combines gated encoder and decoder features, whereas the parallel branch combines gated encoder features at a common resolution.
  • Gated Dual Branch: High-level encoder features receive larger FPN-branch gate values, while parallel-branch gate values decrease from lower to higher levels.The reported distributions associate the FPN branch with main-object prediction and the parallel branch with detail restoration.
  • Gated Dual Branch: The dual-branch decoder integrates its predictions through a residual connection, while the FPN branch receives intermediate cross-entropy supervision.The final prediction combines the two branch outputs, and the FPN branch is supervised alongside it.
  • Folded Atrous Spatial Pyramid Pooling: Folded atrous convolution merges more context while preserving certain local correlations, providing a trade-off between atrous and vanilla convolution in the spatial dimension.The Fold-ASPP module is placed at the top of the encoder and consists of three folded convolutional layers.
  • Folded Atrous Spatial Pyramid Pooling: Fold-ASPP applies Fold, atrous convolution, and Unfold operations to top-level features, using dilation rates for multiscale processing.Folding changes N × N × C features to N/2 × N/2 × 4C before atrous convolution; the reverse Unfold operation restores the spatial arrangement.

4 Experiments

Experiments evaluate GateNet across five datasets, multiple metrics, qualitative comparisons, and component ablations. GateNet consistently outperforms competing methods, while multilevel gates, Fold-ASPP, and the parallel branch each contribute measurable improvements.

  • Performance Comparison with State-of-the-art: GateNet consistently outperforms seventeen state-of-the-art methods across five datasets and different evaluation metrics.It also achieves real-time speed of 30 fps.
  • Performance Comparison with State-of-the-art: 0.870 vs 0.852 and 0.888 vs 0.872 are GateNet’s F-measure comparisons with BANet on DUTS-test.On PASCAL-S, the corresponding comparisons are 0.882 vs 0.866 and 0.883 vs 0.877.
  • Performance Comparison with State-of-the-art: GateNet’s PR curves are significantly higher than those of other methods on five datasets and require no post-processing.The study uses precision-recall curves, F-measure, S-measure, and MAE for quantitative evaluation.
  • Ablation Studies: Using ResNet-50, ResNet-101, or ResNeXt-101 backbones significantly improves the gated network’s performance.Table 1 identifies backbone settings and reports F-measure, S-measure, and MAE comparisons.
  • Ablation Studies: Multilevel gate units improve the VGG-16 FPN baseline by 2.94%, 2.17%, and 11.67% for F-measure, S-measure, and MAE, respectively.Feature visualizations show maintained object-background contrast, recovered details, and reduced suppression of slender object parts.
  • Ablation Studies: Folded convolution consistently improves over corresponding atrous convolution settings across all three metrics.Fold-ASPP further outperforms ASPP by 1.17% in F-measure and 8.0% in MAE.
  • Ablation Studies: Adding the parallel branch restores object details, while the gate units, Fold-ASPP, and parallel branch complement one another.Qualitative comparisons report better boundary retention and separation of adjacent objects.

5 Conclusions

The conclusion presents GateNet as a gated architecture that controls encoder-decoder information flow, gathers multiscale context, and restores details. Experiments on five benchmark datasets show superiority over seventeen state-of-the-art methods.

  • 5 Conclusions: Multilevel gate units balance encoder-block contributions and suppress non-salient activations while minimizing interference in decoder inputs.The authors propose a gated FPN as a new baseline for dense prediction tasks.
  • 5 Conclusions: Fold-ASPP gathers multiscale semantic information while folded atrous convolution expands the receptive field and retains local sampling-point correlations.The folded operation provides a local-in-local effect for the atrous convolution.
  • 5 Conclusions: Parallel combination of encoder features forms a residual structure that supplements object details.The conclusion describes this branch as complementary to the gated and Fold-ASPP components.
  • 5 Conclusions: GateNet outperforms seventeen state-of-the-art methods under different evaluation metrics on five benchmark datasets.The conclusion reports this as the overall experimental result.

A Appendix

The appendix extends GateNet beyond RGB salient object detection to RGB-D salient object detection and video object segmentation.

  • A Appendix: GateNet is expanded to RGB-D salient object detection and video object segmentation to further demonstrate its effectiveness.These are additional tasks beyond the main RGB SOD setting.

A.1 Network Architecture

For RGB-D SOD and VOS, the proposed dual-branch GateNet adds a modality-specific encoder while retaining the gated FPN design. The architecture is presented as easy to follow and suitable as a baseline.

  • A.1 Network Architecture: The dual-branch GateNet adds an extra encoder to extract depth or optical-flow features for RGB-D SOD and VOS.The RGB SOD network is otherwise used as the comparison basis.
  • A.1 Network Architecture: Fig. 10 depicts the proposed dual-branch gated FPN network for RGB-D SOD and VOS.The figure is identified as the network pipeline.
  • A.1 Network Architecture: The dual-branch GateNet is described as easy to follow and usable as a new baseline.This statement concerns the architecture applied to the additional modalities and tasks.

A.2 RGB-D Salient object detection

The evaluation uses five RGB-D salient object detection datasets with a shared training and testing split for fair comparison. GateNet is compared with ten state-of-the-art methods using six saliency metrics.

  • Dataset: The evaluation covers NJUD, RGBD135, NLPR, SSD, and SIP, following prior splitting protocols for fair comparison.NJUD and NLPR provide 1,485 and 700 training samples, respectively; remaining images support testing.
  • Comparison: The proposed model is compared with ten state-of-the-art approaches across five benchmark datasets.The comparison methods include DES, DCMC, CDCP, DF, CTMF, PCA, MMCI, TANet, CPFP, and DMRA.
  • Metrics: Performance is evaluated using maximum, mean, and weighted F-measures, S-measure, E-measure, and MAE.The experiments use VGG-16 as the backbone for each stream.

A.3 Video Object Segmentation

The paper applies its dual-branch GateNet structure to zero-shot video object segmentation and evaluates it on the DAVIS-16 validation set. The reported tables define the direction of better performance and identify the best results.

  • Task setting: Video object segmentation is divided into zero-shot and one-shot settings according to whether the first-frame mask is provided during testing.The paper mainly uses the dual-branch GateNet structure for zero-shot video object segmentation.
  • Dataset and metrics: DAVIS-16 is used as a benchmark dataset containing 50 high-quality video sequences, with 30 for training and 20 for validation.The paper evaluates zero-shot video object segmentation on the DAVIS-16 validation set.
  • Results: Table 4 reports quantitative comparisons using maximum, mean, and weighted F-measures, S-measure, E-measure, and MAE scores.For CNN-based methods, the best results are indicated separately, with arrows denoting metric direction.
  • Results: Table 5 presents quantitative comparisons of zero-shot video object segmentation methods on the DAVIS-16 validation set.The table marks whether larger or smaller scores are better and highlights the best results.
Loading 2007.08074v3…