Source-linked AI summary
Reverse Attention for Salient Object Detection
Shuhan Chen, Xiuli Tan, Ben Wang, Xuelong Hu
TL;DR
Salient object detection needs higher-resolution outputs and smaller models for embedded and downstream vision applications. The paper combines side-output residual learning with top-down reverse attention, achieving comparable state-of-the-art performance with 45 FPS speed and an 81 MB model.
Problem
Salient object detection is constrained by low-resolution outputs and heavy model weights, limiting accurate boundary and small-object localization and embedded-device use.
Method
The network progressively refines coarse saliency predictions with side-output residual features, while reverse attention erases current predictions to guide discovery of missing details.
Results
The approach consistently achieves comparable state-of-the-art performance with 45 FPS efficiency and an 81 MB model size.
Takeaways & Limitations
Compact residual refinement and reverse attention provide an efficient saliency-detection network that improves map resolution with limited parameters.
Abstract
from arXiv · showhide
Benefit from the quick development of deep learning techniques, salient object detection has achieved remarkable progresses recently. However, there still exists following two major challenges that hinder its application in embedded devices, low resolution output and heavy model weight. To this end, this paper presents an accurate yet compact deep network for efficient salient object detection. More specifically, given a coarse saliency prediction in the deepest layer, we first employ residual learning to learn side-output residual features for saliency refinement, which can be achieved with very limited convolutional parameters while keep accuracy. Secondly, we further propose reverse attention to guide such side-output residual learning in a top-down manner. By erasing the current predicted salient regions from side-output features, the network can eventually explore the missing object parts and details which results in high resolution and accuracy. Experiments on six benchmark datasets demonstrate that the proposed approach compares favorably against state-of-the-art methods, and with advantages in terms of simplicity, efficiency (45 FPS) and model size (81 MB).
1 Introduction
The paper targets low-resolution saliency outputs and heavy models by combining compact side-output residual learning with reverse attention. It reports comparable state-of-the-art performance while reducing model size and supporting real-time use.
- Motivation: Salient object detection supports downstream vision tasks but remains challenging for embedded devices because outputs lose resolution and models are heavy.Repeated CNN stride and pooling impair boundaries and small-object localization; existing deep models are described as larger than 100 MB.
- Results: The visual comparison reports progressively improved saliency-map resolution from deep to shallow side-outputs, with reverse attention outperforming short connections.The comparison includes DSS, the method without reverse attention, and the method with reverse attention.
- Approach: The proposed network learns side-output residual features to refine a coarse deepest-layer saliency prediction progressively.Residual learning is introduced into an HED-style architecture, using limited convolutional parameters for refinement.
- Approach: Reverse attention guides residual learning top-down by erasing current predictions so missing object parts and details can be discovered.The method is intended to improve saliency-map resolution and accuracy without directly fusing multi-level features.
- Results: 45 FPS and an 81 MB model provide the reported efficiency and compactness advantages alongside comparable state-of-the-art performance.The paper reports these figures as benefits of combining side-output residual learning with reverse attention.
2 Related Work
The related work reviews feature-fusion and refinement strategies for high-resolution saliency maps, then positions reverse attention as a top-down guide for residual refinement.
- Prior saliency methods: Recent FCN-based saliency methods address low-resolution maps using recurrent refinement, guided filtering, short connections, or multi-level convolutional fusion.These approaches target complementary shallow spatial details and deep semantic information.
- Residual learning: Residual learning has been used for side-output features and image super-resolution, but its direct application to salient object detection was reported as insufficient.The paper motivates adding an attention mechanism to improve residual-detail learning.
- Reverse attention: The paper employs reverse attention in a top-down manner to guide side-output residual learning and obtain more accurate residual details.This differs from attention methods that weight multi-scale features or correct reverse-class predictions in semantic segmentation.
- Proposed architecture: The proposed architecture is illustrated as a stacked network with reverse-attention blocks and supervised residual units whose error decreases along the stacking direction.The figure shows supervision on both the input and output of the residual unit.
3 Proposed Method
The proposed network combines side-output residual learning with top-down reverse attention to refine coarse saliency predictions into more complete, high-resolution maps efficiently.
- Architecture: The network uses HED with a VGG-16 backbone and side-outputs at strides 1, 2, 4, 8, and 16 pixels.Pool5 is reduced to 256 channels, followed by three 5×5 convolutional layers for global saliency.
- Side-output Residual Learning: Residual features progressively remedy errors between upsampled saliency predictions and ground truth across side-output stages.Deep supervision is applied to both the input and output of each residual unit, creating shortcut connections across scales.
- Top-down Reverse Attention: Reverse attention erases confident deep-layer predictions from side-output features to guide discovery of missing object regions and details.The reverse attention weight is formed by subtracting the upsampled prediction from one, then applied through element-wise feature multiplication.
- Top-down Reverse Attention: Reverse attention helps the network capture residual details near object boundaries, whereas removing it produces redundant features inside objects.The benefit is especially apparent in shallow side-outputs, where spatial detail supports saliency refinement.
- Supervision: Each side-output receives deep supervision through a class-balanced cross-entropy loss, and the first side-output becomes the final test prediction after sigmoid activation.Unlike HED and DSS, the proposed approach does not include a fusion layer.
- Method Differences: The proposed reverse attention differs from prior reverse attention by erasing confident predictions before residual learning rather than weighting non-target-class predictions.It also differs from SRN and LRN in applying reverse attention before side-output feature learning.
4 Experiments
Experiments evaluate the method’s metrics, ablations, benchmark performance, qualitative behavior, and execution efficiency. The results support reverse attention and side-output residual learning while showing strong accuracy and practical speed.
- Experimental Setup: The method is evaluated on six benchmark datasets using PR curves, F-measure, and MAE.The datasets include MSRA-B, HKU-IS, ECSSD, PASCAL-S, SOD, and DUT-OMRON.
- Ablation Studies: The best ablation performance is obtained with depth D=2, which is used in subsequent experiments.Depth was varied from 1 to 3 on PASCAL-S and DUT-OMRON.
- Ablation Studies: Performance gradually improves as more side-output residual features are combined.
- Ablation Studies: Reverse attention yields an average 1.4% F-measure gain and a 0.5% MAE decrease.Without reverse attention, performance is similar to DSS without CRF-based post-processing; applying reverse attention improves it by a large margin.
- Performance Comparison with State-of-the-art: The approach significantly outperforms competing methods on F-measure and MAE, especially on challenging datasets such as DUT-OMRON.Its PR-curve performance is comparable to state-of-the-art methods except at recall>0.9, and nearly identical or better than DSS+ across the board.
5 Conclusions
The paper presents an accurate, compact network for efficient salient object detection by combining side-output residual learning with top-down reverse attention. This progressively improves saliency-map resolution with limited parameters and supports favorable quantitative and qualitative comparisons.
- Residual learning refines saliency through side-output residual features instead of directly learning multi-scale saliency features at each stage.
- The deepest convolutional layer generates a global saliency map whose resolution is gradually improved with very limited parameters.
- Reverse attention guides side-output residual learning in a top-down manner, producing more accurate residual features.
- More accurate residual features lead to significant performance improvement.
- Experiments show favorable quantitative and qualitative comparisons against state-of-the-art methods, supporting the network’s use in real-world applications.