Source-linked AI summary
Selectivity or Invariance: Boundary-aware Salient Object Detection
Jinming Su, Jia Li, Yu Zhang, Changqun Xia, Yonghong Tian
TL;DR
SOD faces a selectivity-invariance dilemma because interiors need appearance-invariant features while boundaries need selective features. The paper addresses this with a three-stream boundary-aware network and successive dilation, outperforming 16 state-of-the-art algorithms on six datasets.
Problem
SOD requires invariant interior features under appearance changes and selective boundary features for weak object-background differences.
Method
A boundary-aware network uses boundary localization, interior perception, and transition compensation streams, with integrated successive dilation enhancing invariant features.
Results
The approach outperforms 16 state-of-the-art algorithms on six datasets.
Takeaways & Limitations
Separating feature extraction strategies across boundaries, interiors, and transitional regions addresses the selectivity-invariance dilemma in image-based SOD.
Abstract
from arXiv · showhide
Typically, a salient object detection (SOD) model faces opposite requirements in processing object interiors and boundaries. The features of interiors should be invariant to strong appearance change so as to pop-out the salient object as a whole, while the features of boundaries should be selective to slight appearance change to distinguish salient objects and background. To address this selectivity-invariance dilemma, we propose a novel boundary-aware network with successive dilation for image-based SOD. In this network, the feature selectivity at boundaries is enhanced by incorporating a boundary localization stream, while the feature invariance at interiors is guaranteed with a complex interior perception stream. Moreover, a transition compensation stream is adopted to amend the probable failures in transitional regions between interiors and boundaries. In particular, an integrated successive dilation module is proposed to enhance the feature invariance at interiors and transitional regions. Extensive experiments on six datasets show that the proposed approach outperforms 16 state-of-the-art methods.
1. Introduction
SOD must reconcile invariant interior features with selective boundary features, motivating a three-stream boundary-aware network with successive dilation.
- SOD detects and segments objects that capture human visual attention for later recognition, tracking, and image parsing.
- Large appearance changes hinder detecting salient-object interiors as a whole, while weak boundaries hinder distinguishing objects from background.
- Interior features should be invariant to changes in size, color, and texture, whereas boundary features should selectively distinguish minor object-background differences.
- The proposed network uses boundary localization, interior perception, and transition compensation streams to handle selective, invariant, and transitional regions.
- An integrated successive dilation module enhances invariant-feature extraction, and the network outperforms 16 state-of-the-art algorithms on six datasets.
2. Related Work
Prior deep SOD work emphasizes feature integration and relationships to other vision tasks; this paper instead uses successive dilation and low-level skip connections for region-specific representations.
- Traditional SOD methods used handcrafted local and global features alongside heuristic saliency priors such as color difference, distance transformation, and contrast.
- Recent deep SOD models commonly integrate multi-level, multi-resolution, contextual, or stage-wise features to improve neural-network performance.
- The proposed integrated successive dilation module captures richer contextual information for interior invariance while skip connections promote selective boundary representations.
- Some SOD models relate saliency detection to human perception, fixation, semantic information, or depth-aware complementarity.
3. The Proposed Approach
The proposed boundary-aware network separates SOD into boundary localization, interior perception, and transition compensation streams, then fuses their features through confidence-guided mosaic selection. An integrated successive dilation module aggregates contextual information at multiple scales to enhance invariant representations.
- Three-stream network: The network extracts common features and processes them through boundary localization, interior perception, and transition compensation streams.The common extractor is based on ResNet-50, while the three streams specialize in selectivity, invariance, and their tradeoff.
- Boundary localization: The boundary localization stream aggregates multi-level features and uses a sigmoid confidence map to represent salient-object boundaries.Its simple feature mapping design emphasizes selective representations at object boundaries.
- Interior perception: The interior perception stream uses the deepest common features and an ISD module to enhance invariance inside salient objects.Its invariance confidence map approximates the salient-object ground-truth mask through cross-entropy supervision.
- Transition compensation: The transition compensation stream addresses uncertain regions between interiors and boundaries by providing features that mediate selectivity and invariance.It is designed to amend failures that remain after combining the boundary and interior streams, including detailed transitional structures.
- Confidence-guided fusion: The feature mosaic map selects boundary, interior, or transitional features according to selectivity and invariance confidence maps rather than directly summing or concatenating them.Boundary features are emphasized where selectivity is high and invariance is low; interior features are emphasized under the opposite confidence pattern, while transitional features handle medium-confidence locations.
- Integrated successive dilation: The ISD module uses parallel branches, skip connections, channel compression, and progressively doubled dilation rates to aggregate local contexts across scales.ISD-N spans dilation rates from 1 to 2^(N−1); the paper uses ISD-5 for interior perception and ISD-3 for transition compensation.
4. Experiments and Results
Experiments evaluate BANet on six benchmark datasets using standard saliency metrics, comparisons with 16 state-of-the-art methods, qualitative examples, and component ablations.
- Experimental Setup: BANet is evaluated on six benchmark datasets using MAE, Fβ, weighted Fβ, PR curves, and F-measure curves.Fβ uses β² = 0.3 to emphasize Precision over Recall; MAE measures pixel-wise absolute error.
- Comparisons with the State-of-the-Arts: BANet consistently outperforms 16 state-of-the-art methods across all six benchmark datasets.The comparison uses saliency maps obtained from authors or their deployment code for fair evaluation.
- Comparisons with the State-of-the-Arts: Fwβ reaches 0.810 versus 0.788 on PASCAL-S, 0.811 versus 0.765 on DUTS-TE, and 0.839 versus 0.818 on XPIE against the second-best results.BANet also achieves the best MAE on all datasets; its Fβ improves on every dataset except DUT-OMRON.
- Comparisons with the State-of-the-Arts: Qualitative comparisons show BANet producing salient objects with clearer boundaries despite large appearance changes or similarity to the background.The reported examples associate interior integrity with ISD and boundary clarity with boundary awareness.
- Ablation Analysis: Adding BLS to IPS decreases MAE and increases F-measure, while adding TCS further improves performance by amending transitional-region failures.On HKU-IS, Fβ increases from 0.914 to 0.929, an improvement of up to 1.5%.
- Ablation Analysis: Removing ISD dramatically decreases performance on all six datasets, with contextual errors such as flowers and water reflections falsely detected as salient.ASPP and ASPP-M improve performance to some extent, but ISD aggregates contextual information through more information-flow paths and better enhances feature invariance.
5. Conclusion
The paper addresses SOD’s selectivity-invariance dilemma with a boundary-aware network that combines specialized streams and successive dilation, validated on six benchmark datasets.
- The proposed network captures selective boundary features, invariant interior features, and transition-region information through three dedicated streams.Their outputs are fused into a saliency mask using boundary-aware feature mosaic selection.
- An integrated successive dilation module enhances feature invariance to support salient-object perception and localization.
- Extensive experiments on six benchmark datasets validate the effectiveness of the proposed approach.