Source-linked AI summary
A Mutual Learning Method for Salient Object Detection with intertwined Multi-Supervision--Revised
Runmin Wu, Mengyang Feng, Wenlong Guan, Dong Wang, Huchuan Lu, Errui Ding
TL;DR
The paper addresses incomplete saliency predictions and inaccurate boundaries by jointly supervising salient object, foreground contour, and edge detection. Its intertwined multi-task strategy and Mutual Learning Module produce uniformly highlighted saliency maps with accurate boundaries, while edge detection runs faster at comparable accuracy, with real-time deployment remaining challenging.
Problem
Saliency detectors struggle to uniformly highlight complex objects and produce accurate boundaries because pixel responses vary across scales and convolutional operations lose spatial precision.
Method
The method jointly trains salient object, foreground contour, and edge detection using intertwined supervision, mutual task guidance, and Mutual Learning Modules with peer-teaching subnetworks.
Results
The model generates uniformly highlighted saliency maps with accurate boundaries, while its edge detection module runs much faster with comparable accuracy.
Takeaways & Limitations
Mutual supervision connects salient-object, contour, and edge learning so the network can improve saliency quality while maintaining comparable edge accuracy.
Takeaways & Limitations
The experiments are mainly benchmark-based, and real-time deployment or use on resource-constrained devices remains challenging.
Abstract
from arXiv · showhide
Though deep learning techniques have made great progress in salient object detection recently, the predicted saliency maps still suffer from incomplete predictions due to the internal complexity of objects and inaccurate boundaries caused by strides in convolution and pooling operations. To alleviate these issues, we propose to train saliency detection networks by exploiting the supervision from not only salient object detection, but also foreground contour detection and edge detection. First, we leverage salient object detection and foreground contour detection tasks in an intertwined manner to generate saliency maps with uniform highlight. Second, the foreground contour and edge detection tasks guide each other simultaneously, thereby leading to precise foreground contour prediction and reducing the local noises for edge prediction. In addition, we develop a novel mutual learning module (MLM) which serves as the building block of our method. Each MLM consists of multiple network branches trained in a mutual learning manner, which improves the performance by a large margin. Extensive experiments on seven challenging datasets demonstrate that the proposed method has delivered state-of-the-art results in both salient object detection and edge detection.
1. Introduction
The paper addresses incomplete saliency regions and inaccurate object boundaries by jointly supervising salient object, foreground contour, and edge detection. Its intertwined strategy and Mutual Learning Module improve saliency quality while maintaining fast, comparable edge detection.
- Deep saliency methods struggle to uniformly highlight complex objects and accurately predict contours after strided convolution and pooling.
- The network uses salient-object and edge inputs with salient masks, edge labels, and foreground contours extracted from salient masks as supervision.
- Intertwining salient-object and foreground-contour detection alternates foreground filling and contour extraction to produce more uniformly highlighted regions.
- Mutual guidance between foreground contour and edge detection improves contour accuracy while reducing noisy local edges.
- The Mutual Learning Module uses multiple peer-taught subnetworks with mimicry loss to provide an additional performance gain.
- Across seven challenging saliency datasets, the method outperforms competing saliency models and achieves comparable edge detection with faster speed.
2. Related Work
Prior work mainly fuses hierarchical features or adds edge information to improve saliency maps, but often leaves object interiors uneven and boundaries blurred. This paper instead uses intertwined saliency-contour learning and reciprocal edge-contour guidance.
- Existing saliency methods fuse multi-scale features through skip connections, recurrent architectures, or refinement modules to recognize targets and recover details.
- Saliency supervision alone can produce uneven object responses because illumination, color, and other cues affect pixels differently across feature scales.
- The proposed intertwined saliency-contour learning helps recognize overall target shape and assign foreground labels across entire objects.
- The framework combines a VGG backbone, Mutual Learning Modules, edge modules, and decoder blocks in an encoder-decoder architecture.
- Unlike prior edge-based methods that use edge knowledge only for saliency, this model makes edge and foreground-contour detection guide each other.
3. The Proposed Method
The proposed encoder–decoder network combines mutual learning, edge features, and intertwined saliency/foreground-contour supervision. Deep supervision and weighted losses train these components to produce uniform saliency regions and fine contours.
- 3.1. Architecture Overview: The network combines a VGG-16 backbone, six mutual learning modules, three edge modules, and deeply supervised decoder blocks.The decoder fuses multi-scale MLM features and upsampled features to generate predictions.
- 3.1.1 Mutual Learning Module: Each MLM contains multiple student subnetworks that receive ground-truth supervision and mutually exchange prediction guidance through an L2 mimicry loss.Only one student branch is randomly selected for testing after mutual training.
- 3.1.2 Edge Module: Edge modules extract edge probability maps and edge features, while residual connections transfer these features to MLMs for contour prediction and reduce noise in edge features.The shared features provide semantic information that helps edge detection ignore useless local edges.
- 3.2. Intertwined Supervision Method: Intertwined supervision alternates salient-object and foreground-contour ground truths across decoder blocks, producing uniformly highlighted saliency predictions while preserving fine contours.S-gt supervises D0, D2, and D4, whereas FC-gt supervises D1 and D3.
- 3.3.1 Loss Function: The encoder loss combines saliency, edge, and mimicry losses with weights 0.7, 0.2, and 0.1, respectively.Binary Cross-Entropy is used for saliency and edge tasks, while the mimicry loss uses MSE.
4. Experiments
Experiments evaluate the method on salient object and edge detection benchmarks using standard metrics, comparisons, and ablations. The method outperforms competing saliency models, while lightweight edge detection achieves comparable accuracy faster.
- Datasets and metrics: Saliency performance is evaluated on DUTS and six additional datasets using PR curves, F-measure, MAE, and S-measure.DUTS provides 10,553 training and 5,019 testing images; the other datasets cover complex, low-contrast, overlapping, and boundary-touching objects.
- Datasets and metrics: Edge detection is evaluated on BSD500 using ODS and OIS F-measures, with 200 training, 100 validation, and 200 test images.ODS uses one dataset-wide threshold, whereas OIS uses an image-specific threshold.
- Performance comparison: Across all datasets, the method outperforms competing saliency methods on MAE, mean F-measure, and S-measure.The comparison includes 14 state-of-the-art methods and uses official implementations or authors’ provided saliency maps.
- Performance comparison: Approximately 3× faster edge detection is achieved with comparable accuracy because the evaluated network uses only the first three backbone blocks and corresponding edge modules.The lightweight architecture is compared with popular methods including gPb-UCM and HED.
- Ablation analysis: The intertwined supervision strategy contributes the most to overall performance in ablations on OMRON and DUTS.The ablation compares a saliency-only baseline, mutual-learning branches, and the full network with intertwined supervision and edge modules.
- Ablation analysis: Joint training with saliency tasks improves edge detection by adding semantic information and suppressing noise from redundant local details.This comparison contrasts edge-only supervision with joint training.
5. Conclusion
The paper proposes a multi-task method combining salient object, foreground contour, and edge detection with intertwined supervision and mutual learning. It produces uniformly highlighted saliency regions and accurate boundaries, while edge detection is faster at comparable accuracy.
- Conclusion: The method jointly addresses salient object detection, foreground contour detection, and edge detection through intertwined supervision, mutual guidance, and Mutual Learning Modules.The intertwined strategy targets complete object highlighting, while shared learning benefits edge and saliency tasks.
- Conclusion: The model generates uniformly highlighted saliency regions and accurate boundaries, with edge detection running much faster at comparable accuracy.Experiments report more accurate saliency maps across diverse images and faster edge processing.
6. Future Work
The paper identifies benchmark realism, computational efficiency, and labeled-data requirements as constraints, while proposing broader validation, efficient deployment, and semi- or self-supervised extensions. It also describes scalability and modularity as directions for larger datasets, streaming scenarios, and integration with emerging technologies.
- Benchmark-only experiments may not fully reflect real-world complexity, where sensor noise, partial data loss, and distribution shifts could affect robustness.
- The framework still relies on labeled data, which may be costly or difficult to obtain in large-scale applications.
- Future work targets semi-supervised or self-supervised learning, domain adaptation or generalization, model compression, and validation in large-scale industrial scenarios.
- The method improves accuracy and interpretability but incurs slightly higher computational overhead than lightweight baselines, creating a deployment trade-off.
- The framework is presented as extensible to larger datasets, streaming data, and emerging technologies such as transformers, additional modalities, and graph-structured information.