Source-linked AI summary
Deep Unsupervised Saliency Detection: A Multiple Noisy Labeling Perspective
Jing Zhang, Tong Zhang, Yuchao Dai, Mehrtash Harandi, Richard Hartley
TL;DR
Deep saliency detection relies on costly pixel-level annotations, while traditional unsupervised methods generalize more broadly but are weaker. The paper learns from multiple noisy unsupervised saliency maps with jointly optimized latent prediction and explicit noise modeling, achieving a wide-margin advantage over unsupervised methods and comparable performance to state-of-the-art supervised methods.
Problem
Deep saliency methods require labor-intensive human annotations that may hinder generalization, motivating saliency learning without labeled data.
Method
The framework jointly optimizes a latent saliency prediction module and a probabilistic noise modeling module using multiple noisy maps from unsupervised saliency methods.
Results
The method outperforms traditional unsupervised saliency methods with a wide margin and achieves highly comparable performance with state-of-the-art deep supervised methods.
Takeaways & Limitations
Unsupervised saliency maps can serve as multiple noisy labels for end-to-end deep saliency learning without human annotations.
Takeaways & Limitations
The noise model assumes zero-mean Gaussian noise that is independent for each pixel and depends on the input image rather than the labeller.
Abstract
from arXiv · showhide
The success of current deep saliency detection methods heavily depends on the availability of large-scale supervision in the form of per-pixel labeling. Such supervision, while labor-intensive and not always possible, tends to hinder the generalization ability of the learned models. By contrast, traditional handcrafted features based unsupervised saliency detection methods, even though have been surpassed by the deep supervised methods, are generally dataset-independent and could be applied in the wild. This raises a natural question that "Is it possible to learn saliency maps without using labeled data while improving the generalization ability?". To this end, we present a novel perspective to unsupervised saliency detection through learning from multiple noisy labeling generated by "weak" and "noisy" unsupervised handcrafted saliency methods. Our end-to-end deep learning framework for unsupervised saliency detection consists of a latent saliency prediction module and a noise modeling module that work collaboratively and are optimized jointly. Explicit noise modeling enables us to deal with noisy saliency maps in a probabilistic way. Extensive experimental results on various benchmarking datasets show that our model not only outperforms all the unsupervised saliency methods with a large margin but also achieves comparable performance with the recent state-of-the-art supervised deep saliency methods.
1. Introduction
Deep supervised saliency methods achieve strong performance but require costly pixel-level annotations that may hinder generalization, whereas unsupervised methods are dataset-independent. The paper therefore learns saliency from multiple noisy unsupervised maps using joint latent prediction and noise modeling.
- Motivation: Deep supervised saliency methods outperform unsupervised methods but depend heavily on labor-intensive pixel-level annotations that may hinder generalization.Unsupervised methods remain dataset-independent and applicable in the wild.
- Motivation: The framework learns saliency without human annotations by exploiting existing unsupervised saliency maps generated from different handcrafted priors.These maps provide complementary information but contain method-specific bias and noise relative to human annotations.
- Framework: The method jointly optimizes a saliency prediction module and a noise modeling module to infer latent saliency maps from noisy labels.The prediction module maps color images to latent saliency, while the noise module estimates noise in different saliency maps.
- Results: Extensive evaluation on seven benchmarking datasets outperforms existing unsupervised methods with a wide margin and achieves comparable results to state-of-the-art supervised methods.The paper reports this as a central contribution and headline result.
- Framework: The framework’s overall optimization converges in several rounds, with each round updating the noise module.An epoch is a complete pass through training data, an iteration is a complete batch pass, and a round updates the noise module.
2. Related Work
Earlier saliency methods rely on handcrafted priors, while deep methods improve semantic representation but typically use supervision. This work positions noisy-label learning as an alternative, replacing manually designed fusion with joint latent-saliency and noise modeling.
- Saliency detection methods: Unsupervised saliency methods use handcrafted priors such as center, global contrast, and background connectivity, while supervised methods learn mappings from annotated images.Deep learning methods are examples of supervised saliency detection.
- Saliency detection methods: Deep networks capture high-level semantic features and have become methods of choice for saliency detection.The passage contrasts their effectiveness with unsupervised and nondeep supervised methods.
- Learning with noisy labels: Few studies explicitly address saliency learning with unreliable or noisy labels.Existing noisy-label research mainly concerns classification models with inaccurate class labels.
- Existing deep unsupervised learning: The prior deep unsupervised method SBF fuses unsupervised maps using manually designed intra-image and inter-image rules, with computationally expensive recursive optimization.Its curriculum replaces low-reliability inter-image maps with corresponding maps.
- Existing deep unsupervised learning: This paper instead jointly optimizes latent saliency prediction and noise modeling to provide a simpler alternative that outperforms SBF and existing unsupervised methods.The proposed framework also reports competitive performance relative to recent deep supervised methods.
3. Our Framework
The framework learns a latent saliency map from multiple handcrafted saliency maps by jointly optimizing a saliency prediction module and an explicit probabilistic noise model. Its losses balance agreement with noisy labels against modeling their estimated noise, while the noise module is used during training but excluded during testing.
- Framework: The method uses an end-to-end deep framework that learns saliency without human annotations from existing unsupervised saliency methods.A fully convolutional network provides the latent saliency prediction component.
- Framework: The saliency prediction module maps an input color image to a latent saliency map, while the noise modeling module fits the noise in the handcrafted labels.The two modules operate collaboratively during training to fit the noisy saliency maps.
- Noise Modeling: Each handcrafted saliency map is modeled as the latent network prediction plus noise, allowing method-specific variability to be represented explicitly.The noise is modeled as zero-mean Gaussian and independent across images and pixel locations, with image-specific variance parameters.
- Loss Function: The objective combines prediction loss and noise loss, with λ balancing their competing effects during optimization.Higher modeled noise variance increases Lpred while decreasing Lnoise; lower variance has the opposite effect.
- Loss Function: Prediction loss measures agreement between modeled noisy-label predictions and handcrafted saliency maps using element-wise cross-entropy across images, pixels, and labellers.Predicted values are truncated to the range [0, 1].
- Training and Testing: The noise module is updated iteratively after prediction loss convergence and is excluded during testing, where the latent prediction is the final saliency map.Noise parameters are updated after the fully convolutional network converges for the prediction loss.
4. Experimental Results
Experiments evaluate the proposed unsupervised saliency framework on seven benchmark datasets against handcrafted, supervised deep, and unsupervised deep methods. Results show that explicit noise modeling improves over direct noisy-label baselines and achieves performance close to clean-label supervision, with strong quantitative and qualitative comparisons.
- Experimental setup: The model is evaluated on seven saliency datasets using MAE, mean F-measure, and PR curves.MSRA-B supplies noisy labels for training and validation, while the remaining data and other datasets support evaluation.
- Experimental setup: The comparison includes four handcrafted unsupervised methods, ten supervised deep methods, and one unsupervised deep method.The handcrafted methods are RBD, DSR, MC, and HS; the supervised deep methods include DSS, NLDF, Amulet, UCF, SRM, DMT, RFCN, DeepMC, MDF, and DC.
- Baseline experiments: The proposed framework clearly outperforms BL1 and BL2, which directly train on noisy labels or their averaged saliency maps.BL1 uses 12,000 image-label pairs, whereas BL2 uses 3,000 averaged noisy labels; averaging can worsen supervision because the source maps may be complementary or controversial.
- Baseline experiments: The unsupervised method achieves highly comparable results to BL3, which trains the latent saliency module with clean ground-truth labels.The comparison supports jointly learning latent saliency maps and modeling noise without human annotations.
- Quantitative comparison: Deep supervised methods reduce MAE by 2%-12% relative to traditional methods across seven benchmark datasets.Table 2 compares the proposed method with eleven recent deep methods and five conventional methods.
- Quantitative comparison: On THUR, the proposed method achieves the best mean F-measure/MAE of 0.7322/0.0811, compared with 0.7115/0.0854 for the state-of-the-art competing method.On MSRA-B, it reaches a mean F-measure of 0.8770 versus 0.8970 for the strongest deep supervised methods, and it ranks third on DUT by mean F-measure.
- Qualitative comparison: Qualitative comparisons report stronger suppression of background and better detection of complex, low-contrast, and small salient objects.The proposed method captures all three penguins in a low-contrast image and the whole salient region in the small-object example.
- Ablation studies: Performance consistently improves over updating rounds as the noise model and latent saliency predictor are iteratively optimized.The method starts from zero-noise initialization and converges to the desired state only after several updating rounds.
5. Conclusions
The framework learns saliency without human annotations by combining multiple noisy unsupervised saliency maps through latent prediction and explicit noise modeling. Experiments show superiority over traditional unsupervised methods and comparable performance with state-of-the-art supervised methods, while future work targets broader dense prediction settings.
- The framework learns saliency without human-annotated maps by treating outputs from conventional methods as multiple noisy saliency maps.It jointly optimizes a latent saliency prediction module and an explicit noise modeling module.
- Extensive experiments across benchmarking datasets show that the method outperforms traditional unsupervised saliency methods by a wide margin.
- The unsupervised method achieves highly comparable performance with current state-of-the-art deep supervised saliency methods.
- The framework is intended for saliency detection, with semantic segmentation and monocular depth estimation identified as possible future extensions.