Source-linked AI summary
Learning Uncertain Convolutional Features for Accurate Saliency Detection
Pingping Zhang, Dong Wang, Huchuan Lu, Hongyu Wang, Baocai Yin
TL;DR
Saliency detection needs more robust predictions and meaningful uncertainty, especially for object boundaries and pixel-wise artifacts. The paper introduces reformulated dropout for uncertain convolutional features and hybrid upsampling, reporting favorable saliency results and improvements on semantic segmentation.
Problem
Deep saliency methods lack probabilistic interpretability, while checkerboard artifacts can make pixel-wise CNN predictions completely wrong.
Method
The model learns uncertain convolutional feature ensembles with reformulated dropout and uses hybrid upsampling to reduce deconvolution artifacts.
Results
The UCF model consistently outperforms existing methods across almost all datasets and evaluation metrics, while its upsampling improves semantic segmentation results.
Takeaways & Limitations
The proposed uncertain feature learning and upsampling methods improve saliency detection and generalize to another pixel-wise vision task.
Takeaways & Limitations
R-Dropout permits any probability-distribution generator, with Bernoulli dropout representing only a special case; its learnable tensor is stochastic during training.
Abstract
from arXiv · showhide
Deep convolutional neural networks (CNNs) have delivered superior performance in many computer vision tasks. In this paper, we propose a novel deep fully convolutional network model for accurate salient object detection. The key contribution of this work is to learn deep uncertain convolutional features (UCF), which encourage the robustness and accuracy of saliency detection. We achieve this via introducing a reformulated dropout (R-dropout) after specific convolutional layers to construct an uncertain ensemble of internal feature units. In addition, we propose an effective hybrid upsampling method to reduce the checkerboard artifacts of deconvolution operators in our decoder network. The proposed methods can also be applied to other deep convolutional networks. Compared with existing saliency detection methods, the proposed UCF model is able to incorporate uncertainties for more accurate object boundary inference. Extensive experiments demonstrate that our proposed saliency model performs favorably against state-of-the-art approaches. The uncertain feature learning mechanism as well as the upsampling method can significantly improve performance on other pixel-wise vision tasks.
1. Introduction
The paper targets robustness and uncertainty in saliency detection, while addressing checkerboard artifacts that can undermine pixel-wise CNN predictions. It proposes uncertain feature learning and new upsampling strategies for accurate saliency inference.
- Saliency detection remains challenging in real-world scenarios, and improving model robustness has been largely ignored.
- Deep saliency methods lack probabilistic interpretability, although uncertainty could provide confidence information and improve boundary inference.
- Checkerboard artifacts arise in low-to-high-resolution pixel-wise prediction and can cause FCN training failure or completely wrong predictions.
- The model uses an encoder-decoder FCN with pixel-wise classification, reformulated dropout for uncertain convolutional features, and hybrid upsampling to reduce artifacts.
- The proposed uncertain features are intended to support more accurate saliency detection without additional parameterization.
2. Related Work
Related work includes deep saliency models and uncertainty modeling in deep learning. The paper distinguishes its encoder-decoder architecture and reformulated convolutional dropout from prior FCN and Bayesian approaches.
- Deep saliency detection: Recent deep saliency methods set benchmarks by modeling object appearances and varied scenarios with strong learned representations.
- Comparison with related FCNs: Unlike related FCNs, this model uses an encoder-decoder architecture and multiple reformulated dropouts to learn uncertain convolutional features.
- Model uncertainty: Prior uncertainty work links dropout to probabilistic deep Gaussian processes, while Bayesian SegNet uses test-time Monte Carlo sampling for pixel-wise uncertainty.
3. The Proposed Model
The proposed model learns uncertain convolutional features with R-Dropout in an encoder–decoder FCN and uses hybrid upsampling to reduce deconvolution artifacts. Its design combines probabilistic feature selection with restricted-filter and interpolation-based upsampling strategies.
- Architecture: The hybrid FCN uses an encoder for high-level feature extraction, a decoder for low-level information reconstruction, and a pixel-wise classifier for saliency prediction.The encoder–decoder architecture is generalized from a stacked denoising auto-encoder.
- Uncertain Convolutional Features: R-Dropout after convolutional layers acts as an uncertain ensemble of convolutional features with element-wise probabilistic behavior.The paper states that the mask generator determines the stochastic mask and that R-Dropout builds uncertainty in deep convolutional features.
- Uncertain Convolutional Features: When R-Dropout precedes convolution, disturbed activations are convolved to produce subsequent feature maps, while the network learns convolutional weights and biases.This is the Conv+R-Dropout+Conv case.
- Uncertain Convolutional Features: Before max-pooling, R-Dropout samples a retained activation from an ordered pooling region according to a multinomial distribution, including an all-dropped event.The retained activation is selected probabilistically; the all-dropped event has probability P0 = q^n.
- Hybrid Upsampling: Deconvolution is equivalent to convolving over an input with inserted zeros, and incompatible filter and stride sizes can create overlapping contributions and checkerboard artifacts.The example uses a 3 × 3 kernel, stride 2, and a 5 × 5 padded input.
- Hybrid Upsampling: Hybrid upsampling combines restricted-filter deconvolution with linear interpolation followed by 1x1 convolution, summing the two outputs.Restricting filter size to a multiple of stride avoids the overlapping issue, while interpolation separates resizing from equivalent convolution.
4. Experiments
Experiments evaluate UCF on multiple saliency datasets and additional pixel-wise tasks, showing strong performance, improved boundaries, and benefits from uncertain learning and hybrid upsampling.
- Experimental setup: The evaluation uses six saliency datasets, three metrics, and comparisons against ten state-of-the-art methods.Metrics are PR curves, F-measure, and MAE; the training set contains 80,000 augmented MSRA10K images.
- Saliency detection: UCF consistently outperforms existing methods across almost all datasets and evaluation metrics.On ECSSD and SED, it improves F-measure over the best existing algorithm by 3.9% and 6.15%, respectively.
- Saliency detection: UCF ranks second on HKU-IS and PASCAL-S, where some competing methods benefit from additional PASCAL VOC segmentation pretraining.The authors also report lower MAE on most datasets.
- Saliency detection: Visual comparisons show reliable detection under low contrast, multiple disconnected objects, and boundary-adjacent objects, with more accurate salient-object boundaries.These cases are illustrated in Fig. 7.
- Ablation studies: Ablation studies indicate that uncertain learning improves robust saliency inference and that interpolation-based upsampling performs better than regular deconvolution.The joint comparisons evaluate uncertain learning and two upsampling strategies.
- Generalization evaluation: UCF also achieves comparable eye-fixation results, supporting generalization to other pixel-wise vision tasks.Experiments use MIT300, iSUN, and SALICON benchmarks.
5. Conclusion
The paper proposes a fully convolutional saliency detector that combines probabilistic training through reformulated dropout with artifact-reducing upsampling. Extensive evaluations show improved saliency detection and good generalization to other pixel-wise vision tasks.
- Conclusion: The proposed fully convolutional network uses reformulated dropout to learn uncertain convolutional features for saliency prediction.The method supports probabilistic training and inference.
- Conclusion: A new upsampling method reduces deconvolution artifacts and explicitly encourages accurate saliency-object boundaries.The conclusion links this design to improved boundary inference.
- Conclusion: Extensive evaluations demonstrate improved saliency-detection performance and good generalization to other pixel-wise vision tasks.The paper reports generalization beyond saliency detection.