Source-linked AI summary
Highly Efficient Salient Object Detection with 100K Parameters
Shang-Hua Gao, Yong-Qiang Tan, Ming-Ming Cheng, Chengze Lu, Yunpeng Chen, Shuicheng Yan
TL;DR
Salient object detection requires accurate pixel-level predictions but is often too resource-intensive for low-power devices. The paper introduces gOctConv and dynamic weight decay to reduce redundancy and learn channel counts, then builds CSNet. CSNet achieves comparable benchmark performance with about 0.2% of the parameters of large models, while 80% parameter reduction causes negligible performance drop.
Problem
SOD models are resource-hungry despite requiring high- and low-level features for accurate pixel localization and boundary refinement, limiting use on low-power devices.
Method
The paper proposes gOctConv for flexible in-stage and cross-stage multi-scale processing, together with dynamic weight decay that supports learnable channels and parameter sparsity.
Results
CSNet achieves comparable performance with ∼0.2% parameters (100k) of large models on popular salient object detection benchmarks.
Takeaways & Limitations
Dynamic weight decay allows 80% of parameters to be reduced with negligible performance drop, enabling an extremely lightweight SOD model.
Takeaways & Limitations
Manually tuning the overall channel split ratio across resolutions may produce a sub-optimal balance between performance and computational cost.
Abstract
from arXiv · showhide
Salient object detection models often demand a considerable amount of computation cost to make precise prediction for each pixel, making them hardly applicable on low-power devices. In this paper, we aim to relieve the contradiction between computation cost and model performance by improving the network efficiency to a higher degree. We propose a flexible convolutional module, namely generalized OctConv (gOctConv), to efficiently utilize both in-stage and cross-stages multi-scale features, while reducing the representation redundancy by a novel dynamic weight decay scheme. The effective dynamic weight decay scheme stably boosts the sparsity of parameters during training, supports learnable number of channels for each scale in gOctConv, allowing 80% of parameters reduce with negligible performance drop. Utilizing gOctConv, we build an extremely light-weighted model, namely CSNet, which achieves comparable performance with about 0.2% parameters (100k) of large models on popular salient object detection benchmarks.
1 Introduction
SOD models must balance accurate pixel-level prediction with the severe resource constraints of low-power devices. The paper addresses this gap with gOctConv, dynamic weight decay, and CSNet, achieving comparable performance with dramatically fewer parameters.
- SOD supports applications including image retrieval, visual tracking, photographic composition, image quality assessment, and weakly supervised semantic segmentation.
- Resource-hungry SOD models are difficult to deploy on low-power devices, motivating extremely lightweight models with strong performance.
- Accurate SOD requires large-scale high-level features for object localization and fine low-level features for boundary refinement.
- gOctConv accepts arbitrary numbers of in-stage and cross-stage feature scales, expanding multi-scale representations beyond vanilla OctConv.
- Dynamic weight decay enables learnable channel counts for each scale and reduces 80% of parameters with negligible performance drop.
- CSNet exploits in-stage and cross-stages multi-scale features and can be trained from scratch without ImageNet pre-training.
- CSNet achieves comparable performance with ∼0.2% parameters (100k) of large models on popular SOD benchmarks.
2 Related Works
Prior SOD research progressed from hand-crafted and patch-based CNN features to end-to-end pixel-level prediction using multi-stage representations and edge cues. Lightweight modules developed for classification are not directly suitable for SOD because SOD requires richer multi-scale detail.
- Early SOD methods mainly used hand-crafted features, while later CNN methods extracted informative features from image patches.
- FCN-based approaches formulate SOD as an end-to-end pixel-level prediction task.
- Recent SOD methods combine fine details and global semantics from different backbone stages, sometimes incorporating edge cues.
- Lightweight classification models use inverted blocks, channel shuffling, and SE attention to improve efficiency.
- Aggressive early downsampling in classification models saves FLOPs but is unsuitable for SOD, which requires multi-scale information and detailed representations.
- Channel-level pruning methods identify redundant filters using norms, next-layer statistics, BatchNorm scaling factors, geometric medians, or generated weights.
3 Light-weighted Network with Generalized OctConv
The network combines generalized OctConv with dynamic weight decay to process flexible multi-scale features and learn compact channel allocations for efficient salient object detection.
- gOctConv accepts arbitrary-resolution inputs from in-stage and cross-stages features, unlike vanilla OctConv’s two fixed-channel scales.
- The network stacks ILBlocks across four feature-extractor stages and uses gOctConvs for cross-stages fusion toward a high-resolution output.The stages contain 3, 4, 6, and 4 ILBlocks, respectively.
- Dynamic weight decay adjusts suppression using channel features, aiming to stabilize weight and output distributions while introducing sparsity.The paper uses global average pooling as the channel feature metric.
- BatchNorm scaling factors γ indicate channel importance, enabling removal of channels whose γ falls below a small threshold.The reported distribution separates important and redundant weights, with unimportant weights suppressed to nearly zero (wi < 1e−20).
- The compression procedure applies dynamic decay during training, eliminates redundant channels in gOctConv, and fine-tunes the remaining weights.The algorithm computes a per-channel metric, performs backward updates with dynamic decay, prunes channels, and then fine-tunes.
4 Experiments
Experiments evaluate CSNet’s salient object detection performance, compression, dynamic weight decay, learned channels, and runtime. The results show substantial efficiency gains with comparable performance, while dynamic weight decay supports sparsity and faster convergence.
- Experimental Settings: CSNet is trained from scratch without ImageNet pre-training and evaluated using Fβ and MAE across six salient object detection datasets.The datasets are ECSSD, PASCAL-S, DUT-O, HKU-IS, SOD, and DUTS-TE.
- Performance Analysis: Cross-stage fusion gives ResNet+CSF similar performance to ResNet+PoolNet with 53% parameters and 21% FLOPs.gOctConvs provide high- and low-resolution features across backbone stages while producing high-resolution outputs.
- Dynamic Weight Decay: Dynamic weight decay yields better performance at the same complexity and makes performance less sensitive to model complexity.It also promotes sparsity through feature-dependent suppression of BatchNorm weights.
- Dynamic Weight Decay: Dynamic weight decay improves MAE and accelerates convergence compared with training without dynamic weight decay.The method stabilizes feature distributions across BatchNorm channels.
- Learned Channels: Larger initial channel widths improve pruned-model performance, while compressed models can match or exceed their initial models.Dynamic weight decay contributes to the stable distributions associated with these compressed-model results.
- Run-Time: CSNet provides more than 10x acceleration over large-weight models and up to 6% higher F-measure than similarly fast models designed for other tasks.Runtime is measured on a single i7-8700K CPU core with 224 × 224 inputs.
5 Conclusion
The paper introduces generalized OctConv and dynamic weight decay to reduce redundancy while exploiting multi-scale features. The resulting CSNet achieves comparable salient object detection performance with approximately 100k parameters, or 0.2% of large models.
- Conclusion: Generalized OctConv flexibly uses in-stage and cross-stage multi-scale features while dynamic weight decay promotes parameter sparsity.The scheme supports learnable channel counts for each scale and allows 80% of parameters to be removed with negligible performance drop.
- Conclusion: CSNet achieves comparable performance to large salient object detection models with approximately 100k parameters, or 0.2% of their parameters.The comparison is reported on popular salient object detection benchmarks.