Source-linked AI summary
EGNet:Edge Guidance Network for Salient Object Detection
Jia-Xing Zhao, Jiangjiang Liu, Den-Ping Fan, Yang Cao, Jufeng Yang, Ming-Ming Cheng
TL;DR
Existing salient object detectors often produce coarse boundaries, despite the benefits of FCNs. EGNet jointly models complementary salient object and edge information through progressive object-feature extraction, non-local edge-feature extraction, and one-to-one guidance, achieving favorable results across six datasets without pre- or post-processing.
Problem
Existing FCN-based salient object detectors still suffer from coarse object boundaries, while salient edge and object information have not been sufficiently modeled together.
Method
EGNet progressively extracts salient object features, combines local edge and global location information for salient edge features, and couples them across resolutions.
Results
EGNet achieves favorable performance against state-of-the-art methods on six widely used datasets without pre-processing or post-processing.
Takeaways & Limitations
Modeling complementary edge and object information improves salient object boundaries and localization within a single network.
Abstract
from arXiv · showhide
Fully convolutional neural networks (FCNs) have shown their advantages in the salient object detection task. However, most existing FCNs-based methods still suffer from coarse object boundaries. In this paper, to solve this problem, we focus on the complementarity between salient edge information and salient object information. Accordingly, we present an edge guidance network (EGNet) for salient object detection with three steps to simultaneously model these two kinds of complementary information in a single network. In the first step, we extract the salient object features by a progressive fusion way. In the second step, we integrate the local edge information and global location information to obtain the salient edge features. Finally, to sufficiently leverage these complementary features, we couple the same salient edge features with salient object features at various resolutions. Benefiting from the rich edge information and location information in salient edge features, the fused features can help locate salient objects, especially their boundaries more accurately. Experimental results demonstrate that the proposed method performs favorably against the state-of-the-art methods on six widely used datasets without any pre-processing and post-processing. The source code is available at http: //mmcheng.net/egnet/.
1. Introduction
Salient object detection methods often produce coarse or irregular boundaries, motivating EGNet’s explicit use of complementary salient edge and object information. EGNet jointly models these cues and reports strong performance across six datasets.
- Salient object detection identifies the most visually distinctive objects in an image and supports applications including image editing, recognition, and retrieval.
- Hand-crafted contrast-based methods can locate salient objects but often produce irregular or unreliable saliency maps.
- Although larger receptive fields improve object positioning, existing FCN and U-Net methods still do not explicitly model salient object boundaries.
- EGNet focuses on the complementarity between salient edge and salient object information to improve object localization, especially at boundaries.
- EGNet explicitly models both information types within one network, using edge features to preserve boundaries and assist localization.
- Joint optimization lets salient edge and object detection mutually assist each other, significantly improving predicted saliency maps.
- EGNet achieves the best performance under three evaluation metrics against 15 state-of-the-art methods on six widely used datasets.
2. Related Works
Salient object detection evolved from hand-crafted cues and patch-based methods toward CNN and FCN architectures. EGNet differs by explicitly modeling and jointly optimizing complementary edge and object information within one network.
- Early salient object detection methods used hand-crafted cues such as contrast, boundary background, and center prior.
- CNNs improved salient object detection performance and contributed to state-of-the-art results across computer vision tasks.
- Patch-based approaches extracted multi-scale or local features but remained limited by the performance of their image patches.
- FCN-based methods enabled pixel-wise saliency prediction, while recurrent and short-connection architectures addressed scale-space integration.
- Unlike prior edge-cue methods, EGNet uses one base network to jointly optimize salient edge and object detection so the tasks can help each other.
3. Salient Edge Guidance Network
EGNet models salient object and salient edge information jointly through progressive object-feature extraction, non-local edge-feature extraction, and one-to-one guidance across resolutions. Its design uses multi-scale fusion, top-down location propagation, explicit edge supervision, and refined side outputs to improve salient-object localization and segmentation details.
- Architecture: EGNet comprises progressive salient object feature extraction, non-local salient edge feature extraction, and one-to-one guidance modules within one architecture.The architecture also includes feature fusion and supervision across side paths.
- Feature extraction: The backbone provides side features from Conv2-2 through Conv6-3, using Conv2-2 for edge extraction and the other side paths for salient-object features.Conv2-2 is selected because it preserves better edge information.
- Progressive salient object features extraction: PSFEM uses a U-Net-like architecture to generate multi-resolution salient-object features, adding convolutional and ReLU layers plus deep supervision on each side path.A transition layer converts each feature map into a single-channel prediction mask.
- Non-local salient edge features extraction: NLSEM combines local edge information from Conv2-2 with high-level location information propagated top-down from deeper side paths.The propagation addresses dilution of high-level location information during progressive top-down feature return.
- Supervision: The model adds salient-edge supervision and combines edge and object losses into a complementary-information loss.The edge prediction assigns each pixel a salient-edge confidence, while object supervision uses salient and non-salient pixel sets.
- One-to-one guidance module: O2OGM injects salient edge features into enhanced salient-object features at resolutions S(3) through S(6), then refines and fuses the resulting prediction maps.The stated purpose is to improve high-level localization and segmentation details.
4.1. Implementation Details
The model is trained on DUTS with VGG and ResNet backbones under specified optimization settings and equal side-output loss weights.
- Training setup: Training uses DUTS, with VGG and ResNet serving as backbone networks for fair comparison.The implementation is in PyTorch.
- Optimization: The optimization settings include learning rate = 5e-5, weight decay = 0.0005, momentum = 0.9, and side-output loss weight = 1.New convolutional weights use truncated-normal initialization with σ = 0.01, and biases are initialized to 0.
4.2. Datasets and Evaluation Metric
The evaluation uses six public salient-object-detection datasets and three metrics—F-measure, MAE, and S-measure—to assess performance. Table 2 compares these metrics across the six datasets, while Table 3 reports ablations on SOD and DUTS-TE.
- Datasets: The experiments use ECSSD, PASCAL-S, DUT-OMRON, SOD, HKU-IS, and DUTS as benchmark datasets.DUTS is used to train the proposed model; the datasets vary in size, scene complexity, object count, and annotation structure.
- Datasets: DUTS contains 10,553 training images and 5,019 testing images, making it the largest salient object detection benchmark listed.Most DUTS images are challenging and contain objects at various locations and scales.
- Evaluation metrics: The model and competing methods are evaluated using F-measure, mean absolute error, and S-measure.S-measure is included because it evaluates structural information in saliency maps.
- Evaluation metrics: Maximum F-measure is reported across all precision-recall threshold pairs, with β^2 = 0.3 weighting precision more than recall.Precision and recall are computed after thresholding prediction maps into binary images.
- Evaluation setup: Table 2 compares max F-measure, MAE, and S-measure across six datasets, while Table 3 presents ablation analyses on SOD and DUTS-TE.The table conventions indicate whether larger or smaller values are better and identify methods using preprocessing or post-processing.
4.3. Ablation Experiments and Analyses
The ablations examine edge information, top-down location propagation, alternative edge penalties, and multi-resolution fusion. Results support complementary edge-object modeling and one-to-one guidance as effective components of the architecture.
- Experimental setup: The ablations use DUTS-TR for training and evaluate components on SOD and DUTS-TE.These experiments investigate the effects of different network components on relatively difficult evaluation datasets.
- The complementary information modeling: Salient edge information is useful for salient object detection when fused with salient object features to produce the prediction mask.The edge PROG strategy derives edge features from local Conv2-2 features and final saliency features.
- Top-down location propagation: Top-down location propagation extracts more accurate location information from top-level features than the side-path strategy used by edge PROG.The comparison is made between edge PROG and edge TDLP.
- Top-down location propagation: 3.1% and 2.4% F-measure improvements are reported through explicit modeling of complementary edge and object information, without additional time or space consumption.The comparison uses the baseline and edge TDLP configurations across the evaluated datasets.
- Comparison with edge penalties: The NLDF edge penalty provides limited improvement and mainly refines boundaries, whereas the proposed complementary modeling also addresses redundant and missing mask regions.The paper reports better segmentation and localization for the proposed method in visual comparisons.
- Salient edge quality: The proposed method generates more accurate salient edges than NLDF, especially under recall and F-measure.This comparison is reported in Table 4 using salient edges generated by the two methods.
- Multi-resolution fusion: One-to-one multi-resolution guidance is reported as the most suitable fusion method for the full architecture.It is compared with default fusion and progressive multi-resolution fusion.
4.4. Comparison with the State-of-the-art
EGNet is compared with 15 state-of-the-art salient object detection methods on six datasets using quantitative, precision-recall, and visual evaluations. It performs favorably across these comparisons, with improved localization and more coherent object edges.
- EGNet is compared with 15 state-of-the-art methods on six widely used datasets using F-measure, MAE, and S-measure.
- 2.9% and 1.7% improvements in F-measure and S-measure are reported on SOD, while DUTS shows 3.0% and 2.5% improvements.
- 1.9% average F-measure improvement over the current best approach is achieved across six datasets without pre-processing or post-processing.
- EGNet outperforms other methods at most thresholds on precision-recall curves over three datasets.
- Visual comparisons report better salient-object segmentation and localization, with coherent edges and results close to ground truth for a relatively small salient object.
5. Conclusion
The conclusion presents EGNet as a single-network approach for preserving salient-object boundaries through complementary object and edge features. The method performs favorably against state-of-the-art methods on six datasets without pre-processing or post-processing.
- EGNet extracts multi-resolution salient-object features with U-Net, salient-edge features using local edges and global location, and fuses them through one-to-one guidance.
- The salient-edge features improve salient-object boundaries and localization within the network.
- EGNet performs favorably against state-of-the-art methods on six widely used datasets without pre-processing or post-processing.