Source-linked AI summary
Learning to predict crisp boundaries
Ruoxi Deng, Chunhua Shen, Shengjun Liu, Huibing Wang, Xinru Liu
TL;DR
CNN-based edge detectors often produce overly thick boundaries, partly in the context of highly imbalanced edge and non-edge pixels, and typically require post-processing for crisp results. The paper introduces a similarity-based loss and an end-to-end bottom-up/top-down network to produce crisp boundaries directly. It reports state-of-the-art performance on BSDS500 and NYUDv2, with ODS F-scores of 0.815 and 0.762, respectively.
Problem
CNN-based edge detectors commonly predict overly thick boundaries and rely on post-processing, while edge and non-edge pixels are highly imbalanced.
Method
The paper combines a bottom-up/top-down fully convolutional network with a Dice-based similarity loss for end-to-end crisp edge prediction.
Results
0.815 ODS F-score on BSDS500 and 0.762 ODS F-score on NYUDv2 are reported as state-of-the-art results.
Takeaways & Limitations
The study concludes that a carefully designed loss and convolutional encoder-decoder network can achieve excellent boundary detection results with crisp predictions.
Takeaways & Limitations
Limited training data constrained exploration of more sophisticated bottom-up/top-down architectures, which failed to achieve better performance in experiments.
Abstract
from arXiv · showhide
Recent methods for boundary or edge detection built on Deep Convolutional Neural Networks (CNNs) typically suffer from the issue of predicted edges being thick and need post-processing to obtain crisp boundaries. Highly imbalanced categories of boundary versus background in training data is one of main reasons for the above problem. In this work, the aim is to make CNNs produce sharp boundaries without post-processing. We introduce a novel loss for boundary detection, which is very effective for classifying imbalanced data and allows CNNs to produce crisp boundaries. Moreover, we propose an end-to-end network which adopts the bottom-up/top-down architecture to tackle the task. The proposed network effectively leverages hierarchical features and produces pixel-accurate boundary mask, which is critical to reconstruct the edge map. Our experiments illustrate that directly making crisp prediction not only promotes the visual results of CNNs, but also achieves better results against the state-of-the-art on the BSDS500 dataset (ODS F-score of .815) and the NYU Depth dataset (ODS F-score of .762).
1 Introduction
CNN-based boundary detectors produce semantically meaningful contours but commonly predict edges that are substantially thicker than crisp boundaries. This work addresses the thickness problem with an end-to-end network and loss that produce crisp predictions without post-processing.
- CNN-based detectors often produce semantically meaningful contours, but their predicted boundaries are much thicker than those from classic methods.An HED polar-bear edge is roughly 10 pixels wide versus roughly 4 pixels for Sobel; similar thickness appears in RCF, Casenet, and CEDN.
- Because edge-thinning post-processing is commonly applied, prior work rarely discusses the thickness of initial boundary predictions.A refinement architecture addressed crisp edges, but experiments reported only slight improvement over HED.
- The proposed method combines a bottom-up/top-down fully convolutional network with a simple loss to predict crisp boundaries without post-processing.The network learns hierarchical features and resolves prediction ambiguity for image-to-boundary prediction.
- 0.815 ODS F-score is achieved on BSDS500, while the fast version reaches 0.808 at 30 FPS.
2 Related work
Deep learning methods have become central to edge detection by learning hierarchical features and producing semantic contours. This work targets their thick predictions, proposing crisp, post-processing-free boundaries and reporting gains over prior state-of-the-art methods.
- Traditional detectors compute image gradients, while Canny additionally removes noise and applies non-maximum suppression to obtain one-pixel-wide contours.
- CNN-based methods learn low-, middle-, and high-level features and use them to detect semantically meaningful contours.DeepEdge and DeepContour use patches, whereas HED directly maps an image to predictions with a fully convolutional network and weighted cross-entropy loss.
- HED, RCF, CEDN, and CASENet extend CNN-based edge detection through skip-layer, encoder-decoder, extra-data, or semantic edge-classification designs.
- The weighted cross-entropy loss used by HED may contribute to thick edges, motivating the paper’s verification and alternative loss design.
- The proposed method aims to predict crisp edges without post-processing and outperforms previous state-of-the-art methods on BSDS500 and NYUDv2.
3 The proposed method
The method combines a Dice-based loss with cross-entropy and a bottom-up/top-down network to address class imbalance and produce crisp edge predictions. Its architecture fuses hierarchical features while refining resolution, achieving strong BSDS500 performance with fewer parameters than an unmodified variant.
- 3.1 Revisiting weighted cross-entropy: The edge/non-edge imbalance makes standard cross-entropy fail, while class-balanced weighting enables training but is associated with overly thick predictions.The authors report that standard cross-entropy produces an embossed image, whereas weighted loss trains successfully but yields thicker contours.
- 3.1 The proposed loss function: The proposed loss compares prediction and ground-truth masks through Dice similarity, avoiding explicit edge/non-edge loss balancing.The Dice loss is combined with cross-entropy to minimize image-level and pixel-level differences.
- 3.2 The proposed loss function: Using only the proposed Dice loss achieves an ODS F-score of .805 on the BSDS500 dataset.
- 3.3 Network architecture: The network adopts a bottom-up/top-down architecture to capture hierarchical features and refine feature resolution for edge detection.The design uses a VGG-16 backbone and repeatedly fuses top-down mask encoding with side-output features.
- 3.3 Network architecture: 15.69M parameters yield an ODS of .808 on BSDS500, whereas removing two modifications increases parameters to 22.64M and lowers ODS to .802.
- 3.3 Network architecture: Limited training data may contribute to overfitting and constrain the effectiveness of more sophisticated bottom-up/top-down architectures.The BSDS500 dataset is described as having only 200 training images.
4 Experiments
Experiments evaluate implementation choices, loss contributions, comparisons with HED and state-of-the-art detectors, and performance on BSDS500 and NYUDv2. The proposed method improves boundary sharpness without post-processing while achieving strong accuracy and real-time inference.
- Experimental setup: Experiments use BSDS500 and NYUD datasets, with ablation, HED comparison, and state-of-the-art evaluations.Accuracy is measured using ODS and OIS, with evaluations performed before and after non-maximum suppression.
- Ablation study: The ablation study evaluates the encoder-decoder network, ResNeXt blocks, balanced cross-entropy, and the proposed fusion loss.The baseline removes ResNeXt blocks from the backward-refining path, while variants isolate the fusion loss.
- Ablation study: 0.693 ODS F-score before NMS is achieved with the fusion loss, increasing from 0.674 and improving boundary sharpness.The encoder-decoder baseline reaches 0.797 ODS on BSDS500, compared with 0.790 for HED.
- Improving the crispness of HED: Almost 9% higher non-NMS performance is obtained by HED-FL than HED-BL, showing that the fusion loss also improves crispness in HED.Both HED variants use deep supervision, while the training data generation differs from the original HED work.
- State-of-the-art comparisons: 0.808 ODS F-score is obtained after adding extra training data, improving from 0.800; multi-scale testing reaches the same top performance as CED.The comparison notes that CED trains on both train and validation sets, whereas this method uses only the train set.
- Efficiency: The single-scale detector runs at 30FPS on a GTX980 GPU, while its non-NMS BSDS500 ODS performance reaches 0.693.The reported runtime and accuracy support use without post-processing in the evaluated setting.
- NYUDv2 dataset: 0.762 ODS F-score is achieved on NYUDv2 by averaging predictions from RGB and HHA models, with sharper boundaries than RCF reported qualitatively.The HHA feature encodes depth as horizontal disparity, height above ground, and angle with gravity.
5 Conclusions
The paper presents a simple edge-detection method combining a carefully designed loss with a convolutional encoder-decoder network. It reports state-of-the-art results and identifies future applications requiring sharp boundaries and fast processing.
- Conclusion: The method achieves state-of-the-art edge detection using a carefully designed loss and a simple convolutional encoder-decoder network.The conclusion characterizes the approach as simple yet effective.
- Future work: Future work will extend the edge detector to object detection and optical flow, where boundary sharpness and fast processing are required.These applications are presented as planned extensions rather than evaluated results.