Source-linked AI summary
Pixel Difference Networks for Efficient Edge Detection
Zhuo Su, Wenzhe Liu, Zitong Yu, Dewen Hu, Qing Liao, Qi Tian, Matti Pietikäinen, Li Liu
TL;DR
CNN edge detectors can be accurate but often require large pretrained backbones, while traditional edge operators are rarely integrated into modern CNNs. The paper proposes PDC and PiDiNet to combine gradient-sensitive pixel differences with efficient CNN representations. Across BSDS500, NYUD, and Multicue, PiDiNet is reported to achieve strong edge-detection accuracy with high training and inference efficiency.
Problem
CNN-based edge detection commonly depends on large pretrained backbones that consume memory and energy, while traditional edge-detector knowledge is rarely integrated with modern CNNs.
Method
The paper introduces pixel difference convolution and builds PiDiNet with an efficient backbone and task-specific side structure for edge detection.
Results
PiDiNet achieves state-of-the-art accuracy-efficiency results across BSDS500, NYUD, and Multicue, including human-level BSDS500 performance when trained from scratch.
Takeaways & Limitations
PiDiNet shows that high-performing edge detection can be obtained with a memory-friendly, high-speed architecture trained using limited data rather than a large-scale pretrained backbone.
Takeaways & Limitations
PiDiNet is slightly slower than the baseline when RPDC becomes a 5x5 convolution after conversion.
Abstract
from arXiv · showhide
Recently, deep Convolutional Neural Networks (CNNs) can achieve human-level performance in edge detection with the rich and abstract edge representation capacities. However, the high performance of CNN based edge detection is achieved with a large pretrained CNN backbone, which is memory and energy consuming. In addition, it is surprising that the previous wisdom from the traditional edge detectors, such as Canny, Sobel, and LBP are rarely investigated in the rapid-developing deep learning era. To address these issues, we propose a simple, lightweight yet effective architecture named Pixel Difference Network (PiDiNet) for efficient edge detection. Extensive experiments on BSDS500, NYUD, and Multicue are provided to demonstrate its effectiveness, and its high training and inference efficiency. Surprisingly, when training from scratch with only the BSDS500 and VOC datasets, PiDiNet can surpass the recorded result of human perception (0.807 vs. 0.803 in ODS F-measure) on the BSDS500 dataset with 100 FPS and less than 1M parameters. A faster version of PiDiNet with less than 0.1M parameters can still achieve comparable performance among state of the arts with 200 FPS. Results on the NYUD and Multicue datasets show similar observations. The codes are available at https://github.com/zhuoinoulu/pidinet.
1. Introduction
Deep CNNs improve edge detection but typically rely on large pretrained backbones, creating efficiency and data demands. PiDiNet addresses this trade-off by combining pixel-difference operations with an efficient CNN architecture.
- Deep CNNs provide rich hierarchical representations for edge detection, extending earlier gradient- and derivative-based approaches.
- Traditional operators explicitly compute pixel differences and encode useful gradient information, but their shallow structures limit representational power.
- Pixel difference convolution first computes image pixel differences and then applies kernel weights, integrating gradient information with CNN learning.The paper reports that PDC improves output edge-map quality.
- Leading CNN edge detectors are memory consuming, energy hungry, inefficient at inference, and dependent on large-scale pretraining because annotated edge data are limited.VGG16 pretrained on ImageNet is given as a representative routine.
- PiDiNet combines an efficient backbone with an efficient task-specific side structure for robust and accurate edge detection at high efficiency.
2. Related Work
Related work includes CNN-assisted traditional edge detectors, lightweight edge-detection architectures, and convolutional operators derived from local binary patterns. These approaches motivate a compact, learnable integration of gradient information.
- Canny and SE can provide candidate contour points or auxiliary relaxed labels before or during CNN-based edge-detection training.
- Existing lightweight edge-detection networks reduce size and improve speed, but reported detection accuracies can remain unsatisfactory.Some lightweight methods do not require large-scale pretraining.
- PDC relates to LBC and CDC by calculating pixel differences during convolution, while retaining a compact and learnable gradient-extraction process.LBC uses predefined sparse binary filters, whereas CDC uses learnable weights.
3. Pixel Difference Convolution
Pixel difference convolution replaces local input pixels with selected pixel differences before weighted summation, deriving flexible operators from LBP-style pixel relations. Its learned kernels can be converted for efficient inference.
- Pixel difference convolution replaces the original pixels in a local feature patch with pixel differences before convolution produces each output feature.The differences are multiplied element-wise by kernel weights and summed.
- PDC computes output as a weighted sum of pixel differences, with the selected pixel pairs defining the difference terms.The passage defines x_i as input pixels and w_i as kernel weights.
- PDC selects pixel pairs according to strategies inspired by LBP and extended LBP descriptors, including angular and radial relations.
- The paper derives central, angular, and radial PDC instances, with APDC using eight angular pixel pairs for a 3 × 3 kernel.
- Training encourages PDC kernels to preserve useful pixel-relation encodings while learning richer representations than discrete ELBP codes.
- After training, PDC can be converted to vanilla convolution by storing differences between kernel weights, preserving inference efficiency.The paper notes that the unconverted PDC has doubled computational cost and memory footprint relative to vanilla convolution.
4. PiDiNet Architecture
PiDiNet combines a slim, efficient backbone with a task-specific side structure for hierarchical, multi-scale edge representations. Replacing depth-wise convolutions with pixel difference convolution yields the proposed architecture, trained with annotator-robust supervision.
- Efficient Backbone: The backbone uses separable depth-wise convolutions with shortcuts to support a slim structure, fast inference, and efficient implementation.The design avoids sophisticated multi-branch lightweight structures because they may not appeal to parallel implementation.
- Efficient Side Structure: The side structure generates an edge map from each stage and applies side losses for deep supervision.A compact dilation convolution module enriches multi-scale edge information before feature refinement.
- Pixel Difference Convolution: Replacing vanilla convolution in each residual block’s 3 × 3 depth-wise convolutional layer with PDC produces PiDiNet.The architecture without this replacement is treated as the baseline.
- Architecture Design: The architecture omits normalization layers because training-image resolutions are not uniform.
- Loss Function: The annotator-robust loss discards pixels marked positive by fewer than η annotators and weights negative samples using β, with α = λ · (1 − β).The loss is applied to every generated edge map, including the final map.
5. Experiments
Experiments evaluate PiDiNet across datasets, configurations, model scales, and comparisons with prior methods. The results show that pixel-difference convolutions improve the baseline while preserving strong accuracy and efficiency.
- Experimental Setup: PiDiNet is evaluated on BSDS500, NYUD, and Multicue using ODS and OIS F-measures, with single-scale evaluation used for efficiency comparisons.BSDS500 experiments use augmented training data, with optional VOC mixing and repeated train/evaluation splits.
- Ablation Study: The best ablation configuration, ‘[CARV]×4’, sequentially combines four convolution types in each stage because they capture gradient information in different encoding directions.The configurations are sampled by progressively increasing the number of pixel-difference convolutions across 16 backbone blocks.
- Ablation Study: PDC-configured architectures consistently outperform corresponding vanilla-convolution baselines across multiple network scales.Table 3 varies the number of channels C while training on BSDS500 and evaluating on its validation set.
- Ablation Study: Adding shortcuts preserves previously captured gradient information, while CSAM and CDCM improve performance with additional computational cost, enabling an accuracy-efficiency trade-off.Removing CSAM and CDCM yields the lighter PiDiNet-L variant.
- Network Scalability: With 710K parameters, PiDiNet supports training from scratch; the normal model reaches 0.789 ODS on BSDS500 without VOC, compared with HED’s 0.788.Training with more data generally raises accuracy, while widening under limited data can cause overfitting.
- Comparison with State-of-the-arts: On BSDS500, PiDiNet reaches 0.807 ODS at nearly 100 FPS, while PiDiNet-Tiny-L exceeds 200 FPS with comparable prediction performance; results remain comparable on NYUD and promising on Multicue.All reported PiDiNet models are trained from scratch without ImageNet pretraining.
6. Conclusion
The paper combines pixel difference convolution with the efficient PiDiNet architecture to target accurate and efficient edge detection. It evaluates the approach across multiple datasets and identifies related vision tasks as future applications.
- PiDiNet is built on pixel difference convolution and designed to balance edge-detection accuracy with memory efficiency and inference speed.
- The method integrates traditional edge-detector information with deep CNN representations to produce robust and accurate edge detection.
- PiDiNet can be trained from scratch using limited data while achieving human-level performance, without relying on a large-scale pretrained backbone.
- The experiments cover the BSDS500, NYUD, and Multicue datasets for edge detection.
- Future Work: Pixel difference convolution and PiDiNet may also be useful for semantic segmentation, object detection, and salient object detection through image-boundary information.
7. Appendix
The appendix explains how pixel difference convolution is represented and converted into vanilla convolution. It describes central, angular, and radial pair-selection strategies and reports qualitative and dataset-specific evaluations.
- PDC replaces local-patch pixels with selected pixel differences during convolution, using paired pixels and learned weights.
- Converting PDC to Vanilla Convolution: PDC can be converted to vanilla convolution during training and inference by transforming the kernel weights according to the selected pixel pairs.
- Pixel-Pair Selection: The appendix distinguishes central, angular, and radial PDC instances based on their pixel-pair selection strategies.
- Pixel-Pair Selection: APDC uses angular pixel pairs in a 3 × 3 patch, while RPDC is converted to a vanilla convolution with kernel size 5 × 5.
- Inference Conversion: During inference, the converted kernel weights are saved so that all convolutional operations are vanilla convolutions.
- Experiments: Qualitative results report that PiDiNet captures more useful boundaries than the baseline, while CDCM and CSAM positively affect edge detection.