PaperlayerView exact revision on arXiv

Source-linked AI summary

DiffusionEdge: Diffusion Probabilistic Model for Crisp Edge Detection

Yunfan Ye, Kai Xu, Yuhang Huang, Renjiao Yi, Zhiping Cai

arXiv:2401.02032v2cs.CV

TL;DR

Edge detectors have had difficulty producing maps that are both correct and crisp without post-processing. DiffusionEdge applies diffusion to edge detection with latent-space training, adaptive Fourier filtering, and uncertainty distillation. It reports superior results across four benchmarks, including substantial NYUDv2 gains over the second best.

  • Problem

    Learning-based edge detectors have difficulty producing edge maps that are both correct and crisp without post-processing.

    Source: Introduction · p. 1

  • Method

    DiffusionEdge uses a diffusion model for edge detection, with latent-space training, a decoupled architecture, adaptive Fourier filtering, and pixel-to-latent uncertainty distillation.

    Source: Introduction · p. 2

  • Results

    On NYUDv2, DiffusionEdge increases ODS, OIS (without post-processing), and AC over the second best by 30.2%, 28.1%, and 65.1%, respectively, and is superior on four benchmarks in correctness and crispness.

    Source: Introduction · p. 2

  • Takeaways & Limitations

    DiffusionEdge directly generates crisp and accurate edge maps without post-processing, with reported potential to benefit subsequent tasks in an end-to-end manner.

    Source: Conclusions and Limitations · p. 7

    Inference efficiency remains an open problem, and faster diffusion-model inference is identified as a future direction.

    Source: Conclusions and Limitations · p. 7

Abstract

from arXiv · show

Limited by the encoder-decoder architecture, learning-based edge detectors usually have difficulty predicting edge maps that satisfy both correctness and crispness. With the recent success of the diffusion probabilistic model (DPM), we found it is especially suitable for accurate and crisp edge detection since the denoising process is directly applied to the original image size. Therefore, we propose the first diffusion model for the task of general edge detection, which we call DiffusionEdge. To avoid expensive computational resources while retaining the final performance, we apply DPM in the latent space and enable the classic cross-entropy loss which is uncertainty-aware in pixel level to directly optimize the parameters in latent space in a distillation manner. We also adopt a decoupled architecture to speed up the denoising process and propose a corresponding adaptive Fourier filter to adjust the latent features of specific frequencies. With all the technical designs, DiffusionEdge can be stably trained with limited resources, predicting crisp and accurate edge maps with much fewer augmentation strategies. Extensive experiments on four edge detection benchmarks demonstrate the superiority of DiffusionEdge both in correctness and crispness. On the NYUDv2 dataset, compared to the second best, we increase the ODS, OIS (without post-processing) and AC by 30.2%, 28.1% and 65.1%, respectively. Code: https://github.com/GuHuangAI/DiffusionEdge.

Introduction

Edge detectors have struggled to produce maps that are both correct and crisp without post-processing. DiffusionEdge uses diffusion-based denoising with latent-space and architectural designs, and reports superior performance on four benchmarks.

Introduction

  • Edge detection identifies object boundaries and visually salient edges, supporting downstream 2D perception, generation, and 3D curve reconstruction tasks.Examples include 2D perception (Zitnick and Doll´ar 2014; Revaud et al. 2015; Cheng et al. 2020), generation (Nazeri et al. 2019), and 3D curve reconstruction.Source: Introduction · p. 1
  • Prior methods have not directly produced edge maps that jointly satisfy correctness and crispness without morphological non-maximal suppression.Traditional gradient-based methods can be crisp but insufficiently correct, while deep learning methods improve correctness but tend to lack crispness.Source: Introduction · p. 1
  • Diffusion models denoise at original image size, unlike CNN edge detectors whose downsampling and upsampling can yield thick edges.Their multi-step denoising also predicts a denoised variant of noisy input, making it easier to learn the target distribution.Source: Introduction · pp. 1–2
  • DiffusionEdge combines latent-space diffusion, a decoupled architecture, an adaptive Fourier filter, and uncertainty distillation from pixel-level gradients.The designs reduce computation, speed inference, adaptively adjust frequency components, and retain pixel-level uncertainty information from multiple annotators.Source: Introduction · p. 2
  • Across four benchmarks, DiffusionEdge achieves superior correctness and crispness; on NYUDv2 it increases ODS, OIS (without post-processing), and AC over the second best by 30.2%, 28.1%, and 65.1%, respectively.The method directly generates edge maps without post-processing and uses fewer augmentation strategies.Source: Introduction · p. 2

Related Work

Earlier edge detectors improve correctness through global or multi-layer features, while the paper frames simultaneous crispness and correctness without post-processing as an unmet need. DiffusionEdge addresses that gap with adaptive Fourier filtering and latent-space uncertainty distillation.

Edge detection

  • CNN methods integrate multi-layer features, while PiDiNet targets efficiency and EDTER combines global context with local cues in a two-stage vision-transformer approach.HED proposed an end-to-end architecture, RCF integrated hierarchical features, and BDCN used layer-specific supervision in a bi-directional cascade.Source: Related Work · p. 2
  • Learning-based detectors improve correctness but often produce thick edges and rely on post-processing; the paper identifies direct correctness and crispness without post-processing as an unmet need.Prior crispness efforts include loss functions and label refinement.Source: Related Work · p. 2

Diffusion probabilistic model

  • DiffusionEdge distinguishes itself with learnable Fourier convolution for adaptive latent-frequency filtering and gradient distillation that preserves pixel-level uncertainty.The paper presents it as the first diffusion-model usage for generic edge detection and reports superiority in correctness and crispness.Source: Related Work · p. 2

Method

DiffusionEdge trains a decoupled diffusion model in latent space, using an adaptive Fourier filter and uncertainty-aware loss distillation. Its design aims to preserve edge-map accuracy and crispness while reducing computational cost.

Method

  • DiffusionEdge trains denoising in latent space after compressing edge labels with an autoencoder, whose weights are fixed during denoising training.The latent representation is spatially downsampled by 4×, and the input image is an extra condition.Source: Method · p. 2, Preliminaries · p. 3
  • The decoupled denoising network predicts both the denoised edge data and noise, with the final objective combining their losses and a time-weighted WCE term.A decoupled diffusion process is adopted to speed sampling; the WCE term permits uncertainty-aware supervision in latent space while reducing computational cost.Source: Preliminaries · pp. 2–3, Uncertainty Distillation · p. 4
  • An adaptive FFT filter learns frequency-specific weights to adjust latent features, while a residual connection preserves potentially useful information.The filter transforms encoder features into the frequency domain, applies learnable weights, and returns them to the spatial domain.Source: Adaptive FFT-filter · p. 3
  • Pixel-level uncertainty cannot be applied directly to latent codes because their numerical range and downsampled spatial layout do not align with image-space labels.The uncertainty-aware WCE loss ignores ambiguous ground-truth pixels, but direct latent supervision risks incorrect uncertainty.Source: Uncertainty Distillation · p. 4
  • Uncertainty distillation applies WCE supervision to latent-space gradients while skipping the gradient through the autoencoder.Backpropagating through the autoencoder in Baseline-A used 1.5× more GPU memory and performed worse; the authors identify this as motivating the gradient-skipping design.Source: Uncertainty Distillation · p. 4

Experiments

Experiments across four edge-detection datasets evaluate correctness and crispness using standard and crispness-focused protocols, with ablations examining key components, backbones, and sampling steps. DiffusionEdge performs strongly across the benchmarks, including substantial NYUDv2 gains without post-processing and crisp predictions with limited augmentation.

Datasets

  • Experiments evaluate edge detection on BSDS, NYUDv2, Multicue, and BIPED; BSDS images have 4 to 9 annotators whose labels are averaged, while each Multicue image is annotated by several people.BSDS has 200 training, 100 validation, and 200 test images. BIPED has 200 training and 50 test images.Source: Datasets · p. 4

Evaluation protocols

  • SEval applies NMS and morphology post-processing, while CEval omits post-processing; AC is the post-NMS to pre-NMS pixel-value ratio, with larger values indicating crisper maps.ODS uses a dataset-wide threshold and OIS an image-specific optimal threshold; maximum allowed pixel distances are 0.011 for NYUD and 0.0075 for other datasets.Source: Evaluation Metrics · p. 5

Ablation Study: The effect of key components

  • Both Adaptive FFT-filter and Uncertainty Distillation improve performance, with UD more critical and their combination achieving the best performance.The ablation is on BSDS with single-scale input.Source: Ablation Study · p. 5, Table 2 · p. 5

Ablation Study: The effect of backbones and diffusion steps

  • Sampling steps from 1 to 50 change ODS and OIS by less than 0.4%, and one step already achieves state-of-the-art performance; step 5 is the standard setting.More steps increase inference time, and Swin-B performs better than ResNet101 and EfficientNet-B7.Source: Ablation Study · p. 5, Table 3 · p. 5

On BSDS

  • DiffusionEdge achieves the best BSDS results in all settings, especially AC, with less noise and more semantically meaningful contours in challenging scenes.Comparisons use single-scale results from models trained only on BSDS; other methods use 96× augmented data, while DiffusionEdge uses random flipping and scaling.Source: Comparison with State-of-the-arts · p. 6, Table 4 · p. 6, Figure 4 · p. 6

On NYUDv2

On Multicue and BIPED

On Crispness

  • DiffusionEdge achieves the best crispness in all comparisons against crisp-edge strategies built on PiDiNet.The comparisons use Dice loss, tracing loss, and Guided Label Refinement; the reported crispness measure is AC.Source: Comparison with State-of-the-arts · p. 7, Table 7 · p. 7

Conclusions and Limitations

DiffusionEdge is presented as the first diffusion-based network for crisp edge detection, producing accurate and crisp maps without post-processing. The authors report that efficiency remains an open problem.

Conclusions

  • The first diffusion-based network for crisp edge detection generates accurate and crisp edge maps without post-processing.The authors report quantitative and qualitative superiority and potential for benefiting subsequent tasks end-to-end.Source: Conclusions and Limitations · p. 7
  • The reported crispness is described as having potential to benefit subsequent tasks in an end-to-end manner.Source: Conclusions and Limitations · p. 7

Limitations

  • Correctness and crispness can be simultaneously qualified for downstream tasks, while efficiency remains an open problem.Faster diffusion-model inference is identified as a promising future direction.Source: Conclusions and Limitations · p. 7

Related papers

Loading 2401.02032v2…