Source-linked AI summary
Deep Graph-Convolutional Image Denoising
Diego Valsesia, Giulia Fracastoro, Enrico Magli
TL;DR
Image denoising needs to exploit non-local self-similarity, which standard CNN convolutions do not capture directly. The paper proposes GCDN, a graph-convolutional denoiser with dynamically learned non-local connections, and reports state-of-the-art performance across synthetic and real-noise evaluations.
Problem
Standard CNN convolutions exploit local information, leaving limited integration of the non-local self-similarity that is effective for image denoising.
Method
GCDN dynamically constructs graphs from hidden-feature similarities and applies graph-convolutional layers with adaptive non-local receptive fields.
Results
The proposed method achieves state-of-the-art denoising performance across standard synthetic-noise evaluations, depth maps, and real images.
Takeaways & Limitations
Graph-convolutional denoising can exploit both local and non-local similarities through adaptive receptive fields and provide strong results on piecewise-smooth and real images.
Abstract
from arXiv · showhide
Non-local self-similarity is well-known to be an effective prior for the image denoising problem. However, little work has been done to incorporate it in convolutional neural networks, which surpass non-local model-based methods despite only exploiting local information. In this paper, we propose a novel end-to-end trainable neural network architecture employing layers based on graph convolution operations, thereby creating neurons with non-local receptive fields. The graph convolution operation generalizes the classic convolution to arbitrary graphs. In this work, the graph is dynamically computed from similarities among the hidden features of the network, so that the powerful representation learning capabilities of the network are exploited to uncover self-similar patterns. We introduce a lightweight Edge-Conditioned Convolution which addresses vanishing gradient and over-parameterization issues of this particular graph convolution. Extensive experiments show state-of-the-art performance with improved qualitative and quantitative results on both synthetic Gaussian noise and real noise.
I. INTRODUCTION
Image denoising remains important for downstream vision tasks, but standard CNN locality limits their ability to exploit non-local self-similarity. GCDN addresses this gap by dynamically constructing graph convolutions over learned feature similarities.
- Image denoising supports higher-level tasks including classification, segmentation, and object recognition.
- Classic methods such as BM3D exploit non-local self-similar patterns, whereas CNN convolution kernels are spatially local.
- Graph convolution extends traditional convolution to data represented on graph vertices, enabling non-local receptive fields.
- GCDN dynamically connects pixels using similarities among hidden-layer features to discover latent self-similarities.
- The proposed architecture improves on a preliminary version through proximal-gradient-inspired design, greater depth, solutions to ECC computational issues, and expanded experiments.
III. PROPOSED DENOISER
GCDN combines local convolutions with dynamically constructed graph-convolutional layers to give pixel neurons non-local receptive fields. Its ECC aggregates feature-similar pixels using edge-dependent transformations and attention, while the network uses multiscale preprocessing and residual blocks.
- Overview: GCDN uses graph-convolutional layers whose graphs are dynamically computed from hidden-feature similarities, creating non-local receptive fields.The network can merge spatially distant pixels that are similar in the feature space.
- Overview: The input preprocessing stage uses parallel 3 × 3, 5 × 5, and 7 × 7 convolutions to stabilize graph construction and extract multiscale embeddings.Embedding pixels over larger receptive fields reduces the influence of input noise during graph construction.
- Overview: The remaining network uses an HPF block and multiple LPF blocks with graph-convolutional layers, batch normalization, leaky ReLUs, and residual connections.The LPF residual connections are included to help backpropagation.
- Graph-convolutional layer: Each graph-convolutional layer combines a classic 3 × 3 local convolution with non-local aggregation over a K-nearest-neighbor graph in feature space.The local neighborhood is excluded from graph construction because it is already processed by the local convolution.
- Graph-convolutional layer: ECC predicts an edge-dependent matrix from feature-difference labels and combines the resulting non-local estimate with the local-convolution output.The edge-attention term downweights connections between nodes with distant feature vectors and is explicitly exponential in their feature distance.
- Graph-convolutional layer: The ECC is more expressive than scalar-edge-weight graph convolutions because it supports edge-dependent affine transformations and a general learned edge-weight function.Its edge weights depend on edge labels, allowing pairs with identical labels to share weights similarly to classical CNNs.
C. Lightweight Edge-Conditioned Convolution
The lightweight ECC addresses the original ECC function’s excessive parameterization by redesigning its final layer with structured approximations.
- C. Lightweight Edge-Conditioned Convolution: The original ECC function can become over-parameterized because its output dimension is F_l+1 × F_l, making its parameter count scale cubically with feature dimension.The resulting excessive parameter count can produce vanishing gradients or overfitting.
1) Circulant approximation of dense layer:
The dense layer is approximated with a structured partial-circulant matrix to reduce parameterization while retaining a compact representation.
- 1) Circulant approximation of dense layer: The F network’s dense layer is replaced with stacked partial circulant matrices whose free parameters are restricted to the first row of each matrix.Only a few shifted versions of each first row are used instead of all shifts in a full square matrix.
- 1) Circulant approximation of dense layer: The partial-circulant approximation reduces the number of free parameters by limiting each matrix to m allowed shifts.The cited discussion connects this structure to prior approximations of fully connected layers and stable embeddings.
2) Low-rank node aggregation:
The paper replaces full edge-conditioned weight matrices with a low-rank factorization, reducing memory, computation, and parameters while retaining adaptive node aggregation.
- Computational challenge: The full ECC operation can require storing a weight matrix for every edge, neighborhood, image, and batch element, creating substantial memory demands.For a K-regular graph and batch size B, the storage scales as B × N × K × F l+1 × F l × 4 bytes.
- Low-rank factorization: The method approximates each edge-conditioned matrix Θl,j→i by retaining its r largest singular-value components as outer products.The resulting structure has rank at most r and is implemented through separate outputs from parallel fully connected layers.
- Memory reduction: The low-rank design stores factor vectors instead of full matrices, reducing memory from 2 GB to 700 MB when r = 10 in the presented example.The reduced memory requirement is B × N × K × r(2F l + 1) × 4 bytes.
- Computational reduction: The decoupled aggregation lowers computational cost from O(F lF l+1) to O(r(F l + F l+1)).The approximation also reduces the final F-network output size from F l+1F l to r(F l + F l+1 + 1).
- Initialization: The network uses careful initialization because the variance of the aggregated features can depend on the fourth power of the number of features.The paper therefore adjusts the variance of the parallel-layer weights to balance this scaling term.
D. Analogy with unrolled graph smoothness optimization
The proposed network is motivated as an unrolled proximal-gradient method for graph-regularized reconstruction, with learned graph filters taking the roles of highpass and lowpass operations.
- Optimization analogy: The architecture generalizes a finite number of iterations of an unrolled proximal gradient descent method.This connects the network design to optimization for inverse problems with graph-based image priors.
- Graph smoothness prior: Graph smoothness regularization measures variation across edges connecting correlated pixels and favors images with low graph smoothness.The graph smoothness term xT Lx is described as the graph analogue of total variation.
- Optimization analogy: The optimization alternates a gradient-descent step over one term with a proximal mapping over the regularizer.The resulting update is reformulated in residual-network coordinates before motivating the network architecture.
- Filter interpretation: The graph Laplacian acts as a highpass filter, whereas (I + βL)^-1 acts as a lowpass filter.The lowpass behavior follows because its spectral coefficients decrease as graph-Laplacian eigenvalues increase.
- Network construction: The network unrolls a finite number of these iterations and replaces the analytical filters with learned graph filters interleaved with nonlinearities.Experiments report that the learned filters exhibit approximate highpass and lowpass behavior.
A. Training details
Training uses randomly extracted image patches from BSD training data, with fixed feature, approximation, and low-rank settings; the lightweight ECC is important for optimization stability.
- Dataset and objective: The network is trained on 42 × 42 patches randomly extracted from 400 BSD images, while 68 validation images form the BSD68 test set.Training minimizes mean squared error between denoised outputs and clean targets.
- Optimization stability: Without the lightweight ECC, the network suffered from vanishing gradients even with substantially fewer layers.The observation motivated using the proposed ECC design during training.
B. Feature analysis
This section examines the properties of features learned in the network's hidden layers.
- Feature analysis: The feature analysis focuses on properties of representations in the network's hidden layers.
1) Adaptive receptive field:
The graph-convolutional layers give each pixel an adaptive receptive field determined by feature-space graph structure rather than a fixed spatial pattern. Experiments examine neighborhood size, aggregation design, feature behavior, and the resulting graph predictions.
- Adaptive receptive field: Graph-convolutional receptive fields vary with graph structure, allowing pixels to aggregate information beyond fixed local neighborhoods.The receptive field is analyzed for pixels on edges and in uniform regions across LPF blocks.
- Design parameters: Increasing the number of graph neighbors improves Set12 PSNR until saturation, after which performance slightly decreases.Each neighbor-count configuration is independently trained for the specified noise level.
- Adaptive receptive field: Feature-space distances determine which pixels are connected to a central pixel within the non-local search window.The analysis compares horizontal-, vertical-, and diagonal-edge cases using Euclidean feature distances.
- Design parameters: Using trainable edge transformations with edge attention outperforms edge attention alone by approximately 0.2 dB with 8 nearest neighbors.The comparison supports using the affine transformation Θ rather than only a scalar edge-weight function.
- Design parameters: The full ECC is excluded from comparison because vanishing gradients make training unstable and its computation is prohibitive.These constraints motivate the approximations used by the proposed lightweight graph convolution.
D. Comparison with state of the art
GCDN achieves state-of-the-art synthetic denoising results across standard natural-image and depth-map benchmarks. Its advantages are especially apparent for low-to-medium Gaussian noise, structured images, and depth maps under stronger noise.
- Natural image denoising: GCDN achieves state-of-the-art performance on Set12, BSD68, and Urban100 using PSNR and SSIM evaluations.The evaluation compares GCDN with model-based and deep-learning denoisers, including non-local methods.
- Natural image denoising: GCDN works especially well at low-to-medium noise levels, while constructing a meaningful graph becomes harder at higher noise.Urban100 results are particularly strong because its regular structures offer useful self-similarity.
- Depth-map denoising: GCDN outperforms NLRN and OGLR on depth-map denoising, including at high noise levels.The reported visual comparison shows sharper edges and a smoother background for GCDN than OGLR under stronger noise.
E. Real image denoising
Real image noise is signal-dependent and more challenging than synthetic Gaussian noise, motivating evaluation after real-data training. The proposed method achieves better real-image results and a significant performance gain in this setting.
- Real image denoising: Real image noise includes quantization, shot, fixed-pattern, and dark-current contributions, making it signal-dependent.Methods trained only on synthetic Gaussian noise have been observed to perform poorly on real noise, whereas retraining with real data generally helps.
- Real image denoising: The proposed method achieves better results on the real-image denoising evaluation.The improvement is confirmed by the corresponding visual comparison.
- Real image denoising: The paper concludes that GCDN can provide a significant performance gain for real image denoising.This conclusion extends the reported evaluation beyond synthetic Gaussian noise.