Robust and interpretable blind image denoising via bias-free convolutional neural networks

Sreyas Mohan; Zahra Kadkhodaie; Eero P. Simoncelli; Carlos Fernandez-Granda

Source-linked AI summary

Robust and interpretable blind image denoising via bias-free convolutional neural networks

Sreyas Mohan, Zahra Kadkhodaie, Eero P. Simoncelli, Carlos Fernandez-Granda

arXiv:1906.05478v3Jun 13, 2019eess.IVcs.CVcs.LGstat.ML

TL;DR

Blind denoising networks often overfit the noise levels seen during training, raising questions about cross-level generalization and their learned mechanisms. The paper removes additive constants from CNN architectures and analyzes the resulting local maps with linear-algebraic tools. Bias-free networks generalize robustly across noise levels while retaining state-of-the-art performance, and the analysis connects them to adaptive filtering and low-dimensional projection methods.

Problem
CNN denoising achieves state-of-the-art results, but its learned mechanisms are poorly understood and conventional networks generalize poorly beyond trained noise levels.
Method
The paper removes all additive constants from CNN architectures and analyzes bias-free denoising maps locally through their Jacobians.
Results
Bias-free networks generalize strongly across noise levels while preserving state-of-the-art denoising performance within limited training ranges.
Takeaways & Limitations
Bias removal supports robust cross-noise denoising and links the learned method to nonlinear adaptive filtering and projection onto low-dimensional subspaces.
Takeaways & Limitations
The linear-algebraic interpretations are local because small input changes can alter ReLU activation patterns and the corresponding linear mapping.

Abstract

from arXiv · show

Deep convolutional networks often append additive constant ("bias") terms to their convolution operations, enabling a richer repertoire of functional mappings. Biases are also used to facilitate training, by subtracting mean response over batches of training images (a component of "batch normalization"). Recent state-of-the-art blind denoising methods (e.g., DnCNN) seem to require these terms for their success. Here, however, we show that these networks systematically overfit the noise levels for which they are trained: when deployed at noise levels outside the training range, performance degrades dramatically. In contrast, a bias-free architecture -- obtained by removing the constant terms in every layer of the network, including those used for batch normalization-- generalizes robustly across noise levels, while preserving state-of-the-art performance within the training range. Locally, the bias-free network acts linearly on the noisy image, enabling direct analysis of network behavior via standard linear-algebraic tools. These analyses provide interpretations of network functionality in terms of nonlinear adaptive filtering, and projection onto a union of low-dimensional subspaces, connecting the learning-based method to more traditional denoising methodology.

1 INTRODUCTION AND CONTRIBUTIONS

The paper examines why denoising networks generalize poorly across noise levels and proposes bias-free architectures that generalize robustly while enabling interpretable analysis.

CNN denoising mechanisms remain poorly understood, including which learned priors and architectural choices shape performance.
A single network can denoise across many noise levels, but conventional training simulates the full noise range.
Removing all additive constants lets networks generalize across noise levels, including when trained over a very limited range.
Bias-free networks preserve state-of-the-art denoising performance while avoiding systematic overfitting to training noise levels.
Local Jacobian analysis interprets denoising as adaptive filtering and projection onto adaptively selected low-dimensional subspaces.

2 RELATED WORK

The paper places CNN denoising alongside adaptive classical filters and sparse, subspace-based methods, while contrasting their reported generalization behavior through an example.

Wiener filtering assumes a translation-invariant Gaussian signal model but can oversmooth fine details and textures.
Modern filtering methods adapt filters to local image structure to address the detail loss of Wiener filtering.
Wavelet denoising uses sparse representations and nonlinear thresholding to discard components small relative to the noise level.
For networks trained on σ ∈[0, 10], BF-CNN performs at state-of-the-art levels at σ = 90 while the CNN performs poorly.
CNNs achieve state-of-the-art denoising by learning cascades of convolutional filters and rectifying nonlinearities from noisy image patches.

3 NETWORK BIAS IMPAIRS GENERALIZATION

The paper models blind denoising under additive Gaussian noise and shows that input-dependent network bias grows outside the training range, impairing generalization.

Blind denoising evaluates CNN generalization across unknown Gaussian noise levels, including levels absent from training.
For a fixed ReLU activation pattern, a denoising network is affine, combining linear transformations, additive constants, and binary masks.
The local denoising map is f(y) = A_y y + b_y, where both the Jacobian and net bias depend on the input's activation pattern.
The residual decomposes as y−f(y) = (I −A_y)y−b_y, separating a linear component from the net bias.
Outside the training range, the bias norm increases dramatically, the residual becomes smaller than the noise, and performance generalizes very poorly.
Across training ranges, CNN performance degrades at new noise levels, whereas corresponding BF-CNNs maintain strong denoising performance.

4 PROPOSED METHODOLOGY: BIAS-FREE NETWORKS

Removing additive bias terms produces bias-free CNNs with scaling-invariant behavior and enables both robust noise-level generalization and local linear-algebraic analysis.

Bias-free architecture: Removing additive terms from every network stage, including batch normalization, yields a bias-free CNN that can generalize beyond its training noise levels.The modification is associated with improved generalization and enables linear-algebraic analysis of denoising behavior.
Scaling invariance: Bias-free ReLU networks are positively homogeneous: scaling the input by α ≥ 0 scales the output by the same α.This follows because ReLU commutes with nonnegative scaling, and the property also holds through linear skip connections.
Scaling invariance: Scaling invariance is desirable for image denoising because rescaling an image preserves its status as an image.Networks with nonzero net bias lack this invariance because input scaling can change ReLU activation patterns.
Local interpretation: Locally, bias-free CNNs can be interpreted using adaptive linear weighting functions whose shapes depend on image content and noise level.The weighting functions sum to one, may contain negative weights, and expand spatially as noise increases while respecting region boundaries.

5 BIAS-FREE NETWORKS GENERALIZE ACROSS NOISE LEVELS

The study compares state-of-the-art CNN architectures with otherwise identical bias-free versions across noise levels inside and outside training ranges. Bias-free models retain strong denoising performance beyond those ranges, while ordinary CNN performance degrades dramatically.

Compared architectures: The comparison covers DnCNN, recurrent CNN, UNet, and simplified DenseNet architectures alongside matching bias-free counterparts.The paired models differ only by removing additive constants, including the batch-normalization additive parameter.
Evaluation setup: Networks are trained on i.i.d. Gaussian noise over specified standard-deviation ranges and evaluated both within and beyond those ranges.Experiments use 180 × 180 natural images from the Berkeley Segmentation Dataset.
Linear-algebraic analysis: BF-CNN Jacobians show many near-zero singular values, indicating approximate projection onto a low-dimensional signal subspace.The effective dimensionality decreases with noise level, while lower noise levels support richer feature subspaces.
Generalization results: Ordinary CNNs perform well over their training ranges but degrade dramatically at new noise levels across architectures and training ranges.The pattern is reported for both PSNR and Structural Similarity Index.
Generalization results: Bias-free CNNs provide strong denoising performance outside the training range across the tested architectures.The reported contrast supplies evidence that removing net bias improves generalization to unseen noise levels.

6 REVEALING THE DENOISING MECHANISMS LEARNED BY BF-CNNS

Bias-free CNNs admit a locally linear analysis that reveals adaptive filtering and projection onto low-dimensional signal subspaces. These subspaces preserve clean-image structure while discarding noise, and their dimensionality and nesting vary systematically with noise level.

Local linear analysis: A bias-free network is strictly linear, with its Jacobian providing a local characterization of the denoising map.This enables linear-algebraic analysis at a fixed noisy input.
Nonlinear adaptive filtering: Each output pixel is computed as an inner product between a Jacobian row and the noisy input, yielding an adaptive filter.The filters average homogeneous regions while avoiding blurring across edges.
Nonlinear adaptive filtering: BF-CNN filters enlarge their averaging regions as noise increases, while remaining adaptive to image structure.This parallels Wiener filtering but learns spatially varying filters directly from data.
Projection onto adaptive low-dimensional subspaces: Most singular values are nearly zero, so the network discards nearly all input directions except a low-dimensional portion whose vectors capture image features.Near-zero singular-value directions are noisy and unstructured, whereas the retained directions reflect the input image.
Projection onto adaptive low-dimensional subspaces: The BF-CNN approximately projects noisy images onto an adaptive signal subspace that preserves image structure while suppressing noise.The interpretation follows from the approximate symmetry of the Jacobian and the alignment of its significant left and right singular vectors.
Projection onto adaptive low-dimensional subspaces: The clean image lies almost entirely in the preserved signal subspace, whose effective dimensionality decreases as noise increases and whose higher-noise subspaces are nested within the σ = 10 subspace.The nesting includes more than 95% of higher-noise subspace axes, while the clean-image containment holds across the trained noise range.
Projection onto adaptive low-dimensional subspaces: The dimensionality trend helps explain denoising across noise levels: with d approximately proportional to 1/σ, mean squared error scales with σ rather than noise variance.The resulting PSNR is predicted to be linear in input PSNR with slope 1/2, consistent with the reported empirical behavior.

7 DISCUSSION

Removing constant terms from CNNs yields strong cross-noise-level generalization and enables local linear-algebraic interpretation, while important scope boundaries remain.

Bias removal preserves in-range performance while enabling near-perfect generalization to noise levels more than 10x beyond training.
Networks with bias develop dramatically larger net bias outside the training range, where performance becomes poor.
Bias-free networks generalize beyond Gaussian noise and across noise levels for fixed blur, but bias removal is insufficient for generalization across blur levels.
The linear-algebraic interpretation is local because small input changes can alter activation patterns and the corresponding linear mapping.

A DESCRIPTION OF DENOISING ARCHITECTURES

This section introduces the denoising architectures used in the computational experiments.

The computational experiments use several denoising architectures described in greater detail in the following subsections.

A.1 DNCNN

BF-DnCNN adapts the DnCNN architecture by removing every additive bias source, including batch-normalization means, while retaining scaling.

DnCNN uses 20 convolutional layers with 3 × 3 filters, 64 channels, batch normalization, ReLU nonlinearities, and a skip connection.
BF-DnCNN removes all additive-bias sources, including batch-normalization mean parameters, but preserves the scaling parameter.

A.2 RECURRENT CNN

The recurrent CNN repeatedly denoises an image using a five-layer convolutional module, while the related UNet and DenseNet variants specify distinct nonrecurrent architectures and bias placements.

RECURRENT CNN: The recurrent CNN computes denoised estimates from the previous estimate and noisy image, initializing the process with the noisy image.
RECURRENT CNN: Training samples recurrence lengths uniformly from 1–4, while testing fixes the recurrence length at 4.
UNET: The UNet uses nine convolutional stages with downsampling, dilation, transpose convolution, skip concatenation, and a one-channel output.
UNET: The UNet with bias adds trainable constants to every layer except conv9 and assumes even image width and height.
SIMPLIFIED DENSENET: The simplified DenseNet has four five-layer convolutional blocks, concatenating each block output with the noisy input before the next block.
SIMPLIFIED DENSENET: In the biased DenseNet, trainable additive parameters are added to all layers except the final layer of the last block.

B DATASETS AND TRAINING PROCEDURE

The experiments use Berkeley natural-image data with augmentation and distinct patch-based training setups for DnCNN versus the remaining architectures. Training uses Adam with architecture-specific epoch, learning-rate, validation, and stopping procedures.

The training set is augmented through downsampling, random flips, and random rotations of image patches.
DnCNN and its bias-free counterpart use 50 × 50 patches, yielding 541,600 clean training patches.
The remaining architectures use 128 × 128 patches, yielding 22,400 training patches.
DnCNN models are trained with Adam for 70 epochs, while the other models use Adam for 50 epochs with validation-based learning-rate adjustment.
The non-DnCNN models use early stopping, whereas DnCNN training uses no early stopping.

C ADDITIONAL RESULTS

Additional experiments examine bias contributions, generalization across noise levels and distributions, adaptive linear filters, and Jacobian structure. Across the reported comparisons, bias-free networks generalize robustly beyond training noise ranges, while their local behavior supports adaptive-filtering and subspace-projection interpretations.

Noise-distribution generalization: Networks trained on zero-mean Gaussian noise generalize to zero-mean uniform noise during testing.
Image restoration with blur: BF-CNNs generalize beyond the training noise range at a fixed blur level but do not outperform CNNs when tested on unseen blur levels.
Residual and bias analysis: When trained over σ ∈[0, 55] or σ ∈[0, 30], net bias grows explosively outside the training range, coinciding with a dramatic performance drop.
Noise-level generalization: For DnCNN and other architectures, bias-free versions achieve excellent generalization across noise levels, unlike biased versions evaluated outside training ranges.
Adaptive filtering: Equivalent filters in bias-free architectures sum to one and compute local averages with shapes adapted to the underlying image.
Jacobian analysis: Jacobian singular vectors capture clean-image features, while many singular values near zero indicate approximate projection onto a signal subspace.