AP-BSN: Self-Supervised Denoising for Real-World Images via Asymmetric PD and Blind-Spot Network

1. Introduction

• Synthetic AWGN training data underrepresents real-world noise, while collecting real noisy-clean pairs requires substantial labor and controlled capture conditions.The introduction also notes that variation across capture devices makes generalization from real-world training pairs difficult.Source: §1 · p. 1
• Large PD strides (> 3) better enforce pixel-wise independent noise but damage image detail, whereas small strides (≤3) preserve structures but fail that assumption.The paper identifies this trade-off as the reason PD is difficult to integrate directly with BSN for fully self-supervised real-world denoising.Source: §1 · p. 2
• AP-BSN combines blind-spot learning with different pixel-shuffle downsampling strides at training and inference to address spatially correlated real-world noise without prior noise knowledge.The authors describe it as the first self-supervised BSN for real-world sRGB inputs and report that it outperforms self-supervised and several unpaired methods.Source: §1 · p. 2
• R3 is a post-processing refinement that further improves AP-BSN without additional parameters or training.The contribution summary characterizes R3 as requiring no additional parameters.Source: §1 · p. 2

2. Related Work

• Earlier denoising work spans AWGN-trained CNNs, real-pair supervised methods, PD, and unpaired noise synthesis, but unpaired approaches still depend on matching clean and noisy scene statistics.CBDNet simulates camera ISP effects; GCBD and C2N synthesize realistic noise, while real-pair datasets such as SIDD and NIND [4] support supervised training.Source: §2 · p. 2
• Self-supervised methods include masking-based Noise2Void and Noise2Self [3], BSN variants [25] [38], Noise2Same [39], and subsampling-based Neighbor2Neighbor [17], but their independence assumptions fail for correlated sRGB noise.The paper says these methods can learn identity mappings on real-world sRGB images with spatially correlated noise.Source: §2 · p. 3
• Noisier2Noise [30], NAC, and R2R create auxiliary pairs by adding synthetic noise, but require noise-distribution or other prior information that may be unavailable in practice.The passage specifies that R2R requires noise level and ISP-function information, while Noisier2Noise requires knowledge of the underlying noise distribution.Source: §2 · p. 3

Blind-spot network

• Under pixel-wise independent, zero-mean noise, minimizing BSN’s self-supervised loss is equivalent to supervised denoising loss up to a constant.BSN predicts each output pixel without seeing the center pixel in its receptive field.Source: §3 · p. 3
• Real-world ISP processing, including Bayer-filter demosaicking, correlates neighboring noise signals, allowing BSN to exploit them and behave approximately as an identity mapping on sRGB images.The correlation between noise signals decreases with relative distance, with similar patterns reported across camera devices in SIDD.Source: §3 · p. 3, Figure 2 · p. 3

Pixel-shuffle downsampling

• PD, introduced by Zhou et al. [46], subsamples the image with stride s to break dependencies between correlated neighboring noise signals before conventional denoising.PD−1 reconstructs the full-sized output after denoising.Source: §3 · p. 3
• Zhou et al. [46] use stride factor 2 (PD2) to preserve image textures and details while achieving the best performance.The downsampling stride is a design choice intended to balance correlation breaking and texture preservation.Source: §3 · p. 3

4. Method

• PD decomposes each noisy input into s^2 tiled sub-images, BSN denoises them, and inverse PD reconstructs the output under an L1 loss chosen for better generalization [27].This pipeline is trained self-supervised on real-world sRGB images.Source: §4 · p. 3

4.1. Trade-offs in PD-BSN

• PD needs a stride of at least 5 to minimize noise dependence, but larger strides induce aliasing because downsampling lacks a low-pass filter.PD2 leaves spatially correlated noise, while larger-stride PD can introduce aliasing artifacts.Source: §4.1 · p. 4, Figure 4 · p. 4

4.2. Effective training stride factor for PD-BSN

• Training with PD stride s=5 reduces real-world noise correlation, and BSN learns to remove the resulting aliasing artifacts because they are effectively uncorrelated and approximately zero-mean.The high-frequency signal changes rapidly, and the aliasing artifacts’ mean is approximately that of the noisy input.Source: §4.2 · p. 4

4.3. Asymmetric PD for BSN

• AP5/2-BSN trains with stride 5 and infers with stride 2, balancing noise decorrelation against aliasing to remove real-world noise while preserving image structures.The method requires no clean training samples and applies directly to noisy sRGB images.Source: §4.3 · p. 5, Figure 5 · p. 5

4.4. Random-replacing refinement

• R3 replaces fixed stride-2 masks with randomized masks, reducing expected noise correlation by factor p; AP-BSN feeds replaced images directly to BSN without another PD step.Random placement makes the expected correlation between two noise signals equal to p times the original correlation.Source: Figure 6 · p. 5, §4.4 · p. 6

5.1. Experimental configurations

• Experiments use SIDD-Medium’s 320 noisy-clean pairs for training, with SIDD validation and benchmark patches for evaluation, and report PSNR and SSIM.The SIDD validation and benchmark sets each contain 1,280 noisy patches of size 256 × 256.Source: §5.1 · p. 6

5.4. AP-BSN for real-world denoising

• AP-BSN can train and evaluate on the same 50 noisy DND inputs, and training on the SIDD benchmark split improves denoising by about 1 dB without modifications.The fully self-supervised setup uses noisy samples directly and does not require clean images; the benchmark split has about 60 times fewer pixels than SIDD-Medium.Source: §5.1 · p. 6, §5.4 · p. 8

5.2. Analyzing Asymmetric PD

• Training stride a = 5 maximizes performance, while inference stride b = 2 best balances spatial noise correlation against aliasing; b = 1 cannot remove real-world noise.Larger b helps in plain regions without high-frequency content, but stronger aliasing can sharply reduce performance in general textured images.Source: §5.2 · pp. 6–7, Figure 8 · p. 6

5.4. AP-BSN for real-world denoising

• AP-BSN + R3 achieves 35.97 dB SIDD PSNR, 0.925 SIDD SSIM, 38.09 dB DND PSNR, and 0.937 DND SSIM, leading the compared unpaired and self-supervised methods.The paper reports that R3 improves SIDD benchmark-track evaluation PSNR by more than 1 dB without adding parameters.Source: §5.2 · p. 7, §5.4 · p. 8, Table 1

5.3. Analyzing Random-Replacing Refinement

• Random-replacing refinement performs best near replacement probability p ≈ 0.16, improves over PD-refinement even at T = 2, and gains performance as T increases.The method sets T = 8 to balance performance and runtime because its computational complexity is proportional to T.Source: §5.3 · pp. 7–8, Figure 9 · p. 7

• AP-BSN + R3 requires no prior knowledge of real-world noise and outperforms recent self-supervised and unsupervised denoising methods.Source: §6 · p. 8
• Asymmetric PD uses different training and inference strides to satisfy the pixel-wise independence assumption while preserving image details.The paper identifies trade-offs among PD stride factors in the context of BSN.Source: §6 · p. 8
• Random-replacing refinement R3 removes visual artifacts from AP-BSN without adding parameters.Source: §6 · p. 8

S1. Optimization

• AP-BSN trains on 120×120 noisy crops, with eight augmented samples per minibatch, for 20 epochs and a 10× learning-rate decay every eight epochs.Each sample receives a random 90° rotation and horizontal and vertical flips. Epochs use 24,542 patches for SIDD-Medium, 24,784 for DND, and 24,320 for the SIDD benchmark.Source: S1. Optimization · p. 9

S2. Network architecture

• AP-BSN replaces the multi-branch MDC modules with a sequence of single-branch dilated convolution modules, following Wu et al. [38].The paper identifies Fig. S1 as a detailed visualization of the BSN architecture used in AP-BSN.Source: S2. Network architecture · p. 9
• 3.7M parameters give AP-BSN a smaller network than the original BSN’s 6.6M parameters [38].The paper also reports that AP-BSN w/o R3 achieves comparable results with a much smaller denoising network while using only noisy images.Source: S2. Network architecture · p. 9

S3. Effects of aliasing artifacts

• The experiment trains AP-BSN using clean SIDD images, with BSN reconstructing the same image without access to the center pixel.The setup treats clean images as containing zero-intensity noise, assumed spatially uncorrelated and zero-mean.Source: S3. Effects of aliasing artifacts · p. 9
• PD5-BSN removes high-frequency information from clean inputs, whereas PD2-BSN does not; AP5/2 preserves high-frequency components and image structure.The comparison is shown in Figs. S2a–d.Source: S3. Effects of aliasing artifacts · p. 9
• Removing aliasing artifacts during inference can significantly degrade PD-BSN, making PD5-BSN infeasible as a denoising model.The authors infer that PD5-BSN learns during training to remove information absent from PD2 sub-images.Source: S3. Effects of aliasing artifacts · p. 9

S4. AP-BSN on the NIND dataset

• Because NIND noise correlations decrease as relative distance increases, the analysis tests whether AP-BSN may perform better with a = 6 or larger.The authors analyze APa/b trade-offs per sample to examine performance across diverse scenes.Source: S4. AP-BSN on the NIND dataset · p. 9
• On the NIND dataset, whose noise property differs from SIDD, AP6/2 may perform slightly better on some specific samples; performance gaps are marginal, while AP5/2 generalizes well on various real-world datasets on average.Source: S4. AP-BSN on the NIND dataset · p. 10

S5.1. Additional qualitative results

• Qualitative comparisons are limited because some methods lack results on specific datasets; examples include NAC on DND and R2R on SIDD, with additional comparisons on DND and SIDD validation.R2R does not provide DND results; the additional comparisons appear in Fig. S4.Source: S5.1. Additional qualitative results · p. 10

S5.2. Results on real-world inputs

• AP-BSN can be applied to a single noisy test image without pretrained knowledge; the real-world inputs were high-ISO Samsung Galaxy captures converted from RAW through a simulated camera pipeline without denoising.The study targets settings where noisy-real supervised pairs or clean images for unpaired learning may be unavailable.Source: S5.2. Results on real-world inputs · p. 10
• On real-world sRGB images, AP-BSN reconstructs sharper edges while suppressing noise versus in-camera denoising, and outperforms SIDD-trained DnCNN [44] using only a single noisy image for training.Results are visualized in Fig. S5.Source: S5.2. Results on real-world inputs · p. 10

S5.3. Qualitative improvement by R3

• R3 removes blocky artifacts and improves image-structure preservation in AP-BSN results on the SIDD validation dataset.Without R3, artifacts persist even at the smallest inference stride factor, b = 2.Source: S5.3. Qualitative improvement by R3 · p. 10, Figure S6 · p. 14
• R3 improves AP-BSN without requiring additional parameters or training.Source: S5.3. Qualitative improvement by R3 · p. 10