PaperlayerView exact revision on arXiv

Source-linked AI summary

sRGB Real Noise Modeling via Noise-Aware Sampling with Normalizing Flows

Dongjin Kim, Donggoo Jung, Sungyong Baik, Tae Hyun Kim

arXiv:2608.29038v1cs.CV

TL;DR

Real-world image denoising is hindered by simple noise-distribution assumptions and the cost of collecting paired real-noise data. NAFlow uses a conditional normalizing flow with camera-dependent latent distributions, GMM-based Noise-Aware Sampling, and multi-scale noise modeling, achieving strong noise quality and state-of-the-art SIDD denoising performance without input metadata during inference.

  • Problem

    Conventional denoising assumes simple noise distributions, while collecting large-scale paired real-noise datasets requires substantial time and resources.

  • Method

    NAFlow uses a conditional normalizing flow to learn camera-dependent Gaussian latent distributions, while NAS uses GMM-based sampling without noisy-image metadata.

  • Results

    NAFlow achieves state-of-the-art denoising performance on SIDD and outperforms NeCA-W by 0.39 dB in PSNR.

  • Takeaways & Limitations

    The framework generates realistic spatially correlated sRGB noise and supports denoising with synthetic noise without metadata or separate models for each camera configuration.

  • Takeaways & Limitations

    Higher multi-scale embedding levels, such as L ≥4, could not be investigated because training became unstable; training also assumes ground-truth clean images as conditioning.

Abstract

from arXiv · show

Noise poses a widespread challenge in signal processing, particularly when it comes to denoising images. Although convolutional neural networks (CNNs) have exhibited remarkable success in this field, they are predicated upon the belief that noise follows established distributions, which restricts their practicality when dealing with real-world noise. To overcome this limitation, several efforts have been taken to collect noisy image datasets from the real world. Generative methods, employing techniques such as generative adversarial networks (GANs) and normalizing flows (NFs), have emerged as a solution for generating realistic noisy images. Recent works model noise using camera metadata, however requiring metadata even for sampling phase. In contrast, in this work, we aim to estimate the underlying camera settings, enabling us to improve noise modeling and generate diverse noise distributions. To this end, we introduce a new NF framework that allows us to both classify noise based on camera settings and generate various noisy images. Through experimental results, our model demonstrates exceptional noise quality and leads in denoising performance on benchmark datasets.

1 INTRODUCTION

Real-world noise is difficult to model because conventional denoising assumes simple known distributions, while collecting paired real-noise data is costly. NAFlow addresses this with unified, metadata-free noise modeling and reports strong noise quality and denoising performance.

  • CNN denoising methods often assume Gaussian or Poisson noise, limiting generalization to real-world noise.
  • Real-noise datasets provide paired clean and corrupted images but require substantial time and resources to collect.SIDD includes smartphone images captured under varied ISO, shutter speed, and aperture settings.
  • Generative methods use GANs or normalizing flows to learn real-noise distributions and synthesize realistic noisy images.
  • NAFlow learns diverse camera-setting-dependent noise distributions with a single unified model and reduces metadata requirements during sampling.It also uses multi-scale noise embedding to model spatially correlated sRGB noise.
  • NAFlow produces outstanding noise quality and reaches state-of-the-art denoising performance on the SIDD benchmark.

2 RELATED WORK

Prior work generates realistic noisy images using paired or unpaired data, adversarial learning, camera pipelines, and other generative frameworks. This paper instead combines a unified NF with GMM-based metadata-free sampling that models camera-dependent noise distributions and spatial correlation.

  • DANet, C2N, and CycleISP generate noisy images using paired distributions, adversarial learning, or camera image pipelines.
  • The proposed approach uses GMMs to learn distinct real-world noise distributions associated with ISO levels and camera models.
  • Unlike methods requiring target-noise metadata during inference, the approach samples realistic correlated noise without metadata using one unified model.

3 BACKGROUND

The paper models image noise with conditional normalizing flows, which transform between image and latent spaces while optimizing likelihood through the change-of-variables formula. Its framework also represents real sRGB noise as camera-dependent distributions and accounts for the effects of image-processing operations.

  • Image noise is represented as the sum of a clean image and added noise.The clean image is x, the noisy image is y, and the noise is n.
  • Figure 1 maps noisy images to camera-configuration-specific Gaussian latent distributions and uses a GMM during inference to obtain metadata-free representations.
  • Real-world noise arises from camera sensors and post-imaging operations that transform RAW-RGB data into sRGB.These operations include demosaicking, gamma correction, and tone mapping.
  • Normalizing flows use invertible transformations to map a simple Gaussian distribution to a complex target distribution and optimize NLL directly.
  • The change-of-variables formulation computes latent density using the data density and the Jacobian determinant of the transformation.The determinant accounts for volume changes, while model parameters are trained by minimizing NLL.

4 PROPOSED METHOD

NAFlow models diverse real-world noise distributions with a single conditional normalizing flow and generates noisy images without requiring target metadata during sampling. Its design combines camera-configuration-specific latent distributions, noise-aware sampling, and multi-scale embedding to address spatially correlated sRGB noise.

  • 4 PROPOSED METHOD: NAFlow learns multiple distinctive noise distributions from various camera settings using a single unified normalizing-flow framework.The learned distributions are associated with camera configurations and support both noise modeling and generation.
  • 4.1 LEARNING MULTIPLE DISTINCT NOISE DISTRIBUTIONS: Camera configuration-specific Gaussian distributions define latent noise models for smartphone model and ISO combinations.The conditional flow maps noisy images to latent variables conditioned on clean images, while each configuration has trainable mean and covariance parameters.
  • 4.1 LEARNING MULTIPLE DISTINCT NOISE DISTRIBUTIONS: The conditional flow is trained with negative log-likelihood to handle multiple distributions within one model.The number of camera configurations is represented by C, and noisy images can be synthesized by inverse mapping random latent samples conditioned on a clean image.
  • 4.2 NOISE-AWARE IMAGE GENERATION: Noise-Aware Sampling classifies an input noise distribution in latent space before generating similar noisy images without camera metadata.The method evaluates the latent representation of the noisy input under each learned Gaussian distribution and samples from the resulting noise model.
  • 4.3 MULTI-SCALE NOISE EMBEDDING: NAFlow uses multi-scale noise embedding to model spatially correlated noise whose correlation areas range from one pixel to over 25 pixels.Squeeze operations address different correlation scales, while the architecture uses three scale-level blocks and an added transition block to reduce checkerboard artifacts.

5 EXPERIMENTS

Experiments evaluate NAFlow’s noise quality, generalization, spatial correlation, denoising utility, and sampling and architecture choices across SIDD-based settings. NAFlow achieves strong noise-modeling and denoising results while using a unified model and avoiding metadata during inference.

  • Noise Generation with SIDD: NAFlow consistently achieves the best average KLD and AKLD on SIDD-Validation, improving over NeCA-W by 0.0035 KLD and 0.013 AKLD.Noise quality is evaluated across five smartphone cameras.
  • Noise Generation with SIDD: NAFlow uses one unified model for all camera models, whereas NeCA-W uses five separately trained models and three sub-models per camera.Despite the smaller parameter count, NAFlow shows comparable performance to NeCA-W and produces noise strongly resembling real-world noise.
  • Noise Generation with SIDD+: NAFlow generates noise for SIDD+ camera configurations without metadata during inference, addressing the challenge of configurations absent from the SIDD training set.SIDD+ contains smartphones not used to capture SIDD training images.
  • Noise Correlation: NAFlow produces spatially correlated noise whose correlation map most closely resembles the real map compared with C2N and NeCA-W.The comparison evaluates correlation between center-pixel noise and neighboring-pixel noise.
  • Application: sRGB Denoising: 0.39 dB is NAFlow’s PSNR advantage over NeCA-W when DnCNN is trained on synthetic noisy images generated by each method.NAFlow achieves equivalent SSIM performance to the upper-bound comparison, although its PSNR remains lower.
  • Ablation Study: NAFlow outperforms naive Rank-k and random latent-distribution sampling, while higher multi-scale embedding factors improve KLD until unstable training prevents testing L ≥4.The sampling ablation evaluates camera configurations ranked by NAFlow; the scale ablation evaluates SIDD-Validation KLD.

6 CONCLUSION

The conclusion presents NAFlow and Noise-Aware Sampling as a unified approach for modeling diverse camera-dependent noise and synthesizing realistic noisy sRGB images. It emphasizes metadata-free inference, multi-scale noise correlation modeling, and improved noise-modeling and denoising performance.

  • 6 CONCLUSION: NAFlow models complex, diverse noise distributions arising from varied camera settings.The framework is paired with the Noise-Aware Sampling algorithm.
  • 6 CONCLUSION: Noise-Aware Sampling uses Gaussian mixture models derived from multiple noise distributions to synthesize realistic noisy sRGB images.The algorithm builds on the NAFlow framework.
  • 6 CONCLUSION: NAFlow eliminates the need to input noisy-image metadata during inference and models noise correlations across multiple scales.The conclusion identifies both properties as advantages of the method.
  • 6 CONCLUSION: Experiments report significant performance improvement in noise modeling and denoising, supporting the effectiveness of the proposed methods.This is the conclusion’s overall experimental claim.

A.1 NAFLOW ARCHITECTURE DETAIL

NAFlow combines invertible normalizing-flow components with clean-image conditioning to model real noise across scales. Its conditional affine coupling and affine injector layers use clean-image features to transform noisy-image representations.

  • Architecture: NAFlow’s architecture draws on prior normalizing-flow designs and includes conditional affine coupling, affine injection, invertible convolution, Actnorm, and Squeeze operations.These components provide invertible transformations for the conditional noise-modeling pipeline.
  • Conditional Affine Coupling: Conditional affine coupling splits the input into x1 and x2, preserves x1, and affinely transforms x2 using conditional input h.The scale and translation networks produce the transformation factors, while the inverse uses element-wise division and subtraction.
  • Affine Injector: Affine Injector transforms input features solely from conditional features, increasing clean-signal influence on each normalizing-flow scale block.The paper adopts it because learning signal dependence is important for modeling real-world noise.
  • Flow Components: Invertible 1×1 convolution independently mixes channels at each spatial coordinate and uses LU decomposition to reduce determinant computation and improve training stability.Actnorm separately normalizes channel-wise features with learned affine parameters.
  • Squeeze: Squeeze is invertible and moves each spatial 2 × 2 pixel neighborhood into channels, reducing resolution while enabling broader spatial correlations.The operation is used within the normalizing-flow module.

A.2 QUALITATIVE RESULTS OF NOISE GENERATION

Qualitative comparisons show that NAFlow generates noise visually closest to real noise. It outperforms the compared methods in noise correlation and KL-based similarity.

  • Visualization: NAFlow’s sampled noise is most similar to real noise among NAFlow, C2N, and NeCA comparisons.The comparison uses visualizations of synthetic noisy images and their extracted noise patterns.
  • Comparison: NAFlow outperforms C2N and NeCA in both noise correlation and KL metric comparisons.C2N can achieve similar KLD for low-light images but does not consider noise correlation, while NeCA performs better than C2N.

A.3 QUALITATIVE RESULTS OF DENOISING PERFORMANCE

Denoising results from DnCNN trained with NAFlow-generated data show stronger performance than the compared synthetic-noise models. NAFlow also produces the highest visual similarity to clean images in the reported comparison.

  • Experimental Setup: The denoising comparison evaluates DnCNN models trained using noise generated by NAFlow, C2N, and NeCA.Official pretrained parameters and test code are used for the compared methods.
  • Denoising Results: NAFlow surpasses C2N and NeCA in DnCNN denoising results, achieving the highest PSNR and greatest similarity to the clean image.The comparison uses denoisers and related code based on official pretrained weights and repositories.

A.4 NOISY GENERATION FROM METADATA

NAFlow can generate noisy images from clean images using camera configurations and ISO levels learned from known distributions. The SIDD dataset limits generalization because of restricted smartphone and scene diversity and its simplified RAW-to-sRGB pipeline.

  • Camera Configuration: NAFlow generates images for three cameras—GP, N6, and S6—across ISO 100, 400, 800, 1600, and 3200 settings.At the same ISO, generated noise differs slightly across camera sensor models.
  • Dataset Limitation: SIDD’s limited smartphone-model range and scene diversity make generalization to real-world noise modeling challenging.The paper identifies a need for larger-scale real-world noise datasets.
  • Dataset Limitation: SIDD’s simplified RAW-to-sRGB acquisition pipeline can create a distribution gap from noise in real-world images.The paper states that this limitation should be addressed in future research.
  • Figures: Figures 8 and 9 present visual comparisons for noise generation and denoising performance, respectively.Figure 10 shows generated images using camera configurations of NAFlow.
Loading 2608.29038v1…