Source-linked AI summary
Denoising Textures is a Different Ballgame Altogether!
Saptarshi Mandal, Oindrila Haldar, Chirayata Bhattacharya, Chandra Sekhar Seelamantula, Zhiwei Xu, Thrasyvoulos N. Pappas, Gaurav Sharma
TL;DR
Texture denoising remains under-explored, and intricate patterns can be blurred by conventional denoisers. The paper benchmarks and fine-tunes state-of-the-art denoisers on texture data, finding improved texture metrics but degraded performance on original datasets.
Problem
Texture denoising is under-explored, and conventional denoisers can blur intricate patterns despite denoising's importance to downstream image and video processing.
Source: §I · pp. 1–2Method
The study benchmarks state-of-the-art denoisers on textures, fine-tunes them on texture data, and evaluates them with texture-oriented metrics alongside PSNR and SSIM.
Source: §I · p. 2, §III · p. 3Results
Fine-tuned models increased PSNR by +6.44 to +9.66 dB and improved LPIPS by around -0.16 to -0.31 over baseline models, but performed worse on original datasets.
Source: §IV · pp. 4–5Takeaways & Limitations
The study highlights the need for denoisers that handle textures better, either separately or alongside general image datasets.
Source: §V · p. 5
Abstract
from arXiv · showhide
The human brain effortlessly deciphers colors and textures from visual stimuli and reconstructs the world around us, yet preserving such details remains challenging in image denoising. While significant progress has been made in generic image denoising, the problem of texture denoising has remained relatively under-explored. Fine details, such as intricate patterns and repetitive structures, often suffer from over-smoothing even in deep learning based approaches for denoising. In this paper, we analyze the behavior of state-of-the-art (SOTA) denoisers on texture datasets and show that performance improvements are possible only by fine-tuning the denoisers on the texture datasets. Surprisingly, fine-tuning on texture datasets result in the models catastrophically forgetting the original dataset. These findings highlight the need for specialized treatment of textures in denoising. Since denoising quality directly impacts downstream tasks such as generative modelling, we argue that dedicated texture aware denoisers are essential for faithful texture reconstruction and synthesis.
I. INTRODUCTION
Image denoising must recover clean images from noisy observations, but texture-specific denoising remains under-explored despite the difficulty of preserving intricate patterns. The paper studies denoiser performance on textures and reports improved results after fine-tuning.
I. INTRODUCTION
- Denoising matters to downstream image and video processing, and it is central to denoising diffusion models,.Source: §I · p. 1
- Image denoising seeks to estimate a pristine image from a noisy observation, commonly modeled as y = x + n with additive white Gaussian noise.The passage frames the estimate as as accurate as possible because perfect restoration is not possible.Source: §I · p. 1
- Fine-tuning pretrained MIRNet improves the displayed texture PSNR and SSIM values.The figure contrasts a noisy texture with denoised results from pretrained and texture-fine-tuned MIRNet.Source: Figure 1 · p. 1, §I · p. 2
- Texture denoising is under-explored, and natural textures combine fine detail and repetition with some stochasticity.The passage also describes texture as spatial variation in pixel brightness.Source: §I · p. 1
- Conventional denoisers tend to blur intricate texture patterns, making their preservation more challenging than in generic image denoising.The paper reports that models trained on standard image-denoising datasets perform poorly on textures, both quantitatively and qualitatively.Source: §I · p. 2
- The study assesses denoiser generalizability with PSNR, SSIM, and perceptual metrics, including texture-focused STSIM1 and STSIM2.It also uses MS-SSIM, CW-SSIM, and DISTS; STSIM1 and STSIM2 are specifically designed to assess texture patterns.Source: §I · p. 2
II. RELATED WORKS
Prior denoising work progressed from classical filters to CNNs and newer restoration architectures, addressing trade-offs in spatial detail, context, noise realism, and computational cost.
II. RELATED WORKS
- Classical methods such as BM3D were surpassed by CNN approaches, with DnCNN using deep residual learning to remove Gaussian noise.DAEs learned noise distributions to recover clean images through meaningful intermediate representations.Source: §II · p. 2
- MIRNet combines parallel multi-scale feature extraction with attention aggregation, while MPRNet pairs encoder-decoder context with a detail-preserving high-resolution branch.Source: §II · p. 2
- CycleISP models camera imaging in forward and reverse directions to generate realistic RAW and sRGB image pairs.The passage notes that AWGN, commonly used in synthetic denoising data, does not accurately represent real-world camera noise.Source: §II · p. 2
- NAFNet reduces complexity by removing or replacing nonlinear activations with multiplications while improving performance.Source: §II · p. 2
- Restormer uses global context, while CAT applies horizontal and vertical rectangle-window attention across heads to reduce global-attention cost.Source: §II · pp. 2–3
- Figure 3 compares noisy, pretrained, and fine-tuned model results on SIDD, showing PSNR relative to the reference.Source: Figure 3 · p. 3
III. EXPERIMENTAL SETUP
The experimental setup benchmarks state-of-the-art denoisers on textures, where image-distribution shifts and texture structure pose distinct challenges.
III. EXPERIMENTAL SETUP
- Texture denoising is challenging because texture-rich images differ structurally from natural images and contain high-frequency information also present in noise.Textures are described as spatially homogeneous repeated structures with some random variation, while natural images predominantly contain smooth regions.Source: §III · p. 3
- The study benchmarks six denoisers on textures at their original noise levels, then fine-tunes them on the texture dataset and evaluates them again.The selected models are MIRNet, MPRNet, CAT, CycleISP, NAFNet, and Restormer.Source: §III · p. 3
- Fine-tuning was used instead of training from scratch because the study lacked the substantially larger dataset required for scratch training.Source: §III · p. 3
A. Datasets
The texture dataset preparation samples patches from NUUR-Texture500 and adds Gaussian noise calibrated to SIDD noise estimates.
A. Datasets
- NUUR-Texture500 contains 500 diverse, spatially consistent texture images spanning regular patterns to stochastic textures.Source: A. Datasets · p. 3
- Ten randomly sampled 256 × 256 patches per image yield 5,000 patches, split 75% for training and 25% for validation.Source: A. Datasets · p. 3
- White Gaussian noise was added with standard deviation set by estimating average noise variance in noisy SIDD images.The aim was to match synthetic texture-patch noise to real-world datasets such as SIDD.Source: A. Datasets · p. 3
B. Implementation Details
Fine-tuning used Adam on six Nvidia A6000 GPUs for 100 epochs with batches of 16. The learning rate was reduced by cosine annealing, and each model retained its original training objective.
B. Implementation Details
- Fine-tuning ran for 100 epochs with batch size 16 on six Nvidia A6000 GPUs using Adam (β1 = 0.9, β2 = 0.999).The initial learning rate was 1 × 10−4.Source: B. Implementation Details · p. 3
- Cosine annealing reduced the learning rate to 5 × 10−5 during training.Source: B. Implementation Details · p. 3
- Fine-tuning retained each architecture’s original training objective, including L1 loss for CAT and Charbonnier loss for MPRNet.Source: B. Implementation Details · p. 3
C. Evaluation Metrics
The evaluation combines fidelity, perceptual, and texture-oriented metrics. STSIM measures texture similarity using image subband statistics, while DISTS assesses structural and texture information.
C. Evaluation Metrics
- Performance was evaluated with PSNR, SSIM, MS-SSIM, CW-SSIM, STSIM1, STSIM2, DISTS, and LPIPS.Table I reports mean ± std on NUUR-Texture500; Table II compares original and fine-tuned models on SIDD.Source: C. Evaluation Metrics · p. 3, Table I · p. 4, Table II · p. 4
- STSIM-1 captures texture similarity through local image statistics, and combining its values over subbands yields an aggregate measure.Nb denotes the number of subbands, while Nc accounts for cross-correlations across orientations.Source: C. Evaluation Metrics · pp. 3–4
- STSIM-M compares image feature vectors containing subband means, variances, autocorrelations, and cross-correlations using Mahalanobis distance.The features are treated as mutually uncorrelated and combined.Source: C. Evaluation Metrics · p. 4
- DISTS assesses structural and texture similarity using perceptual features from different layers of a pretrained VGG network.Source: C. Evaluation Metrics · p. 4
IV. RESULTS
Fine-tuning improved denoising on NUUR-Texture500 across fidelity and perceptual measures, but reduced performance on the original natural-image dataset. The authors interpret this as catastrophic forgetting during adaptation.
IV. RESULTS
- PSNR increased by +6.44 to +9.66 dB and LPIPS improved by around -0.16 to -0.31 over the baseline models after fine-tuning.The corresponding validation-set comparisons are reported in Table I; fine-tuned models also improved on non-pointwise perceptual metrics.Source: §IV · p. 4
- STSIM1 and STSIM2 improved by +0.065 and +0.034 on average, respectively, after fine-tuning.The reported gains span multiple scales and texture-specific structural similarity measures.Source: §IV · p. 5
- Fine-tuned models performed worse on the original natural-image dataset, which the authors characterize as catastrophic forgetting during texture adaptation.They infer that natural images and textures have minimally overlapping low-dimensional embeddings and identify a unified denoiser as future work.Source: §IV · p. 5
V. CONCLUSION
Off-the-shelf image denoisers perform subpar on textures, while texture fine-tuning improves texture denoising at the cost of significant degradation on the original dataset. The study calls for better texture-capable denoisers and identifies perceptual-loss training as future work.
V. CONCLUSION
- Texture fine-tuning improves texture denoising but significantly degrades performance on the original dataset.The study reports that off-the-shelf denoisers perform subpar on textures compared with generic natural images.Source: §V · p. 5
- Point-by-point training losses such as L1 leave further work needed to use perceptual metrics such as STSIM-M and STSIM-VGG.Source: §V · p. 5
- The proposed next step is unified denoisers trained with non-point-by-point perceptual losses, which the authors believe could support natural-image generation with rich texture.They identify diffusion or flow-based modeling frameworks as potential settings.Source: §V · p. 5