Source-linked AI summary
LLNet: A Deep Autoencoder Approach to Natural Low-light Image Enhancement
Kin Gwn Lore, Adedotun Akintayo, Soumik Sarkar
TL;DR
Low-light and noisy images from inexpensive sensors can impair critical visual applications. The paper trains stacked sparse denoising autoencoders on synthetically darkened and noisy patches to enhance and denoise images, reporting strong visual and quantitative performance across varied conditions.
Problem
Insufficient lighting and low sensor quality produce noisy images that can inhibit surveillance, military, and commercial applications.
Method
Stacked sparse denoising autoencoders are trained on synthetically darkened and Gaussian-noisy image patches to reconstruct clean, enhanced images.
Results
LLNet and S-LLNet perform consistently well across varied lighting and noise situations, with LLNet achieving the highest PSNR on an artificially darkened image.
Takeaways & Limitations
Deep autoencoders can learn low-light signal characteristics and noise structures without handcrafting and apply the learned model to natural low-light images.
Takeaways & Limitations
The study uses Gaussian-only noise as a preliminary feasibility model, although natural low-light images may also contain quantization and Poisson noise.
Abstract
from arXiv · showhide
In surveillance, monitoring and tactical reconnaissance, gathering the right visual information from a dynamic environment and accurately processing such data are essential ingredients to making informed decisions which determines the success of an operation. Camera sensors are often cost-limited in ability to clearly capture objects without defects from images or videos taken in a poorly-lit environment. The goal in many applications is to enhance the brightness, contrast and reduce noise content of such images in an on-board real-time manner. We propose a deep autoencoder-based approach to identify signal features from low-light images handcrafting and adaptively brighten images without over-amplifying the lighter parts in images (i.e., without saturation of image pixels) in high dynamic range. We show that a variant of the recently proposed stacked-sparse denoising autoencoder can learn to adaptively enhance and denoise from synthetically darkened and noisy training examples. The network can then be successfully applied to naturally low-light environment and/or hardware degraded images. Results show significant credibility of deep learning based approaches both visually and by quantitative comparison with various popular enhancing, state-of-the-art denoising and hybrid enhancing-denoising techniques.
1. Introduction and motivation
Low-light, noisy imagery from inexpensive sensors can hinder critical visual decision-making. LLNet applies deep autoencoders trained on synthetically degraded images to jointly enhance and denoise natural low-light images.
- Inexpensive sensors and insufficient lighting can produce noise that inhibits surveillance, military, and commercial imaging applications.
- Prior deep networks addressed denoising, but the paper identifies no existing deep-network work jointly enhancing and denoising poorly illuminated images.
- LLNet uses stacked sparse denoising autoencoders to learn low-light signal features and adaptively brighten and denoise images.
- The method uses synthetic image modification to simulate low-light environments for training.
- Two architectures are explored: LLNet jointly learns contrast enhancement and denoising, while S-LLNet learns them sequentially in separate modules.
2. Related work
The paper situates LLNet among established contrast-enhancement and denoising methods, including histogram-based techniques, BM3D, KSVD, and denoising autoencoders.
- Histogram equalization and related methods enhance contrast by redistributing or mapping image intensities.
- BM3D, KSVD, and nonlinear filters represent established approaches to image denoising.
- Denoising autoencoders and convolutional neural networks have been used to learn features or reconstruct clean images from noisy inputs.
3. The Low-light Net (LLNet)
LLNet is a stacked sparse denoising autoencoder trained on synthetically darkened and Gaussian-noisy patches to reconstruct clean, brighter images. S-LLNet separates enhancement and denoising into sequentially trained modules.
- Learning features from low-light images: Stacked denoising autoencoders are pretrained layerwise to learn invariant low-light features, then fine-tuned through backpropagation.
- A denoising autoencoder maps corrupted input x through hidden activations to reconstructed output ŷ(x).
- LLNet learns from internet-image patches synthetically darkened and corrupted with Gaussian noise, using clean bright patches as reconstruction targets.
- S-LLNet trains separate contrast-enhancement and denoising modules sequentially, increasing training flexibility but slightly increasing inference time.
- Gamma adjustment randomly darkens 17×17 patches, while noise levels vary to simulate multiple low-light conditions.
- The study uses Gaussian noise as a preliminary feasibility model, while noting that natural low-light images may also contain quantization and Poisson noise.
- Training uses 422,500 patches, including 211,250 training examples and 211,250 validation samples, with mean-squared reconstruction error.
- Image reconstruction: During inference, overlapping 17 × 17 patches with stride 3 × 3 are enhanced and denoised, then reassembled by averaging overlaps.
4. Evaluation metrics and compared methods
The evaluation compares LLNet and related enhancement-denoising methods using reconstruction and perceptual image-quality metrics. Training reconstructs clean images from synthetically darkened and noisy inputs.
- Figure 2 depicts training on synthetically darkened and noisy images, comparison with uncorrupted targets, and backpropagation of reconstruction error.
- The evaluation uses PSNR and SSIM to assess image reconstruction and perceived similarity to reference images.
- PSNR measures image corruption relative to the original, with higher values generally indicating better denoising for the same compression code.
- SSIM measures similarity and perceived image quality relative to an uncompressed or distortion-free reference image.
- Compared methods include histogram equalization, CLAHE, gamma adjustment, and HE followed by BM3D denoising.
- The HE+BM3D comparison applies histogram equalization before BM3D because reversing the order significantly amplifies patch boundaries.
5. Results and discussion
LLNet and S-LLNet generally enhance and denoise synthetically darkened, noisy, and naturally low-light images while limiting overamplification. Results also reveal trade-offs between denoising, sharpness, structural similarity, and inference flexibility.
- Algorithm adaptivity: LLNet learns adaptive brightening that avoids strong over-brightening on already-bright images, although histogram-equalization methods can score slightly higher on some original images.On the ‘Town’ image, LLNet is slightly brighter without the washed-out appearance of gamma adjustment; HE-based methods generally achieve slightly higher PSNR and SSIM on other original images.
- Synthetic darkening and noise: S-LLNet attains the highest PSNR and SSIM on darkened ‘Town’ images corrupted with Gaussian noise at σ = 18 and σ = 25.Across Table 1’s dark, noisy images, only LLNet and S-LLNet achieve the highest PSNR and SSIM results.
- Natural low-light images: LLNet improves local contrast while suppressing noise in naturally dark images, whereas histogram equalization methods can amplify noise and create blooming.HE and HE+BM3D overamplify light from the computer display; CLAHE reduces blooming but still tends to amplify image noise.
- Patch size and image quality: Decreasing relative patch size produces sharper edges with more noise, while PSNR favors smoother, less-sharp images and SSIM provides a more balanced structural measure.An optimal patch size can maximize PSNR or SSIM, but selecting by PSNR yields lower noise and less sharpness, whereas the smallest patch size yields greater sharpness and more noise.
- Practical applicability: Deep models trained on varying darkening and noise levels can be applied to real-world low-light images without detailed knowledge of the camera or environment.This supports using LLNet and S-LLNet when enhancement parameters are otherwise difficult to obtain through trial and error.
6. Conclusions and future works
LLNet and S-LLNet show that deep autoencoders can enhance naturally low-light images across varying lighting and noise conditions. The paper identifies broader realism, sharpness, robustness, and human-evaluation improvements for future work.
- Conclusions: A variant of the stacked sparse denoising autoencoder learns brightening and denoising filters from synthetic examples for naturally low-light and degraded images.The learned filters are applied to images with varying degrees of degradation.
- Results: Natural-image testing shows that LLNet can achieve optimal PSNR or SSIM at suitable relative patch sizes.The USAF resolution-chart experiment compares patch-size choices using PSNR and SSIM and contrasts them with histogram equalization.
- Conclusions: LLNet and S-LLNet compete favorably with histogram equalization, CLAHE, gamma adjustment, and HE+BM3D across lighting and noise conditions.Some competing methods remain competitive in particular scenarios, but the proposed networks perform consistently well across varied conditions.
- Future works: Future work includes modeling Poisson noise and quantization artifacts, explicitly improving deblurring and sharpness, and handling combined noise types.The proposed extensions also include foggy and dusty scenes and subjective evaluation by human users.