Source-linked AI summary
Retinex-inspired Unrolling with Cooperative Prior Architecture Search for Low-light Image Enhancement
Risheng Liu, Long Ma, Jiaao Zhang, Xin Fan, Zhongxuan Luo
TL;DR
Low-light enhancement must address degraded visibility, contrast, and noise, while existing methods often depend on elaborate architectures and training data. RUAS combines Retinex-based optimization unrolling with cooperative, reference-free architecture search over compact low-light priors. The resulting networks are reported as memory- and computation-efficient and favorable against state-of-the-art approaches across challenging benchmarks.
Problem
Low-light images suffer poor visibility, low contrast, and noise, while existing CNN methods rely on elaborate architectures and carefully selected paired or unpaired training data.
Method
RUAS models low-light structure using the Retinex rule, unrolls optimization processes, and cooperatively searches illumination and noise-removal architectures without paired or unpaired supervision.
Results
RUAS establishes memory- and computation-efficient enhancement networks that perform favorably against state-of-the-art approaches on challenging benchmark datasets.
Takeaways & Limitations
The framework provides a lightweight, reference-free approach for constructing enhancement networks suited to different low-light scenarios.
Takeaways & Limitations
The search models assume certain terms are associated only with the architecture variables αt and αn.
Abstract
from arXiv · showhide
Low-light image enhancement plays very important roles in low-level vision field. Recent works have built a large variety of deep learning models to address this task. However, these approaches mostly rely on significant architecture engineering and suffer from high computational burden. In this paper, we propose a new method, named Retinex-inspired Unrolling with Architecture Search (RUAS), to construct lightweight yet effective enhancement network for low-light images in real-world scenario. Specifically, building upon Retinex rule, RUAS first establishes models to characterize the intrinsic underexposed structure of low-light images and unroll their optimization processes to construct our holistic propagation structure. Then by designing a cooperative reference-free learning strategy to discover low-light prior architectures from a compact search space, RUAS is able to obtain a top-performing image enhancement network, which is with fast speed and requires few computational resources. Extensive experiments verify the superiority of our RUAS framework against recently proposed state-of-the-art methods.
1. Introduction
Low-light images degrade downstream vision through poor visibility, low contrast, and noise, while existing enhancement methods can depend on elaborate architectures and data. RUAS addresses these issues with Retinex-guided unrolling and cooperative, reference-free architecture search to build efficient enhancement networks.
- Motivation: Low-light images exhibit poor visibility, low contrast, and unexpected noise, motivating enhancement before downstream vision tasks.The affected applications include object detection, segmentation, and recognition.
- Limitations of Existing Methods: Classical Retinex-based methods use handcrafted priors that may be insufficiently adaptive and can produce noise or over- and under-enhancement.
- Limitations of Existing Methods: CNN-based methods often require elaborate architectures and carefully selected paired or unpaired training data, yet can perform unsatisfactorily on noisy real-world images.The cited reason is that their underlying models lack principled physical constraints for capturing intrinsic low-light structure.
- RUAS: RUAS uses Retinex-informed optimization models and unrolls their processes to establish a holistic enhancement-network propagation structure.The framework is designed to exploit latent low-light structures in real-world noisy scenarios.
- RUAS: RUAS automatically discovers embedded prior architectures through a cooperative bilevel strategy for illumination estimation and noise removal without paired or unpaired supervision during search.
- Results: RUAS produces memory- and computation-efficient networks that perform favorably against state-of-the-art approaches and can search priors for different low-light scenarios.The paper points to Figure 1 for the comparative evaluation.
2. Related Work
Related low-light enhancement work spans diverse CNN designs and supervision settings, while neural architecture search offers a route to discovering effective or efficient architectures. Earlier search strategies, however, can require substantial computation.
- CNN-based LIE: CNN-based low-light enhancement methods include autoencoders, attention-enhanced U-Nets, frequency-based networks, Retinex decompositions, bilateral learning, self-supervision, zero-reference curves, and recursive bands.
- CNN-based LIE: These methods use varied supervision regimes, including paired, unpaired, self-supervised, zero-reference, and semi-supervised training.
- Neural Architecture Search: Neural architecture search has explored evolutionary algorithms and reinforcement learning to generate or optimize network architectures and parameters.
- Neural Architecture Search: Evolutionary and reinforcement-learning NAS approaches require a large amount of computation.
3. The Proposed Method
RUAS combines Retinex-inspired optimization unrolling with cooperative, reference-free architecture search to build lightweight low-light enhancement networks. Its pipeline estimates illumination, removes noise, and jointly searches compact prior architectures for both modules.
- Framework overview: RUAS unrolls Retinex-inspired optimization models into a holistic enhancement network, followed by a distillation-cell search space and cooperative bilevel architecture search.The framework targets illumination estimation and noise removal while integrating optimization structure with learned priors.
- Retinex-inspired optimization unrolling: The Retinex rule models the observed image y as the element-wise product of the desired recovery x and illumination map t.IEM estimates t, while NRM suppresses noise in challenging low-light scenarios.
- Illumination Estimation Module: IEM warm-starts illumination from u_k = y ⊘ t_k, uses local-channel maxima and residual correction, then refines t across K unrolled stages.The residual term adaptively suppresses overexposed pixels, and the CNN parameterizes the regularization gradient in each stage.
- Noise Removal Module: NRM solves a regularized model anchored to the IEM output u_K and unrolls updates whose regularization gradient is parameterized by CNRM.The module produces the recovered image x_N after N stages.
- Compact search space: The compact search space represents CIEM and CNRM as five-node distillation cells with convolution, residual, dilation, and skip operations encoded by architecture vectors α.Architecture parameters and layer weights are jointly learned over mixed operations.
- Reference-free bilevel learning: Cooperative bilevel learning alternates updates for illumination and noise-removal architectures using reference-free training and validation losses.The validation objective combines IEM and NRM losses, with the NRM task depending on the illumination architecture.
- Experiments: Table 1 reports PSNR and SSIM comparisons on MIT-Adobe 5K and LOL, while Figure 3 visualizes DarkFace comparisons with red boxes marking obvious differences.The supplied figure and table passages identify the evaluation layouts but do not provide the individual numerical table entries.
4. Experimental Results
RUAS achieves strong low-light enhancement across challenging real-world datasets while using lightweight, fast architectures discovered through cooperative search. Ablations show benefits from warm-start residual rectification, noise removal, and task-informed architecture search.
- RUAS obtained the best scores on the MIT-Adobe 5K benchmark among the evaluated methods.
- On LOL, RUAS produced vivid colors and removed undesired noise, while competing methods introduced artifacts or failed to restore realistic colors.
- RUAS achieved strong detail restoration and noise removal on extremely challenging real-world examples, where competing methods produced visible artifacts and color distortions.
- RUAS required very small model size, FLOPs, and inference time compared with manually designed CNN-based methods.The comparison reports model size, FLOPs, and GPU-seconds on 100 LOL images sized 600×400.
- Residual rectification in the warm-start strategy suppressed over-exposure and produced more comfortable exposure than alternative strategies.
- NRM improved noisy-scene enhancement by removing visible noise that damaged results without the module.
- The searched architecture achieved the highest PSNR with less inference time than naively designed alternatives on MIT-Adobe 5K.
- Cooperative search connected illumination estimation and denoising, with the searched NRM containing more residual convolutions and skip connections.
5. Conclusion
The paper presents RUAS, which combines Retinex-based optimization unrolling with cooperative, reference-free prior architecture search. Experiments on challenging benchmarks yield effective yet lightweight enhancement networks and new state-of-the-art results.
- RUAS integrates principled optimization unrolling with cooperative prior architecture search to design effective, lightweight low-light enhancement networks.
- Retinex-based optimization models formulate latent illumination and desired-image structures, while unrolling produces the enhancement network’s holistic structure.
- A cooperative reference-free strategy discovers specific architectures from a compact search space for low-light enhancement.
- Experiments on challenging benchmark datasets produced new state-of-the-art results.