Source-linked AI summary
FourLLIE: Boosting Low-Light Image Enhancement by Fourier Frequency Information
Chenxi Wang, Hongjun Wu, Zhi Jin
TL;DR
Low-light enhancement needs to recover visibility and hidden information, while prior LLIE methods often underuse Fourier frequency information. FourLLIE estimates amplitude changes for lightness, then uses an SNR-guided spatial stage to recover details; it outperforms existing SOTA LLIE methods on four datasets with a lightweight architecture.
Problem
Low-light images suffer from poor visibility, and most LLIE methods rely on spatial information while rarely considering Fourier frequency information.
Method
FourLLIE uses a frequency stage to estimate an amplitude transform map for lightness improvement and an SNR-guided spatial stage to integrate global Fourier and local spatial information for detail refinement.
Results
FourLLIE outperforms existing SOTA LLIE methods on four representative datasets while maintaining good model efficiency.
Takeaways & Limitations
The positive correlation between amplitude magnitude and lightness provides a Fourier-space route for lightness improvement, while Fourier features supply global information efficiently.
Takeaways & Limitations
The authors plan to make FourLLIE adaptive for more diverse degradation.
Abstract
from arXiv · showhide
Recently, Fourier frequency information has attracted much attention in Low-Light Image Enhancement (LLIE). Some researchers noticed that, in the Fourier space, the lightness degradation mainly exists in the amplitude component and the rest exists in the phase component. By incorporating both the Fourier frequency and the spatial information, these researchers proposed remarkable solutions for LLIE. In this work, we further explore the positive correlation between the magnitude of amplitude and the magnitude of lightness, which can be effectively leveraged to improve the lightness of low-light images in the Fourier space. Moreover, we find that the Fourier transform can extract the global information of the image, and does not introduce massive neural network parameters like Multi-Layer Perceptrons (MLPs) or Transformer. To this end, a two-stage Fourier-based LLIE network (FourLLIE) is proposed. In the first stage, we improve the lightness of low-light images by estimating the amplitude transform map in the Fourier space. In the second stage, we introduce the Signal-to-Noise-Ratio (SNR) map to provide the prior for integrating the global Fourier frequency and the local spatial information, which recovers image details in the spatial space. With this ingenious design, FourLLIE outperforms the existing state-of-the-art (SOTA) LLIE methods on four representative datasets while maintaining good model efficiency.
1 INTRODUCTION
LLIE methods increasingly incorporate Fourier frequency information because amplitude primarily represents lightness, while Fourier features provide global information without massive neural-network parameters. FourLLIE uses these properties in a two-stage design combining amplitude-based lightness enhancement with SNR-guided spatial detail recovery.
- Fourier-based LLIE research identifies amplitude as the main carrier of lightness representation, while phase contains lightness-irrelevant information such as structure or noise.
- Amplitude swapping follows the light conditions of images with shared context, and enlarging low-light amplitude magnitudes brightens the image.
- Fourier transforms extract global image information without introducing the massive neural-network parameters associated with MLPs or Transformers.
- FourLLIE estimates an amplitude transform map in the frequency stage to improve lightness, then uses an SNR map to integrate Fourier and spatial information for detail refinement.
- The method further investigates the positive correlation between amplitude magnitude and lightness magnitude and reports improved performance with good model efficiency on four LLIE datasets.
2 RELATED WORK
LLIE research spans non-learning and learning-based methods, while recent work increasingly combines spatial processing with Fourier frequency information. Fourier-based methods motivate FourLLIE but leave further amplitude-relationship exploration as an open direction.
- Earlier LLIE methods include histogram-equalization and Retinex-based approaches, but they may struggle to handle noise and color well.
- Learning-based LLIE architectures progressed from CNNs and bilateral up-sampling to attention, multi-scale, Transformer, and SNR-map designs.
- Prior Fourier-based LLIE studies associate amplitude with lightness and decompose lightness and noise in Fourier space, then combine frequency and spatial information.
- Fourier frequency information has also been applied to augmentation, super-resolution, dehazing, pan-sharpening, and up-sampling across computer vision.
3 METHOD
FourLLIE uses Fourier frequency information to improve lightness, then combines global Fourier and local spatial processing with an SNR prior to recover details. Its amplitude-transform design and Fourier processing achieve effective enhancement with substantially lower model complexity than Transformer-based processing.
- 3.1 Fourier Frequency Information: Fourier frequency information represents an image through amplitude and phase components, with lightness improvement linked to enlarging amplitude magnitude.The Fourier transform is also used to extract global information without introducing massive neural-network parameters.
- 3.2 Frequency Stage in FourLLIE: FourLLIE’s frequency stage estimates an amplitude transform map to enlarge the input amplitude and improve low-light image lightness.The stage uses Fourier Processing blocks with skip connections and supervises the output amplitude against the ground truth amplitude.
- 3.2 Frequency Stage in FourLLIE: The amplitude transform map is constrained within (0,1), exploiting the positive correlation between amplitude and lightness for more reasonable Fourier-space enhancement.This design differs from directly constraining the amplitude component as in FECNet.
- 3.2 Frequency Stage in FourLLIE: Setting 3, which predicts an amplitude transform map, recovers image details and amplitude structure better than directly predicting amplitude or predicting the spatial output.The ablation reports the best results for setting 3 in both spatial and Fourier spaces.
- 3.3 Spatial Stage in FourLLIE: The spatial stage uses an SNR map to route low-SNR regions toward global Fourier processing and high-SNR regions toward local spatial processing.Features are separately processed by Fourier Processing and Spatial Processing blocks before decoding the enhanced output.
- 3.3 Spatial Stage in FourLLIE: Replacing Transformer blocks with Fourier Processing achieves competitive performance with about 1/20 of the parameters and remains effective with only 16 latent channels.The comparison uses SNR-Aware-Transformer and SNR-Aware-Fourier on the LSRW-Huawei dataset.
4 EXPERIMENT
FourLLIE is evaluated on paired, unpaired, qualitative, exposure-correction, and ablation settings across representative LLIE datasets. The results generally show strong enhancement quality, competitive visual recovery, and efficient Fourier-based processing.
- Datasets: FourLLIE is evaluated on LOL-Real, LOL-Synthetic, LSRW-Huawei, LSRW-Nikon, and five unpaired datasets.The paired datasets contain low-/normal-light image pairs, while the unpaired evaluation includes DICM, LIME, MEF, NPE, and VV.
- Quantitative comparison: The method reaches the best results in most quantitative cases and is nearly second-best in the remainder.Table 2 compares FourLLIE with thirteen state-of-the-art LLIE methods using PSNR, SSIM, and LPIPS.
- Model efficiency: FourLLIE achieves generally superior performance to SNR-Aware with only 0.03 % parameters.The comparison attributes the efficiency to Fourier frequency information as a global representation.
- Unpaired evaluation: On five unpaired datasets, FourLLIE outperforms most existing LLIE methods according to NIQE evaluation.Lower NIQE scores indicate higher naturalness image quality; SNR-Aware remains competitive but uses much more parameters.
- Qualitative comparison: Qualitative comparisons report the best visual results, including improved detail and lightness, recovery of complex textures, and clearer, less noisy outputs.These comparisons cover LOL-Real, LOL-Synthetic, LSRW-Huawei, and LSRW-Nikon.
- Exposure correction: FourLLIE also outperforms existing methods in PSNR and SSIM on SICE exposure correction.The frequency stage estimates an additional amplitude transform map for lightness depression.
5 CONCLUSION
The paper concludes that Fourier amplitude magnitude is positively related to image lightness and can guide enhancement. FourLLIE combines amplitude transformation with SNR-guided Fourier–spatial interaction in a lightweight architecture, while future work targets more diverse degradation.
- Conclusion: Lightness can be improved by enlarging the magnitude of the amplitude component in Fourier space.The conclusion presents this relationship as the basis for Fourier-space lightness improvement.
- Conclusion: FourLLIE first estimates an amplitude transform map, then uses an SNR map to interact between global Fourier information and spatial features for detail recovery.The two-stage architecture is designed around Fourier frequency properties and global information extraction.
- Conclusion: FourLLIE outperforms existing state-of-the-art LLIE methods with a lightweight architecture.The conclusion attributes this result to the effectiveness of Fourier frequency information.
- Future work: Future work will explore additional Fourier frequency properties to make the method adaptive to more diverse degradation.This is the paper’s stated future direction.
A DETAILED EXPERIMENT SETTINGS IN SECTION 3.2
Section 3.2 compares three experiment settings for Fourier-based enhancement: directly predicting amplitude, predicting the enhanced image with amplitude constraints, and estimating an amplitude transform map.
- Setting 1: Setting 1 directly predicts the amplitude component with a residual connection for faster convergence.
- Notation: The neural networks in the experiment are denoted by NN, while GT denotes the ground truth.
- Setting 2: Setting 2 directly predicts the enhanced image, adopts a residual connection, and constrains its amplitude component.
- Setting 3: Setting 3 estimates an amplitude transform map because amplitude magnitudes can reflect lightness magnitudes.
- Setting 3: The setting 3 constraint uses the amplitude transform map, with 𝜖 = 1e−8 to avoid zero-division.
B VARIANT FOR EXPOSURE CORRECTION
FourLLIE extends exposure correction by predicting separate amplitude transform maps for lightness improvement and lightness depression.
- Exposure correction: FourLLIE predicts two amplitude transform maps to correct exposure: one improves lightness and the other depresses it.
C COMPARISON WITH UHDFOUR
FourLLIE is compared with the Fourier-based LLIE method UHDFour, with the proposed method reported to achieve overall better performance.
- Comparison: FourLLIE is compared with the recent Fourier-based LLIE method UHDFour.
- Comparison: The proposed method reaches overall better performance than UHDFour in Table 6 and Fig. 9.