Source-linked AI summary
A Bio-Inspired Multi-Exposure Fusion Framework for Low-light Image Enhancement
Zhenqiang Ying, Ge Li, Wen Gao
TL;DR
Low-light enhancement must reveal under-exposed information without introducing contrast or lightness distortion. The paper proposes an HVS-inspired dual-exposure fusion framework that synthesizes and evaluates exposures before fusing them. Experiments report less contrast and lightness distortion than several state-of-the-art methods, while the authors identify over-enhancement and unresolved over-exposure as limitations.
Problem
Low-light images have poor visibility, and existing enhancement methods can introduce contrast under- and over-enhancement.
Method
The method estimates fusion weights, uses a camera response model to synthesize exposures, selects an exposure ratio, and fuses the input with the synthetic image.
Results
The method obtains less contrast and lightness distortion than several state-of-the-art methods.
Takeaways & Limitations
The framework provides an HVS-inspired dual-exposure approach for enhancing under-exposed regions while using fusion to preserve visual information.
Takeaways & Limitations
The method can over-enhance scene content mistaken for background illumination, and its two-image design leaves over-exposure unresolved.
Abstract
from arXiv · showhide
Low-light images are not conducive to human observation and computer vision algorithms due to their low visibility. Although many image enhancement techniques have been proposed to solve this problem, existing methods inevitably introduce contrast under- and over-enhancement. Inspired by human visual system, we design a multi-exposure fusion framework for low-light image enhancement. Based on the framework, we propose a dual-exposure fusion algorithm to provide an accurate contrast and lightness enhancement. Specifically, we first design the weight matrix for image fusion using illumination estimation techniques. Then we introduce our camera response model to synthesize multi-exposure images. Next, we find the best exposure ratio so that the synthetic image is well-exposed in the regions where the original image is under-exposed. Finally, the enhanced result is obtained by fusing the input image and the synthetic image according to the weight matrix. Experiments show that our method can obtain results with less contrast and lightness distortion compared to that of several state-of-the-art methods.
I. INTRODUCTION
Low-light scenes can hide information because raising exposure may saturate already bright regions, while single-image enhancement struggles to preserve accurate lightness and contrast. The paper addresses this with an HVS-inspired multi-exposure fusion framework and a dual-exposure method that synthesizes and fuses exposures.
- Limited camera dynamic range makes under-exposed information invisible while increasing exposure can over-expose or saturate well-exposed regions.
- Accurate lightness and contrast enhancement from a single image remains challenging because existing techniques may not reflect the scene’s true appearance.
- Human vision dynamically combines differently exposed views, motivating the use of exposure fusion for low-light enhancement.
- The framework uses a brightness transform function to generate correlated multi-exposure images and fuse them into an enhanced result.
- The proposed dual-exposure method estimates fusion weights, models camera response, selects an exposure for under-exposed regions, and fuses the input with the synthetic image.
- Existing global enhancement applies the same processing to all pixels, so local areas may lose detail or remain inadequately enhanced.
B. Local Enhancement Algorithms
The framework generates and evaluates synthetic exposures before combining them, using exposure selection to avoid unnecessary computation and pixel-wise weights to favor well-exposed content.
- A. Multi-Exposure Sampler: The Multi-Exposure Sampler selects the number of generated images and their exposure ratios, retaining only exposures that add useful scene information.Using fewer informative images reduces computation and may prevent deterioration of the fused result.
- B. Multi-Exposure Generator: The Multi-Exposure Generator maps the input into correlated multi-exposure images using a camera response model and brightness transform function.
- C. Multi-Exposure Evaluator: The Multi-Exposure Evaluator assigns larger weights to well-exposed pixels and smaller weights to poorly exposed pixels.
- C. Multi-Exposure Evaluator: The evaluator normalizes each image’s weight matrix pixel-wise so the matrices sum to one at every pixel.
D. Multi-Exposure Combiner
The combiner fuses the input image with a generated exposure using pixelwise weights, preserving well-exposed regions while enhancing under-exposed ones.
- The enhanced result is formed by fusing exposure images according to a weight matrix, with color channels combined into R.Alternative multi-scale and Boosting Laplacian Pyramid fusion techniques may also be used.
- The dual-exposure implementation reduces complexity by generating one appropriately exposed image and fusing it with the input.
- The fusion equation combines each input pixel with its generated counterpart using evaluator weights and complementary weights.The framework separates the evaluator, generator, and sampler before fusion.
- Weights should be large for well-exposed pixels and small for under-exposed pixels so preserved contrast is favored in well-exposed regions.
- The enhancement parameter µ is set to 0.5 because µ > 1 can saturate pixels and cause detail loss.µ = 0 produces no enhancement, while µ = 1 enhances both under-exposed and well-exposed pixels.
1) Optimization Problem:
The illumination estimator starts from image lightness and refines it with an optimization that preserves meaningful structures while smoothing textural variation.
- Image lightness provides the initial estimate of scene illumination, which is then refined to maintain meaningful structures and remove textural edges.
- The refinement objective balances fidelity to the initial illumination map against smoothness of the refined map.Its first term minimizes map differences, while its second term maintains smoothness.
- The weight matrix assigns smaller weights to windows containing meaningful edges than to windows containing only complex textures.
- The optimization is approximated so that only quadratic terms remain, enabling the solution to be obtained through a linear system.
- The proposed illumination map is faster than the method in, while the resulting enhanced images show no significant visual difference.The compared runtimes are 0.15s for the proposed map and 0.21s for.
B. Dual-Exposure Generator
The dual-exposure generator models exposure changes with camera response functions and brightness transforms, then selects an exposure ratio using image entropy.
- The camera response model contains a Camera Response Function model and a Brightness Transform Function model.CRF parameters depend only on the camera, whereas BTF parameters depend on the camera and exposure ratio.
- The BTF is modeled from two images differing only in exposure, using linear amplification followed by gamma correction to approximate a well-exposed image.
- The BTF parameters β and γ are related to exposure ratio k, and approximately shared across color channels for general cameras.The shared-channel assumption follows from approximately identical response curves across color channels.
- The CRF has separate cases for γ = 1 and γ ≠ 1, yielding respectively a power-function CRF with linear BTF or a two-parameter CRF with nonlinear BTF.
- The optimal exposure ratio is selected by maximizing enhancement-image entropy because well-exposed images have higher entropy than under- or over-exposed images.The entropy examples are illustrated with multi-exposure image sets.
- The method uses fixed camera parameters a = −0.3293 and b = 1.1258 when camera information is unavailable.
C. Dual-Exposure Sampler
The dual-exposure sampler generates one synthetic exposure focused on under-exposed regions and selects its exposure ratio by maximizing image entropy.
- The method excludes well-exposed pixels to obtain a globally under-exposed image for exposure-ratio optimization.
- Because exposure changes brightness more than color, the sampler estimates the exposure ratio using only the brightness component.Brightness is defined as the geometric mean of the red, green, and blue channels.
- The geometric mean is chosen because all three color channels share the same BTF parameters β and γ.
- Image entropy measures information content, with well-exposed images expected to have higher entropy than under- or over-exposed images.The entropy uses histogram-bin probabilities p_i and N bins, often N = 256.
- The optimal exposure ratio is found by maximizing enhancement-brightness entropy and solving the resulting one-dimensional optimization problem.The input is resized to 50 × 50 to improve optimization efficiency.
V. EXPERIMENTS
The experiments compare the proposed method with six state-of-the-art enhancement methods across five public low-light image datasets under fixed implementation settings.
- The evaluation compares the proposed method with MSRCR, NPE, Dong, MF, LIME, and SRIE.
- The test suite includes hundreds of low-light images drawn from five public datasets: VV, LIME-data, NPE, DICM, and MEF.
- VV contains scenes combining correctly exposed and severely under- or over-exposed regions, testing selective enhancement without altering correctly exposed content.
- MEF provides high-quality multi-exposure sequences, from which one poorly exposed image is selected as the evaluation input.
- For fair comparison, all methods run in Matlab on Windows 10 with fixed parameters, 64G RAM, and a 3.4GHz CPU without GPU acceleration.The most time-consuming component is illumination-map optimization.
A. Lightness Distortion
Lightness distortion is evaluated with LOE, which measures changes in relative lightness ordering between input and enhanced images; the proposed method performs best across datasets.
- LOE objectively measures lightness distortion in enhanced results.
- LOE uses relative order difference RD to compare lightness ordering between the original and enhanced images.
- All images are down-sampled to a fixed 100 × 100 size before LOE computation because RD changes with pixel count.
- The proposed algorithm outperforms the other methods on every dataset in Table I.The authors interpret this as better preservation of image naturalness.
- The proposed results show the smallest visualized lightness distortion, while competing methods lose global or well-exposed-region lightness order.
B. Contrast Distortion
Contrast and visual-information distortion are evaluated against exposure-related references using DRIM and reverse-mode VIF; the proposed method outperforms competitors across datasets.
- DRIM visualizes contrast differences between enhanced results and exposure-related reference images without brightness-change interference.
- The proposed method produces the most realistic results with the least contrast distortion in the DRIM comparison.
- VIF compares mutual information between the reference and distorted images with information extracted by the human visual system.
- Reverse-mode VIF evaluates visual-information distortion by treating the enhanced image as reference and the original low-light image as degraded.
- The proposed algorithm outperforms the other methods across all datasets in Table II, indicating better preservation of visual information.
D. Time Cost
The method achieves the smallest reported distortion among compared methods with acceptable time cost, while visual comparisons reveal remaining enhancement failures and limitations.
- Our method achieves the smallest distortion among the compared methods with an acceptable time cost.SRIE and NPE produce small distortion but are described as quite time-consuming.
- MSRCR may produce whitish results, halo artifacts, noise, and detail loss in dark or bright regions.
- LIME can saturate bright areas and amplify noise in dark regions, requiring denoising for better results.
- The method can over-enhance black hair when illumination estimation mistakes it for the dark background.The paper attributes this failure to existing illumination-map estimation techniques and suggests semantic understanding as future work.
- Using only two images leaves the over-exposure problem unresolved; smaller-exposure images should be considered in future work.
APPENDIX DERIVATION OF CAMERA RESPONSE MODEL
The appendix derives the camera response function by solving a comparametric equation, applying logarithmic and differential transformations, and integrating under parameter constraints.
- The camera response function is derived from the comparametric equation by substituting the power model g = βf^γ into the exposure relation.
- Taking logarithms of both sides and differentiating with respect to E transforms the equation into a form suitable for solving f.
- A power-function derivative, f′(kE), provides a solution form for the transformed equation.
- Integration yields separate response-function forms for γ ≠ 1 and γ = 1, with parameter a defined as log_k γ.
- The constants are determined using the constraints f(1) = 1 and f(k) = β, producing the final model parameters a and b.