Source-linked AI summary
Image Correction via Deep Reciprocating HDR Transformation
Xin Yang, Ke Xu, Yibing Song, Qiang Zhang, Xiaopeng Wei, Rynson Lau
TL;DR
Existing image correction methods struggle to recover details in under- and overexposed LDR regions. DRHT reconstructs details in HDR with one CNN and maps them back to LDR with a second; the authors report favorable performance against state-of-the-art methods on standard benchmarks.
Problem
Pixel-focused correction methods struggle to recover details in under- and overexposed regions, where LDR values diminish information and limit recovery.
Source: §1 · pp. 1–2Method
DRHT uses an end-to-end framework with one CNN reconstructing missing details in HDR and another transferring them back to LDR.
Source: §1 · p. 2Results
DRHT performs favorably against state-of-the-art image correction methods on standard benchmarks.
Source: §1 · p. 2Takeaways & Limitations
The paper formulates image correction as HDR mapping, recovering details in HDR before transferring the reconstructed information back to LDR.
Source: §5 · p. 8DRHT recovers limited detail around large overexposed sunshine, which is rare in its training dataset.
Source: §4.4 · p. 8
Abstract
from arXiv · showhide
Image correction aims to adjust an input image into a visually pleasing one. Existing approaches are proposed mainly from the perspective of image pixel manipulation. They are not effective to recover the details in the under/over exposed regions. In this paper, we revisit the image formation procedure and notice that the missing details in these regions exist in the corresponding high dynamic range (HDR) data. These details are well perceived by the human eyes but diminished in the low dynamic range (LDR) domain because of the tone mapping process. Therefore, we formulate the image correction task as an HDR transformation process and propose a novel approach called Deep Reciprocating HDR Transformation (DRHT). Given an input LDR image, we first reconstruct the missing details in the HDR domain. We then perform tone mapping on the predicted HDR data to generate the output LDR image with the recovered details. To this end, we propose a united framework consisting of two CNNs for HDR reconstruction and tone mapping. They are integrated end-to-end for joint training and prediction. Experiments on the standard benchmarks demonstrate that the proposed method performs favorably against state-of-the-art image correction methods.
1. Introduction
Existing pixel-focused correction methods struggle to recover details in under- and overexposed regions, where the LDR image has diminished information. DRHT instead reconstructs details in HDR and transfers them back to LDR using an end-to-end two-network framework.
1. Introduction
- Prior correction methods balance pixels through histogram equalization, edge-preserving filters, and CNN encoder-decoders [41], often as preprocessing for vision tasks.The passage lists optical flow, decolorization, deblurring, face stylization, and visual tracking as examples of downstream tasks.Source: §1 · p. 1
- Existing methods struggle with under- and overexposed regions because pixel values near 0 or 255 diminish details, making recovery ill-posed.The paper frames recovering those details during correction as an open question.Source: §1 · pp. 1–2
- Camera response functions convert a natural scene into an LDR image using an exposure duration that photographers must choose.The paper notes that inappropriate exposure cannot be corrected by existing CRFs or the resulting LDR image.Source: §1 · p. 2
- DRHT uses one CNN to reconstruct missing details in HDR and a second to transfer them back to LDR for image correction.The two networks are integrated end-to-end.Source: §1 · p. 2
- The authors report extensive standard-benchmark evaluations comparing DRHT with state-of-the-art image correction methods.The supplied introduction states the evaluation scope but gives no individual scores.Source: §1 · p. 2
2. Related Work
Related work includes restoration and filtering, pixel manipulation, and image enhancement, but these approaches remain constrained by LDR inputs and operations. As a result, they cannot completely recover missing details in bright and dark regions while maintaining global illumination.
Image Restoration and Filtering
- Restoration methods reduce noise but cannot recover missing details, while edge-aware filters smooth overexposed regions and leave their details difficult to recover.Filtering also addresses local structures, repeated textures, and high-contrast details.Source: §2 · p. 2
Image Manipulation
- Pixel-manipulation work includes color enhancement and style mimicry, but mainly adjusts LDR images to user preferences.The passage also describes tone transfer and learned photographer-style tone adjustments.Source: §2 · p. 2
Image Enhancement
- Enhancement approaches include histogram and contrast adjustments, region-specific operators, reflectance or illumination estimation, and learned low-light mapping.The cited learned low-light method can recover details but tends to raise global illumination and produce surrealistic images [32].Source: §2 · p. 2
Related Work
- The reviewed methods cannot completely recover bright- and dark-region details because LDR inputs and operations lack sufficient information while preserving global illumination.This limitation applies across the discussed restoration, manipulation, and enhancement approaches.Source: §2 · p. 2
3. Deep Reciprocating HDR Transformation
DRHT reframes image correction as a two-stage transformation: reconstruct lost details in HDR, then map them back to LDR. Its framework combines HDR estimation and LDR correction networks for end-to-end training and prediction.
3.1. Image Correction Reformulation
- Under/overexposure loses scene details during tone mapping; DRHT uses f1 to reconstruct HDR details, then f2 maps them back to corrected LDR.The reformulation traces correction back to image formation.Source: §3.1 · p. 3, Figure 2 · p. 3
3.2. HDR Estimation Network
- The HDR estimator uses a fully convolutional encoder-decoder with corresponding skip connections and a direct input-to-output connection.The skip connections enrich local detail during coarse-to-fine decoding; the direct connection lets the network predict the input-output difference. It uses MSE and is pretrained before integration with the remaining modules.Source: §3.2 · p. 3
3.3. LDR Correction Network
- The LDR correction network applies inverse gamma mapping and a logarithm to reconstructed HDR before producing the corrected LDR image.The logarithm compresses the HDR range for convergence while retaining a relatively large intensity range, and δ removes zero values. The loss balances LDR correction against HDR reconstruction accuracy, with hierarchical supervision added coarse-to-fine.Source: §3.3 · p. 4
3.4. Implementation Details
- Training takes about ten days, and testing takes about 0.05s per 256×512 image.The reported setup uses an NVIDIA GTX 1080 GPU.Source: §3.4 · p. 4
- 61.51 average Q score and 0.9324 SSIM index are reported for reconstructed HDR versus ground truth on the test set.The comparison uses HDR-VDP-2.Source: Figure 3 · p. 4
4. Experiments
Experiments evaluate DRHT on city-scene and Sun360 data, analyze HDR reconstruction, and compare corrected images with existing methods. DRHT performs favorably in these comparisons, with a stated limitation under significant illumination contrast.
4.1. Experiments Setups
- Training uses city-scene LDR/HDR pairs, PBRT-generated patches, and Sun360 end-to-end finetuning, with evaluation on city-scene and Sun360 test images.The city-scene set supplies ground-truth HDR, while Sun360 does not.Source: §4.1 · pp. 4–5
4.2. Internal Analysis
- The HDR estimation network achieves HDR-VDP-2 Q 61.51 and SSIM 0.9324 on the city-scene test set, indicating synthesized HDR data close to ground truth.The analysis identifies high illumination contrast at a sun-building boundary as a difficult region.Source: §4.2 · p. 6
4.3. Comparison with State-of-the-arts
- DRHT performs favorably against existing correction methods on standard benchmarks and visually recovers details in overexposed and low-light scenes while maintaining global illumination.The quantitative comparison uses PSNR, SSIM, FSIM, and Q scores.Source: Table 1 · p. 7, §4.3 · p. 7
- On Sun360, DRHT performs favorably against DrTMO and Hdrcnn [6], with visual comparisons and quantitative evaluation supporting the comparison.The paper attributes DRHT's advantage to residual learning that targets missing details rather than reconstructing the whole HDR image.Source: Table 2 · p. 7, §4.3 · p. 8, Figure 6 · p. 8
4.4. Limitation Analysis
- DRHT has difficulty recovering details under significant illumination contrast, particularly around large overexposed sunshine that is rare in the training data.The paper proposes augmenting training with such extreme cases.Source: §4.4 · p. 8
5. Conclusion
The paper reformulates under/overexposed image correction as an HDR mapping problem and proposes DRHT, which reconstructs details in HDR before transferring them back to LDR. It reports state-of-the-art correction performance on two benchmarks, while noting limits in overexposed sunshine regions.
5. Conclusion
- DRHT recovers missing details in underexposed regions but has limitations in overexposed sunshine regions.Source: Figure 7 · p. 8
- DRHT achieves state-of-the-art correction performance on two benchmarks by reconstructing missing details in HDR and transferring them back to LDR.The HDR estimation and LDR correction networks are formulated end-to-end.Source: §5 · p. 8
- The method reformulates image correction as an HDR mapping problem to recover details that existing LDR methods cannot completely recover.Source: §5 · p. 8