PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Chain HDRI: Reconstructing a High Dynamic Range Image from a Single Low Dynamic Range Image

Siyeong Lee, Gwon Hwan An, Suk-Ju Kang

arXiv:1801.06277v1cs.CVeess.IV

TL;DR

A single LDR image may lack scene information needed to recover physical brightness, and direct mapping to HDR is difficult. The paper uses a deep network to generate an exposure stack from that image and reconstruct HDR. The authors report results closer to ground truth than previous methods, while noting that learning relatively low exposures is difficult.

  • Problem

    A single LDR image loses information in saturated and dark regions, and directly mapping it to HDR is difficult.

    Source: §1 · p. 1, §3 · p. 3

  • Method

    A deep neural network generates a sequential stack of different-exposure LDR images from one input and merges inference results into HDR.

    Source: §1 · pp. 1–2

  • Results

    The proposed network generates HDR images closer to ground truth than previously proposed methods.

    Source: §7 · p. 8, §6.2 · p. 8

  • Takeaways & Limitations

    The network is reported to address ghosting and tearing and can be extended to a wider dynamic range.

    Source: §7 · p. 8

    Learning relatively low exposures is difficult.

    Source: §7 · p. 8

Abstract

from arXiv · show

In this paper, we propose a novel deep neural network model that reconstructs a high dynamic range (HDR) image from a single low dynamic range (LDR) image. The proposed model is based on a convolutional neural network composed of dilated convolutional layers, and infers LDR images with various exposures and illumination from a single LDR image of the same scene. Then, the final HDR image can be formed by merging these inference results. It is relatively easy for the proposed method to find the mapping between the LDR and an HDR with a different bit depth because of the chaining structure inferring the relationship between the LDR images with brighter (or darker) exposures from a given LDR image. The method not only extends the range, but also has the advantage of restoring the light information of the actual physical world. For the HDR images obtained by the proposed method, the HDR-VDP2 Q score, which is the most popular evaluation metric for HDR images, was 56.36 for a display with a 1920$\times$1200 resolution, which is an improvement of 6 compared with the scores of conventional algorithms. In addition, when comparing the peak signal-to-noise ratio values for tone mapped HDR images generated by the proposed and conventional algorithms, the average value obtained by the proposed algorithm is 30.86 dB, which is 10 dB higher than those obtained by the conventional algorithms.

1. Introduction

Recovering physical scene brightness from a single LDR image is difficult because the image has lost information and conventional inverse tone mapping adjusts brightness without inferring physical brightness. The paper proposes a deep network that generates multiple exposure images from one LDR input to reconstruct HDR.

1. Introduction

  • Conventional inverse tone mapping adjusts brightness in selected regions rather than inferring physical brightness, while widening the range makes the LDR-to-HDR relationship difficult to learn.Recovering lost information is especially challenging when the original scene cannot be recaptured.Source: §1 · p. 1
  • The proposed network expresses exposure-dependent image changes to produce an HDR image close to the real world from a single LDR image.It forms a multiple-exposure image stack from the input.Source: §1 · p. 2
  • A chain structure sequentially generates different-exposure LDR images from the input, treated as a middle-exposure image, and an additional loss addresses vanishing gradients.The chain learns relationships between neighboring exposure levels.Source: §1 · p. 2
  • Minus PReLU (MPReLU) transforms PReLU so the network can learn residuals between the input and darker-exposure images more easily.Source: §1 · p. 2

2. Related work

The related work reviews HDR capture, inverse tone mapping, and CNN approaches, emphasizing the difficulty of recovering physical scene information from limited-range images.

2.1. High dynamic range imaging (HDRI)

  • Traditional HDR imaging estimates a camera response from differently exposed LDR images, but simultaneous capture is difficult and motion or illumination changes can degrade merged HDR images.Source: §2.1 · p. 2

2.2. Inverse tone mapping (ITM)

  • Inverse tone-mapping methods broaden LDR ranges through brightness or contrast adjustments, but can require input-specific parameters, create contour artifacts, or fail to match actual brightness.Saturated and dark regions lack information, making their restoration difficult.Source: §2.2 · p. 2

2.3. Convolutional neural network (CNN)

  • CNN restoration methods learn mappings from data, with skip connections and residual blocks supporting deeper networks.Source: §2.3 · p. 3

3. Difficulties of direct LDR-to-HDR mapping

Direct LDR-to-HDR mapping is difficult because wider ranges lack mapping information, and absent exposure metadata makes scene luminance ambiguous. The proposed approach instead infers a stack of differently exposed LDR images from a single middle-exposure input.

3. Difficulties of direct LDR-to-HDR mapping

  • The proposed method infers a multiple-exposure LDR image stack from one middle-exposure LDR image instead of mapping directly to HDR.The inferred images span ±1, ±2, and ±3 EV.Source: §3 · p. 3, Figure 2 · p. 3
  • The assumed middle exposure is treated as EV 0, with brighter and darker exposure images inferred from it.Source: §3 · p. 3
  • Wider dynamic ranges leave insufficient mapping information, while missing input metadata can make different scenes share pixel values and prevent accurate HDR-luminance estimation.The method assumes the input is a properly exposed middle-exposure image.Source: Figure 1 · p. 3, §3 · p. 3

4. Proposed Deep Chain HDRI

Deep Chain HDRI sequentially infers differently exposed LDR images from a middle-exposure input, then merges them into an HDR image. Its method combines exposure-dependent network depth, specialized subnetworks, staged training, a purpose-built exposure-stack dataset, and HDRI reconstruction.

4.1. Overall network architecture

  • Six subnetworks sequentially infer the three brighter and three darker exposure images, using deeper structures for exposures farther from the middle input.The resulting stack is used to generate the HDR image.Source: §4.1 · p. 3, Figure 3 · p. 4

4.2. Subnetwork architecture

  • Each CNN subnetwork maps a 64 × 64 patch from EV i to EV j using feature-extraction and reconstruction blocks; PReLU handles brighter outputs and MPReLU darker ones.The feature-extraction blocks use dilations 1, 1, 2, 3, 5, 8, and 13, giving a receptive field of 67.Source: §4.2 · pp. 3–4

4.3. Training

  • Training minimizes pixel L1 loss plus total-variation regularization, while intermediate EV ±1 and ±2 errors help prevent vanishing gradients during EV ±3 learning.The regularization weight is λ = 0.001; training also uses batch normalization.Source: §4.3 · pp. 4–5

4.4. Dataset

  • The authors created 96 exposure stacks (672 images) because only five existing HDR-dataset stacks met the required EV 0, ±1, ±2, and ±3 conditions.The middle-exposure image is selected by the most evenly distributed grayscale histogram in the stack.Source: §4.4 · p. 5

4.5. HDRI reconstruction

  • Reconstruction repeatedly infers patches at successive EVs to build the LDR stack, then merges that stack into HDR using Debevec et al.’s method.The input image is sliced into 64 × 64 patches with stride 1.Source: §4.5 · p. 5

5. Understanding the properties of the proposed method

The section motivates sequential exposure inference for larger EV differences and introduces MPReLU for learning darker-exposure residuals. It evaluates the chain structure and activation choices.

5.1. Why need the deep chain structure?

  • As EV difference grows, images become less similar, motivating a deeper structure that infers exposure relationships sequentially.The authors calculate average PSNR between the middle exposure and images at various EVs.Source: §5.1 · p. 6, Table 1 · p. 6
  • The chain structure infers EV ±3 images sequentially through EV ±1 and ±2, and the authors report greater accuracy than a shallow network for EV 0 to EV +3.Intermediate losses at EV ±1 and ±2 are added to address gradient vanishing as network depth increases.Source: §5.1 · p. 6, Table 2 · p. 6

5.2. Residual learning and activation function

  • MPReLU extends PReLU to learn residuals between the input and darker-exposure images; the network uses PReLU for brighter images and MPReLU for darker ones.The authors motivate the design by noting that PReLU’s negative-domain slope is relatively small and error does not flow easily there.Source: §5.2 · p. 6

6. Experimental results

The experiments compare inferred exposure stacks and HDR images with ground truth using 48 image stacks. The proposed method is reported to approximate scene luminance better than conventional methods.

6. Experimental results

  • The experiments compare inferred results with conventional ITM algorithms using 48 image stacks and the HDR Toolbox.The toolbox generates and tone-maps HDR images for comparison.Source: §6 · p. 6

6.1. Comparison between the ground truth LDR image stack and inferred LDR image stack

  • Similarity between inferred and ground-truth LDR images decreases as exposure difference from EV 0 increases, and darker exposures are less similar than brighter ones.The LDR stack comparison uses PSNR, SSIM, and MS-SSIM; Figure 6 also compares exposure trends and light-source properties.Source: §6.1 · p. 7, Table 4 · p. 7, Figure 6 · p. 7

6.2. Comparison between ground truth HDR images and inferred HDR images

  • The proposed network inferred actual scene luminance from a single LDR image better than conventional methods, whose HDR results can show brightness boosting and detail loss.The comparison evaluates tone-mapped HDR PSNR and HDR-VDP-2 and DRIM under specified display and viewing conditions.Source: §6.2 · pp. 7–8, Table 5 · p. 8

7. Conclusion

The proposed network infers HDR images from single LDR images by training on image-pair residuals and arranging exposure levels sequentially. The conclusion notes both potential benefits and remaining difficulty with low exposures.

7. Conclusion

  • The network is reported to solve ghosting and tearing found in conventional HDRI, and patch-based learning makes HDR restoration less restricted by image resolution.The network is described as scalable to a wider dynamic range.Source: §7 · p. 8
  • Learning relatively low exposures remains difficult, and the authors identify network-structure improvement and further study as future work.The proposed subnetworks are described as brightness dictionaries arranged sequentially in exposure space.Source: §7 · p. 8

Related papers

Loading 1801.06277v1…