PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Learning for HDR Imaging: State-of-the-Art and Future Trends

Lin Wang, Kuk-Jin Yoon

arXiv:2110.10394v3eess.IVcs.CVcs.LG

TL;DR

HDR imaging represents a wide range of real-scene intensities, but conventional HDR cameras are prohibitively expensive for general users. This survey organizes deep HDR methods and reviews their technical landscape. It reports that deep-learning methods better address artifacts from large foreground motion and recover content in saturated regions, while requiring greater computational expense and diverse exposure information.

  • Problem

    Conventional HDR cameras are prohibitively expensive for general users, motivating HDR reconstruction from LDR images.

    Source: §1 · p. 1

  • Method

    The paper surveys deep HDR imaging, organizes methods into a hierarchical taxonomy, and reviews technical details, applications, and challenges.

    Source: §9 · p. 17

  • Results

    Deep-learning methods better address artifacts from large foreground motions and can generate content in saturated and occluded regions than prior alignment methods.

    Source: §8 · p. 16

  • Takeaways & Limitations

    The review connects approaches to identify active research fields and potential new problems and methods.

    Source: §9 · p. 17

    Unsupervised HDR reconstruction quality is worse than supervised methods, and no unsupervised method addresses single-exposure HDR imaging.

    Source: §6.1 · p. 13

Abstract

from arXiv · show

High dynamic range (HDR) imaging is a technique that allows an extensive dynamic range of exposures, which is important in image processing, computer graphics, and computer vision. In recent years, there has been a significant advancement in HDR imaging using deep learning (DL). This study conducts a comprehensive and insightful survey and analysis of recent developments in deep HDR imaging methodologies. We hierarchically and structurally group existing deep HDR imaging methods into five categories based on (1) number/domain of input exposures, (2) number of learning tasks, (3) novel sensor data, (4) novel learning strategies, and (5) applications. Importantly, we provide a constructive discussion on each category regarding its potential and challenges. Moreover, we review some crucial aspects of deep HDR imaging, such as datasets and evaluation metrics. Finally, we highlight some open problems and point out future research directions.

1 INTRODUCTION

HDR imaging represents scene intensities across a wider exposure range than traditional imaging, while conventional HDR cameras are prohibitively expensive for general users. This survey organizes deep HDR imaging methods across five aspects and reviews their opportunities, challenges, and open problems.

1 INTRODUCTION

  • Conventional HDR cameras are prohibitively expensive for general users, while virtual-environment HDR content is mostly explored in entertainment, including gaming and VR.HDR imaging aims to represent real-scene intensity levels from sunlight to shadows.Source: §1 · p. 1
  • Deep HDR imaging methods reconstruct HDR content by fusing multiple LDR exposures or by mapping a single-exposure image.Capturing different LDR exposures can require software or hardware technology, making dataset creation difficult.Source: §1 · p. 1
  • The survey organizes deep HDR imaging methods by input exposures, learning tasks, sensors, learning strategies, and applications.It also reviews problem definitions, datasets, evaluation metrics, opportunities, challenges, and future directions.Source: §1 · p. 1
  • Unlike earlier surveys focused on conventional HDR algorithms, this study comprehensively examines recent deep HDR imaging advances.Earlier survey topics included deghosting, multiple-exposure fusion, and stack-based reconstruction.Source: §1 · p. 1
  • The paper proceeds from definitions, datasets, and metrics to methods, joint tasks, novel sensors, learning paradigms, applications, and future directions.Its taxonomy is presented as hierarchical and structural.Source: §1 · p. 1, Figure 1 · p. 2

2 PROBLEM, DATASETS, AND EVALUATION

Deep HDR imaging learns to reconstruct HDR content from one or more LDR images, with different pipelines and limits depending on exposure information. The section surveys dataset construction and image-quality assessment, including the measures used in the first HDR imaging challenge.

2.1 Definition of Deep HDR Imaging

  • Deep HDR imaging learns a mapping from one or more LDR images to an HDR image, with multi-exposure reconstruction aligning and merging inputs and single-exposure reconstruction learning inverse tone mapping.Single-exposure reconstruction is ill-posed because the input conveys limited exposure information, and its reconstructed dynamic range is limited.Source: §2.1 · p. 2, Figure 2 · p. 2

2.2 Datasets for Deep HDR Imaging

  • Benchmark datasets vary in exposure levels, size, resolution, quality, and scene diversity, and may use simulated LDR formation or real-world LDR images with HDR ground truth.Datasets are also developed for multi-exposure fusion, video HDR, and novel sensors; Table 1 summarizes dataset size, type, ground-truth availability, resolution, and scene details.Source: §2.2 · pp. 2–3, Table 1 · p. 3

2.3 HDR Image Quality Assessment (IQA)

  • HDR image quality assessment includes subjective and objective approaches, with statistical and HDR-oriented measures commonly used.The section introduces these measures for evaluating reconstructed HDR images.Source: §2.3 · p. 3

2.3.1 Difference Measures and Statistical-Oriented Metrics

  • PSNR and MSE assess reconstructed HDR against ground truth before and after µ-law tone mapping.PSNR measures reconstruction differences; MSE averages squared pixel-intensity differences.Source: §2.3.1 · p. 3

2.3.2 Structural Similarity Measures

  • SSIM assesses structural differences using image statistics, and MS-SSIM and MEF-SSIM are additional structural-similarity measures.HDR-image SSIM is computed before and after µ-law tone mapping.Source: §2.3.2 · p. 3

2.3.3 Tailored HDR Statistical Measures

  • HDR-VDP-2 predicts visibility and quality by comparing reconstructed HDR images with ground truth, while HDR-VQM evaluates HDR video quality.Source: §2.3.3 · p. 3

2.3.4 Deep Learning–Based Evaluation Metrics

  • Deep-learning-based HDR quality metrics include full-reference methods that assess perceptual quality and non-reference methods that quantify extracted features.The non-reference methods described measure low- and high-level features from DNNs.Source: §2.3.4 · p. 3

2.3.5 HDR Imaging Challenge

  • The first NTIRE HDR imaging challenge evaluated reconstruction from single-exposure or multiple-exposure LDR inputs using PSNR against ground truth.It used existing and newly curated HDR datasets.Source: §2.3.5 · p. 3

3 SUPERVISED HDR RECONSTRUCTION

The section reviews supervised HDR reconstruction from multiple LDR exposures and from a single LDR image, organizing methods around alignment, fusion, learned image formation, and sensor design. It highlights persistent challenges in recovering detail, controlling artifacts, obtaining training data, and balancing computational cost.

3.1.1 Optical Flow-based Image Alignment

  • Dynamic multi-exposure reconstruction aligns LDR inputs before merging them, with optical-flow methods aligning low- and high-exposure images to a medium-exposure reference.Kalantari et al. then use a DNN supervised by ground-truth HDR images; later work uses optical-flow networks such as FlowNet.Source: §3.1 · p. 3, §3.1.1 · p. 4, Figure 3 · p. 4

3.1.2 Direct Feature Concatenation

  • Feature-concatenation methods learn aligned features for HDR reconstruction, addressing optical flow’s difficulty with large foreground motion.Some frameworks add cycle reconstruction of static LDR images or attention masks to constrain restoration or reduce ghosting; multiscale encoders fuse features at multiple scales.Source: §3.1.2 · p. 4

3.1.3 Correlation-Guided Feature Alignment

  • Correlation-guided methods refine LDR features to exclude misaligned or saturated information before merging, using attention or multiscale alignment.Approaches place correlation in different stages, including encoding and merging; over-saturated regions can make feature exclusion discard texture, and the best stage remains unsettled.Source: §3.1.3 · p. 4, §3.1.6 · p. 6

3.1.4 Image Translation-Based Alignment

  • Image-translation methods generate reference-like LDR images from other exposures before merging, with later work training alignment and merging end to end.Earlier optical-flow and feature-concatenation approaches can produce artifacts or less realistic details in highly saturated regions.Source: §3.1.4 · pp. 4–5

3.1.5 Deep Static Exposure Fusion

  • For static scenes, deep exposure fusion targets artifacts and lost texture when merging LDR images at different exposures.DeepFuse uses an unsupervised framework and no-reference quality loss; later methods include weight-map fusion, GAN-based fusion, and weighted feature-similarity losses without ground-truth images.Source: §3.1.5 · p. 5

3.1.6 Potential and Challenges

  • Multi-exposure methods face costly correlation learning, limited labeled capture data, and inference latency, while alignment and saturated-region detail remain unresolved challenges.Correlation-guided alignment can be sensitive to over-saturation, and further theoretical work is needed to determine where correlations should be learned.Source: §3.1.6 · pp. 5–6

3.2.1 Direct Learning from a Single LDR Image; 3.2.2 Generating Bracketed LDR Image Stacks

  • Single-exposure methods reconstruct HDR either directly from one LDR image or by generating bracketed LDR stacks, addressing an ill-posed task with limited quality training data.Direct methods use encoder–decoder or multiscale features; masking saturation and perceptual, adversarial, or cosine-similarity losses address reconstruction artifacts and detail. Stack-generation methods synthesize up- and down-exposed LDR images, often using a camera-response-function database, before reconstruction.Source: §3.2.1 · p. 6, §3.2.2 · p. 6

3.2.3 Computationally Efficient Learning; 3.2.4 Learning Camera Imaging Pipeline; 3.2.5 Learning Neural Sensors; 3.2.6 Potential and Challenges

  • Single-exposure work also explores recurrent efficiency, learned camera-imaging pipelines, and neural sensors, but sensor approaches require more complex camera settings and hardware.Feedback can guide low-level features with high-level features over iterations, while a learned imaging pipeline reverses clipping, response mapping, and quantization. Sensor methods model optical or exposure encoding; broader challenges include saturated-pixel recovery and losses that can yield less realistic results.Source: §3.2.3 · pp. 6–7, §3.2.4 · p. 7, §3.2.5 · p. 7

3.3 Deep 3D Panorama HDR imaging

The review surveys deep HDR methods for 3D panoramas, stereo images, and video, alongside loss strategies and representative method evaluations. It highlights data, alignment, and computational challenges that constrain these approaches.

3.3 Deep 3D Panorama HDR imaging

  • 3D panorama HDR methods learn spatially varying lighting from LDR panoramas, but real-world labeled datasets are scarce and current approaches often rely on synthetic data and multistep parametric models.The review suggests exploring knowledge transfer from 2D HDR imaging.Source: §3.3 · p. 8

3.4 Deep Stereo-Based HDR Imaging

  • Stereo HDR methods use either a multistep conventional pipeline or learned modules; the GAN approach transfers exposure views and fuses them, while another method replaces conventional steps with learned networks.The review notes that the latter increases computational cost and requires end-to-end training of multiple modules; exposure calibration and texture filling remain challenging.Source: §3.4 · p. 8

3.5 Deep Video HDR

  • Deep video HDR receives less attention than image HDR, in part because it must maintain temporal consistency while aligning frames with different exposures.The review identifies frame alignment as a challenge in this setting.Source: §3.5 · p. 8

3.5.1 HDR Videos with Single Alternating Exposure

  • For single alternating-exposure video, methods align neighboring frames to a reference before merging, or use adversarial generation with a 3D convolutional autoencoder to address flicker.One method adds intrinsic loss to maintain output luminance and color; related HDR-plus-SR methods discussed here do not consider temporal consistency.Source: §3.5.1 · p. 9

3.5.2 HDR Videos with Multiple Alternating Exposures

  • For multiple alternating exposures, reviewed methods use coarse-to-fine alignment and fusion or tri-exposure quad-Bayer sensors with feature fusion, temporal denoising, and super-resolution.The first approach uses optical flow in coarse reconstruction and deformable convolutions in refinement; the second uses attention-based temporal denoising.Source: §3.5.2 · p. 9

3.5.3 Discussion and Challenges

  • Video HDR remains rare; optical-flow alignment can err in saturated regions, multiexposure methods require many modules, and synthetic datasets have a considerable domain gap from real-world data.The review points to self-supervised and transfer learning, and temporal-consistency losses, as research directions.Source: §3.5.3 · p. 9

3.6 Optimization and Loss Functions

  • The review analyzes optimization and loss strategies and reports experiments validating commonly used loss functions, with details and results deferred to supplementary material.Tables 2 and 3 summarize commonly used loss functions.Source: §3.6 · p. 9

3.7 Examining SoTA deep HDR imaging methods

  • The paper reports representative multi-exposure and single-image HDR comparisons, but the experiments are deferred to supplementary material and statistics come from original papers or official models.Single-image methods are compared on two benchmark datasets; the supplied passage gives no comparison outcomes.Source: §3.7 · p. 9

4 DEEP HDR IMAGING WITH OTHER TASKS

This section reviews approaches that combine HDR reconstruction with super-resolution, denoising, or deblurring. It contrasts sequential and joint learning and identifies limits in noise modeling and sensing scope.

4.1 Deep HDR Imaging with Super-Resolution (SR)

  • HDR with super-resolution seeks to recover high-frequency detail, contrast, and signal amplitude while converting plentiful LR LDR images into HR HDR images.The task combines LDR-to-HDR reconstruction with LR-to-HR mapping, both described as ill-posed or complex.Source: §4.1 · p. 9

4.1.1 Sequential Learning

  • Sequential HDR-SR methods choose whether to apply super-resolution or HDR mapping first; a reviewed study found sequential learning causes error accumulation.One method reports its best results when enhancing luminance before feeding it to an SR network trained on luminance.Source: §4.1.1 · p. 10

4.1.2 Joint Learning

  • Joint single-exposure methods combine LDR super-resolution, LR HDR reconstruction, and HDR super-resolution, with variants adding base/detail branches or a GAN reconstruction module.Multi-exposure work is less common: reviewed approaches align and fuse LR inputs before upsampling, or recursively learn under- and over-exposed information in separate branches.Source: §4.1.2 · p. 10

4.2 Deep HDR Imaging with Denoising

  • Noise in under- and over-exposed LDR regions can degrade HDR reconstruction, so reviewed methods combine denoising with HDR imaging.The section also reviews noise removal across exposure settings and sensors.Source: §4.2 · p. 10

4.2.1 Single-Exposure Noise Removal

  • Single-exposure HDR denoising includes Noise2Noise without clean data, RNN burst denoising that handles low-light noise, and a spatially dynamic denoising and dequantization network.The latter won second place in the NITRE2021 single-exposure HDR imaging challenge.Source: §4.2.1 · p. 10

4.2.2 Multi-Exposure Noise Removal

  • Multi-exposure denoising methods merge exposure-specific branches or process dual-exposure video, but clean ground-truth HDR videos are lacking and blur and noise may be generated manually.The review notes that generic denoising can lose HDR detail and that real-world LDR noise is difficult to model.Source: §4.2.2 · pp. 10–11

4.3 Deep HDR Imaging with Deblurring

  • Long exposures in extreme imaging conditions can blur LDR inputs, prompting HDR methods that model sensing with learnable functions or jointly reconstruct from blurred images.The reviewed sensing-modeling methods are limited to single-exposure HDR; extending them to dynamic multi-exposure scenes may be more challenging.Source: §4.3 · p. 11

5 DEEP HDR IMAGING WITH NOVEL SENSORS

The review covers HDR reconstruction using event, thermal, and modulo cameras, describing both sensor-specific methods and image-fusion approaches. It highlights challenges including latency, limited data, and recovering color or detail.

5.1 Deep HDR Imaging with Neuromorphic Cameras

  • Neuromorphic cameras capture intensity changes asynchronously and offer higher dynamic range than frame-based cameras, motivating HDR reconstruction from event data.The review gives a 140 dB versus 60 dB comparison for DAVIS240C and conventional cameras.Source: §5.1 · p. 11

5.1.1 Reconstruction-Based HDR Imaging

  • Reconstruction-based methods turn events into HDR intensity images or video, with extensions for low-light scenes, super-resolution, downstream tasks, and high-speed video.Event data can provide scene edges when APS frames are under-exposed, but event-camera resolution is low.Source: §5.1.1 · p. 11

5.1.2 Neuromorphic camera-Enhanced HDR Imaging

  • Event-guided methods combine events with LDR images to address sensor gaps, but reviewed approaches face alignment and exposure-scope limits and may rely on separately reconstructed intensity maps.The review describes joint event and image learning as a potential direction.Source: §5.1.2 · pp. 11–12

5.2 Deep HDR Imaging with Thermal Sensors

  • Thermal sensors provide grayscale contours and HDR capability in low light, supporting either IR-to-HDR generation or fusion with LDR images.The review describes the challenge of translating grayscale IR information into color HDR images.Source: §5.2 · p. 12, §5.2.1 · p. 12

5.2.1 Thermal to HDR Image Generation

  • IR-to-HDR generation methods include supervised color reconstruction, unsupervised grayscale reconstruction followed by colorization, and registration-based reconstruction that preserves thermal profiles.The reviewed approaches use different supervision and reconstruction strategies.Source: §5.2.1 · p. 12

5.2.2 Thermal and Image Fusion

  • IR-LDR fusion methods either stack the images as network inputs or fuse their features; one reviewed method combines image-content and VGG-feature fusion.IR contributes grayscale HDR content in a domain distinct from LDR images.Source: §5.2.2 · p. 12

5.3 Deep Modulo Camera-based HDR Imaging

  • Modulo cameras reset pixels when they reach the maximum value to avoid saturation; reviewed deep methods use neural priors or iteratively estimate binary rollover masks.The review says such deep methods remain limited.Source: §5.3 · pp. 12–13

5.4 Discussion and Challenges

  • IR-based GAN generation usually performs worse than fusion and adds inference latency, while grayscale inputs, scarce high-quality datasets, and exposure-insensitive feature fusion constrain reconstruction.The review notes that color information is not naturally encoded in IR and suggests filtering pixels according to LDR exposure.Source: §5.4 · p. 13

6 HDR WITH NOVEL LEARNING STRATEGIES

The review surveys unsupervised learning, GANs, transfer and meta-learning, attention, and reinforcement learning for HDR imaging. It discusses where these strategies have been applied and the limits or opportunities identified for each.

6.1 HDR Imaging with Unsupervised Learning

  • Unsupervised HDR work includes exposure fusion and multi-exposure LDR-to-HDR mapping, but single-exposure unsupervised reconstruction is absent and multi-exposure quality is reported below supervised methods.The review notes that direct LDR-to-HDR mapping methods are rare.Source: §6.1 · p. 13

6.2 HDR Imaging with GAN

  • GANs model HDR reconstruction with a generator and discriminator, and reviewed applications span single- and multi-exposure inputs.The review identifies training stability under diverse LDR distributions as a challenge; its multi-exposure examples use a single discriminator for the generated HDR image.Source: §6.2 · pp. 13–14

6.3 HDR Imaging with Transfer Learning

  • Transfer learning has been used to adapt classification features to limited HDR datasets, while the review proposes transferring knowledge across datasets and tasks.One cited study reports that ImageNet classification knowledge considerably improves single-exposure HDR reconstruction.Source: §6.3 · p. 14

6.4 HDR Imaging with Meta-Learning

  • Meta-learning targets scene-specific nonlinear mappings from LDR pixels to HDR radiance, but the reviewed method still has color-saturation artifacts linked to ineffective loss functions.The review suggests exploring meta-transfer and few-shot approaches.Source: §6.4 · p. 14

6.5 HDR Imaging with the Attention Mechanism

  • HDR attention methods guide multiexposure alignment or merging and are reported to yield fewer ghost artifacts than optical flow, but can require more parameters and computation.The review also notes limitations when LDR images have few sharp edges and textures.Source: §6.5 · p. 15

6.6 HDR Imaging with Deep Reinforcement Learning

  • A reviewed reinforcement-learning method selects local exposures for image subregions using a policy network and adversarial learning, but it is the only such method identified.Its regions come from image segmentation rather than exposure-level segmentation.Source: §6.6 · p. 15

7 APPLICATIONS

Deep HDR imaging has been applied to computer-vision tasks including recognition, detection, segmentation, stereo matching, and depth estimation, as well as graphics tasks such as illumination estimation. The reviewed methods use HDR information or HDR-related mappings to support these applications.

Applications

  • HDR imaging applications span computer-vision tasks and graphics illumination estimation.The surveyed vision tasks include recognition, segmentation, depth estimation, and visual odometry.Source: §7 · p. 15

Scene Recognition and Detection

  • Traffic-light recognition uses low-exposure frames to detect candidates and high-exposure frames for DNN classification.Other reviewed approaches map HDR images to LDR while preserving salient details for object detection, or jointly train exposure control and detection.Source: §7 · p. 15

Semantic Segmentation

  • HDR-based semantic segmentation includes training with LDR and HDR images and domain adaptation across synthetic and real-world datasets.A separate method classifies cloud types per pixel using features extracted from HDR sky-image patches.Source: §7 · p. 15

Stereo and Depth Estimation

  • Stereo matching methods use multi-exposure LDR pairs, while burst-based depth estimation can align LDR images for HDR reconstruction or fusion.Source: §7 · p. 15

Illumination Estimation

  • DNN methods estimate indoor or outdoor HDR illumination through image-to-illumination mapping, but require high-quality datasets with corresponding illumination conditions.Source: §7 · p. 15

8 DISCUSSION AND NEW PERSPECTIVES

The discussion contrasts deep HDR methods with prior approaches and identifies practical constraints and research directions spanning exposure selection, data, computation, learning, and sensors. It also highlights real-world generalization as an unresolved challenge.

Pros and Cons of DL-based Methods vs. the Prior Arts

  • Deep HDR methods better address large-motion artifacts and generate content in saturated or occluded regions, while single-image methods use contextual information to improve saturated-region results.The review also notes that deep methods are more computationally expensive and require diverse LDR exposure information.Source: §8 · p. 16

Exposure Bracket Selection for Deep HDR Imaging

  • Three exposures at -2/0/+2 EV usually handle most scenes, but highly overexposed scenes may need more brackets, increasing computation cost.Most deep HDR methods therefore consider three exposure brackets.Source: §8 · p. 16

Feature Representations and GNNs

  • Attention can improve LDR feature learning but adds computation; efficient attention, contrastive learning, privileged HDR features, and cross-scale graph aggregation are proposed directions.The graph approach is motivated by recurrence and correlation of local LDR patches for alignment or merging.Source: §8 · pp. 16–17

Data-efficient Learning

  • Large-scale labeled datasets are expensive and challenging to build for multi-exposure HDR reconstruction, motivating self-supervised, unsupervised, data-free transfer, and few-shot approaches.The passage also points to augmentation or adversarial examples as ways to learn from unlabeled LDR images.Source: §8 · p. 16

Computationally Efficient Learning

  • Deep multi-exposure reconstruction models are large because encoding and merging require many convolutions, motivating lightweight models, pruning, distillation, and efficient convolutional structures.Source: §8 · p. 16

Potential of Cross-Task Consistency

  • Future multitask HDR work may exploit consistency among predictions for the same image, while sequential or joint learning can incur high computational costs and dataset requirements.Joint HDR and super-resolution also requires high-resolution cameras to capture high-resolution scenes.Source: §8 · p. 16

Potential of Novel Sensors

  • Event and thermal sensors capture relatively high-HDR scenes, but sensor-based reconstruction methods need further study; end-to-end event-guided learning or feature fusion are potential directions.The reviewed event method's reconstructed intensity maps are not differentiable and cannot fully convey event HDR information.Source: §8 · p. 16

Toward Real-World HDR

  • Deep HDR performance can drop substantially from simulated datasets to complex real-world LDR scenes, where unexpected saturation and limited exposure levels pose challenges.Suggested directions include unsupervised domain adaptation and transferring knowledge from DNNs trained on varied LDR content.Source: §8 · p. 17

9 CONCLUSION

The conclusion synthesizes the review of deep HDR imaging methods and applications, including a formal problem definition and hierarchical taxonomy. It connects current approaches to challenges and possible future research directions.

Conclusion

  • The review covers technical details and applications of deep HDR imaging and formally defines its problem.Source: §9 · p. 17
  • Its hierarchical, structural taxonomy supports review of the technical status, potential, and challenges of each method category.Source: §9 · p. 17
  • The analysis discusses ways to address existing challenges and research gaps, with connections among approaches pointing to potential new research problems and methods.Source: §9 · p. 17

1 DETAILS OF DATASET FOR DEEP HDR IMAGING

Deep HDR datasets vary in exposure levels, size, resolution, quality, scene diversity, and ground-truth availability. The review describes synthetic and real-world dataset construction for single-image and multi-exposure HDR reconstruction.

Dataset Overview

  • Benchmark datasets differ in exposure levels, size, resolution, quality, scene diversity, and availability of HDR ground truth.Some simulate LDR formation with camera response functions, while others include real-world LDR images and HDR ground truth.Source: §1 · p. 22

Single-Image HDR Datasets

  • Synthetic single-image HDR datasets commonly generate LDR inputs using camera response functions or virtual-camera captures.One real-world dataset asked 600 amateurs to capture multiple exposures from a steady tripod, then fused the stacks into HDR images.Source: §1 · p. 22

Multi-Exposure HDR Datasets

  • A representative dynamic-scene dataset simulates motion between LDR exposures while capturing its HDR ground truth from static scenes at different exposure values.The HDR ground truth is aligned with the middle-exposure reference image.Source: §1 · p. 22

2 DETAILS OF OPTIMIZATION AND LOSS FUNC-

This section reviews the losses used to train deep HDR reconstruction models and tests their effects using FHDR. Pixelwise losses provide a baseline, while added loss terms improve reconstruction quality to varying degrees.

2.1 Loss functions

  • Pixelwise loss compares the µ-law tone-mapped reconstruction with the ground truth; µ sets compression, and direct linear-HDR error can underestimate differences.The tested losses use FHDR trained for 20 epochs, with PSNR and SSIM for evaluation.Source: §2.1 · p. 22
  • Beyond pixelwise matching, perceptual loss transfers classifier features, while adversarial, gradient, cycle, style, and cosine losses target image quality and reconstruction constraints.The described goals include perceptual similarity, sharpness, reduced ghosting, texture, and color stability.Source: §2.1 · pp. 22–23

2.2 Experimental validations

  • Pixelwise L1 and L2 losses are common and effective, but L2 scores slightly higher on PSNR and there is no absolute standard for choosing between them.Representative methods use both loss types.Source: §2.2 · p. 23
  • Adding loss terms to the pixel-wise loss improves HDR reconstruction: perceptual loss raises PSNR by around 1.2dB, cosine similarity loss by around 0.9dB, and gradient loss is 0.3dB less effective than perceptual loss.Source: §2.2 · p. 23

3 DISCUSSION AND CHALLENGES FOR HDR IMAGING WITH SR

The discussion identifies practical and data-related challenges in joint HDR imaging and super-resolution, alongside possible research directions.

3 DISCUSSION AND CHALLENGES FOR HDR IMAGING WITH SR

  • Joint learning is described as more efficient for HDR imaging with super-resolution, but most methods follow the single-exposure SISR paradigm.Capturing high-frequency textures while enhancing local contrast remains challenging, including for the cited GAN-based approach.Source: §3 · p. 23
  • Current methods use ideal bicubic degradation, whereas real-world low-resolution images have unknown degradations; future work could estimate degradation while enhancing local contrast.The authors suggest combining blind super-resolution methods with reference-based HDR imaging.Source: §3 · p. 23
  • All discussed state-of-the-art methods are supervised and require substantial labeled data, while high-quality datasets are limited.Few-shot learning and knowledge transfer are suggested as possible directions to reduce labeled-data needs.Source: §3 · p. 23

4 COMPARISON OF THE REPRESENTATIVE DEEP HDR IMAGING METHODS

The comparisons find advantages for deep methods over patch-based methods in multi-exposure HDR imaging and over non-deep methods in single-image HDR imaging. The reported results also distinguish strong approaches within each setting.

4.1.1 Experiment Settings and Datasets

  • Multi-exposure comparisons used Kalantar et al.'s 89-scene dataset with ground-truth HDR images and three metrics; some unavailable implementations were represented by reported results.PSNR and SSIM were evaluated in tone-mapped and linear domains, alongside HDR-VDP-2.Source: §4.1.1 · pp. 23–24

4.1.2 Experimental Analysis

  • Niu et al. outperforms the other multi-exposure methods on HDR-VDP-2, with an approximately 10-point difference from the state-of-the-art patch-based method.Patch-based methods miss details in saturated regions and show artifacts from misaligned exposures.Source: §4.1.2 · p. 24

4.2.1 Experiment Settings and Datasets

  • Single-image HDR methods were compared on the RAISE and HDR-EYE datasets using mean and standard deviation HDR-VDP-2 scores on linearly mapped images.The comparisons included non-deep methods as well as deep methods.Source: §4.2.1 · p. 25

4.2.2 Experimental results

  • Kim et al. gains 4 HDR-VDP-2 points over AEO, while Liu et al. achieves comparable performance to Kim et al.The authors attribute Kim et al.'s approach to end-to-end learning of LDR brackets and HDR synthesis, and Liu et al.'s to modeling LDR image formation.Source: §4.2.2 · p. 25

Related papers

Loading 2110.10394v3…