PaperlayerView exact revision on arXiv

Source-linked AI summary

Recurrent Dynamic Range Extension

Sebastian Dille, Keru Fu, S. Mahdi H. Miangoleh, Yağız Aksoy

arXiv:2609.13135v1eess.IVcs.CVcs.GR

TL;DR

Clipped highlights are difficult to reconstruct from a single image, and HDR training data are scarce. The paper trains bounded single-EV extensions on RAW images and applies the network recurrently with HDR supervision. It reports that the recurrent approach outperforms SI-HDR baselines and reconstructs diverse in-the-wild highlights.

  • Problem

    Clipped regions in LDR images lack bright-area information, while long-tailed radiance complicates direct HDR regression and HDR ground-truth datasets remain scarce.

    Source: §1 · p. 2

  • Method

    The method learns bounded single-EV extensions from diverse RAW images, then recurrently applies and trains the network with HDR supervision to recover wider highlight ranges.

    Source: §1 · p. 2

  • Results

    The recurrent method outperforms all baselines on SI-HDR, with strong perceptual alignment and gains on highlight-related metrics.

    Source: §4.1 · p. 6

  • Takeaways & Limitations

    The approach reconstructs diverse in-the-wild highlights, including skin tones, specular reflections, and light sources.

    Source: §5 · p. 10

    The approach assumes reasonably exposed midtones; large severely overexposed regions with limited semantic information can yield repetitive patterns or overestimated brightness.

    Source: §5 · p. 10

Abstract

from arXiv · show

We present an approach to progressively extend the highlights of an image. Instead of reconstructing the full dynamic range of a complex scene directly, we learn a simpler task first: We extend the dynamic range of an input image by a single exposure value. Once this is mastered, we retrieve the full HDR image for the scene by executing our network recurrently, progressively increasing the dynamic range of the input. Our formulation is agnostic to the input dynamic range and targets a bounded output domain. This enables us to use widely available RAW images for the reconstruction task and adapt adversarial losses to construct realistic images. By incorporating Memory Replay for backpropagation, we can train our network recurrently over multiple inference stages and reduce reconstruction errors. As a consequence, our system reconstructs challenging long-tailed HDR scenes robustly and shows powerful recovery of bright light sources and highlights.

1 Introduction

Clipped highlights make single-image HDR recovery difficult, while scarce, low-diversity HDR training data limits data-driven reconstruction. The paper learns bounded single-EV extensions from RAW images, then applies the network recurrently to recover wider highlight ranges.

1 Introduction

  • Clipped bright regions lose information, and long-tailed radiance makes direct HDR regression difficult to optimize without compression that can reduce physical accuracy.Large highlight errors can destabilize training; tone mapping compresses luminance but can obscure local differences and amplify radiance errors.Source: §1 · p. 2
  • HDR ground truth is scarce because multi-exposure capture requires static scenes, precise alignment, and tripod-mounted cameras, limiting training-data scale and diversity.Source: §1 · p. 2
  • A single-EV network extends linear clipped inputs from [0, 1] to [0, 2], independent of input exposure, keeping outputs bounded for linear-space training.RAW collections provide diverse training pairs, and multiple pairs can be derived from one re-exposed RAW photo.Source: §1 · p. 2
  • Repeated network application progressively restores the full long-tailed highlight distribution, while recurrent HDR training teaches later steps to compensate for earlier prediction errors.Source: §1 · p. 2
  • The method combines bounded RAW-trained extension steps with recurrent HDR training to reconstruct long-tailed luminance and generalize to in-the-wild images.The introduction illustrates recovery of skin tone, bokeh, flower color, reflections, spotlights, and a sunlit window.Source: §1 · p. 2

2 Related Work

Prior approaches reconstruct clipped highlights through heuristic enhancement, direct HDR prediction, or generated exposure stacks. The paper contrasts these strategies with bounded recurrent extension and memory-efficient recurrent training.

2 Related Work

  • Earlier inverse-tone-mapping methods heuristically brighten highlights, sometimes infer color through cross-channel correlation, but struggle to reproduce details and generalize to natural scenes.Source: §2 · p. 3
  • Direct LDR-to-HDR regression predicts an unbounded range, so prior methods compress luminance and rely on limited HDR training data, which can hinder in-the-wild generalization.The paper instead combines RAW-image diversity with HDR radiance information in a two-stage training scheme.Source: §2 · p. 3
  • Exposure-stack methods generate differently exposed LDR images, but recursive estimation can accumulate errors and shifted exposure bounds may remove valid input content.Source: §2 · p. 3
  • The paper’s recurrent extension progressively expands dynamic range, while generative approaches can fill clipped areas more effectively but may alter image content and require large models and slow inference.The paper reports high-resolution operation and reconstruction of a 4k image in half a second.Source: §2 · p. 3
  • Memory Replay reduces recurrent-backpropagation compute, allowing full ResNet-based training without sacrificing resolution or network capacity.Source: §2 · p. 3

3 Recurrent Dynamic Range Extension

The method reconstructs clipped highlights through bounded single-EV residual extensions, repeated until clipping is removed. It trains those steps on RAW and HDR data, backpropagating through recurrent inference with Memory Replay.

3 Recurrent Dynamic Range Extension

  • The same network repeatedly extends a linear input by one EV using a residual added to the input, then reverses normalization and blends over the input to form the HDR estimate.Inference stops when the output maximum falls below the empirical clipping threshold of 1.8; the number of steps depends on scene content.Source: §3 · p. 4

3.1 Learning Single-EV Extension

  • Single-EV training derives input-target pairs one EV apart from RAW collections and uses LPIPS with separate discriminators for the extended image and residual.Conditioning the extended-image discriminator on the input helps avoid trivial reproduction, while the residual discriminator constrains residual distributions; both address distinct degenerate solutions.Source: §3.1 · p. 4, Figure 3 · p. 4

3.2 Learning Accurate Highlight Distributions

  • Recurrent HDR training repeatedly applies the network for N inference steps, uses each step’s prediction as the next input, supervises intermediate predictions with HDR-derived ground truth, and backpropagates losses through the entire chain.Source: §3.2 · pp. 4–5, Figure 4 · p. 5

3.3 Network Architecture And Training Details

  • The recurrent stage uses HDR images to learn long-tailed highlight distributions while retaining the prior from single-stage training, on a fully convolutional MiDaS 2.1 network with a ResNext101 backbone.The recurrent training data include HDRT, S2R-HDR, and HDR-Real.Source: §3.2 · p. 5, §3.3 · p. 5

4 Experiments

The experiments compare progressive extension with direct HDR and exposure-stack methods on SI-HDR and BtP-HDR, using quantitative and visual analyses. The method outperforms baselines on SI-HDR, performs competitively on BtP-HDR, and recovers highlights and clipped details in qualitative examples.

4.1 Quantitative Evaluation

  • Evaluation uses SI-HDR and BtP-HDR, with best-scale alignment based on valid input pixels and a third-degree polynomial correction of the camera response function.The benchmarks were not used for this method's training; evaluation and visual results use inverse gamma as a baseline linearization method.Source: §4.1 · p. 6, §4 · p. 6
  • On SI-HDR, recurrent dynamic range extension outperforms all baselines, with significant gains on most metrics and large gains on FLIP and P-H.The authors associate the gains with wide-dynamic-range reconstruction and realistic highlights; Table 1 reports quantitative results on SI-HDR and BtP-HDR.Source: §4.1 · p. 6, Table 1 · p. 6
  • On BtP-HDR, the method performs competitively with GaSLight, which was trained on that benchmark, while BtP-HDR was not used to train this method.The authors present this as evidence of generalization to indoor as well as outdoor images.Source: §4.1 · p. 6, Table 1 · p. 6
  • With recurrent training and full HDR supervision, the network averages five inference steps, more than doubles the number of stops, and outperforms the strongest baselines.Without this phase, training saturates early and additional executions do not improve performance; the dynamic-range evaluation masks extended shadows and uses the original LDR's 8-stop range as input.Source: §4.1 · pp. 6, 8, Table 2 · p. 8

4.2 Qualitative Evaluation

  • On SI-HDR highlight examples, the method matches the bright specular spot and recovers some lost book-cover texture, while baselines produce flat regions or leave highlights unchanged.In the Plane and Owl examples, the method recovers strong reflections and extends dynamic range beyond four exposure values, consistent with the ground truth.Source: Figure 5 · p. 7, §4.2 · p. 9
  • In the Cloudy example, the method reconstructs a realistic sun disc against the overcast sky, and the examples show extension beyond the recurrent-step count used in training.The authors describe the formulation as adaptive to scene content rather than limited to the exposure brackets used during training.Source: §4.2 · p. 9
  • Against exposure-stack methods, recurrent training preserves bright light sources and specular highlights while reducing accumulated artifacts across inference steps.In the City example, the method retains the source's brightness, realistically dims the diffuse wall, and recovers its color; in Sewing, it preserves the pin's shape and metallic appearance.Source: Figure 6 · p. 8, §4.2 · pp. 9–10
  • On in-the-wild Dress and Portrait examples, the method fills clipped regions with matching colors and recovers details that comparison methods leave clipped or render unnaturally.The authors attribute this behavior to a natural-image prior learned from diverse RAW collections and maintained through dynamic range extension.Source: Figure 7 · p. 9, §4.2 · p. 10

5 Conclusion And Limitations

The scale-invariant recurrent approach recovers highlights across diverse scenes and supports arbitrary dynamic ranges, but has limits with severely overexposed complex regions and does not address other parts of HDR reconstruction.

5 Conclusion And Limitations

  • The approach generalizes to diverse in-the-wild scenes, recovering skin tones, specular reflections, and light sources.Source: §5 · p. 10
  • Recurrent training supports extension to arbitrary dynamic ranges without restriction during training.Source: §5 · p. 10
  • The method assumes casual photographs with reasonably exposed midtones; severe clipping of complex structures can produce repetitive patterns or overbright spots.Large all-white regions may lack enough semantic information for the network to infer context.Source: §5 · p. 10, Figure 8 · p. 10
  • The method addresses clipping, not shadow noise or quantization cleanup, or inversion of the nonlinear camera response function.Source: §5 · p. 10
Loading 2609.13135v1…