PaperlayerView exact revision on arXiv

Source-linked AI summary

Di$^2$CycleSB: Towards High-Quality Unsupervised Nighttime Visibility Enhancement via Schrödinger Bridge Transformer

Hanting Li, Xin Sun, Wei Ye, Jungong Han, Liang-jie Zhang

arXiv:2608.29043v1cs.CV

TL;DR

Nighttime visibility enhancement remains challenged by light effects, which existing methods may amplify and fixed, hand-crafted priors struggle to estimate when glow is non-uniform. Di²CycleSB combines spatially adaptive glow priors with a cycle Schrödinger bridge framework, and experiments report improved night-image quality and visual perception over existing approaches.

  • Problem

    Existing nighttime enhancement methods can amplify light effects, while fixed hand-crafted priors struggle to estimate their non-uniform distribution in real-world scenes.

    Source: A. Nighttime Light Effects Suppression · p. 3, B. Di²LE · p. 5

  • Method

    Di²CycleSB combines spatially adaptive Gaussian-like glow priors and a prior-informed generator with forward and backward Schrödinger bridges and cycle consistency for unpaired enhancement.

    Source: A. Di²CycleSB · p. 4, §VI · p. 11

  • Results

    Experiments demonstrate that Di²CycleSB substantially improves night-image visual perception and image quality and significantly outperforms existing approaches.

    Source: §I · p. 2

  • Takeaways & Limitations

    The framework achieves end-to-end light-effect suppression without complex hand-crafted priors, specialized training strategies, or image decomposition.

    Source: §I · p. 2

Abstract

from arXiv · show

Light-effect contamination poses a significant challenge to nighttime visibility enhancement. Most methods suppress light effects by estimating and decomposing them through prior-driven regularization, yet they are often limited by hand-crafted priors and ill-posed nature of decomposition. This work proposes Di$^2$CycleSB, a unsupervised Cycle Schrödinger Bridge Transformer framework guided by dynamic integral image priors, for high-quality unsupervised nighttime visibility enhancement. Specifically, a novel light-effect estimator is introduced to parameterize Gaussian-like adaptive priors by aggregating dynamic integral image representations for non-uniform glow estimation. Then, we propose a prior-informed Generator that exploits light-effect representations to guide long-range dependency modeling within our specific Transformer blocks. We formulate light-effect suppression as a Schrödinger bridge problem and construct forward and backward bridges with cycle consistency constraints to achieve visually pleasing enhancement. Extensive experiments on real-world datasets demonstrate the remarkable effectiveness of our Di$^2$CycleSB in enhancing nighttime visibility. In particular, it achieves effective end-to-end light-effect suppression without any regularization constraints and image decomposition. The code and models are available at https://github.com/LHTcode/Di2CycleSB.

I. INTRODUCTION · II. RELATED WORK

Di²CycleSB addresses nighttime light-effect contamination with a cycle Schrödinger Bridge framework guided by spatially adaptive light-effect priors. Its estimator and Transformer generator enable end-to-end suppression without image decomposition or hand-crafted priors.

II. RELATED WORK

  • Prior low-light enhancement methods – overlook light effects and may amplify them, while nighttime dehazing methods – can over-clean images and produce unnatural appearances.These methods may alleviate glow artifacts, but typically enforce global haze suppression.Source: §I · p. 1

I. INTRODUCTION

  • Di²CycleSB achieves the best performance across six quality metrics against UNIE, NightEnhance, and DehazeSB on the Light-Effects and NTIRE 2025 Challenge datasets.The introduction also reports that the method substantially improves visual perception and image quality and significantly outperforms existing approaches.Source: Figure 1 · p. 1, §I · p. 2

II. RELATED WORK

  • Existing light-effect methods,, – rely on constrained or ill-posed decomposition, hand-crafted or prior-based guidance, or synthetic-data pseudo-labels, limiting suppression in diverse real scenes.GAN-based unpaired approaches also risk mode collapse, and CNN generators can miss long-range dependencies, leading to insufficient suppression.Source: §I · pp. 1–2

I. INTRODUCTION

  • Di²CycleSB formulates suppression as a Schrödinger Bridge problem with forward and backward bridges and cycle consistency, avoiding ill-posed image decomposition.The bridges impose cycle-consistency constraints at arbitrary time steps during iterative adversarial learning.Source: §I · p. 2
  • Di²CycleSB is presented as the first method to suppress light effects on high-resolution unpaired night images.The paper attributes this capability to adaptive light-effect estimation and cycle multi-step models for solving the Schrödinger Bridge.Source: §I · p. 2
  • Di²LE aggregates dynamic low-pass integral-image responses with learnable weights to estimate non-uniform light effects through spatially adaptive Gaussian-like priors.The adaptive-receptive-field operators parameterize these priors without loss-function constraints.Source: §I · p. 2
  • The PI Generator uses estimated light effects at multiple scales to guide Transformer blocks in modeling long-range dependencies among regions with different lighting conditions.The paper describes it as the first Transformer-based unpaired Schrödinger Bridge.Source: §I · p. 2

A. Nighttime Light Effects Suppression · B. Schr¨odinger Bridge based methods · III. SCHR ¨ODINGER BRIDGE PRELIMINARY

Di²CycleSB targets nighttime glow with a spatially varying prior-guided estimator and cyclic Schrödinger Bridge training on unpaired data. The sections motivate this design by outlining limitations of existing suppression methods and the bridge formulation.

A. Nighttime Light Effects Suppression

  • Nighttime glow forms as atmospheric scattering sends light from sources to the camera, degrading visibility; conventional low-light enhancement can amplify these effects.The cited low-light methods improve dark-region illumination but do not suppress light effects.Source: A. Nighttime Light Effects Suppression · pp. 2–3
  • Prior approaches include CRF, frequency-decomposition, and HDR processing, GAN-based layer decomposition, and APSF models,, but face paired-data needs or decomposition and estimation limitations.Layer-decomposition methods rely on multiple prior-driven loss constraints, while APSF methods use globally consistent estimation and ill-posed decomposition.Source: A. Nighttime Light Effects Suppression · p. 3
  • Paired datasets include synthetic Flare7K, real-flare-augmented Flare7K++ [30], and NightLight, but synthetic generalizability and real-pair collection remain challenges, making unpaired data promising.Collecting paired real-world nighttime images often requires specialized equipment.Source: A. Nighttime Light Effects Suppression · p. 3
  • The method avoids explicit image decomposition and prior-driven losses, using a dynamic integral-image estimator to guide a Transformer-based Schrödinger Bridge in suppressing spatially varying light effects.The estimator is prior-embedded and produces a spatially varying light-effect map.Source: A. Nighttime Light Effects Suppression · p. 3

B. Schr¨odinger Bridge based methods

  • Schrödinger Bridge image translation includes paired methods I²SB and InDI, unpaired methods UNSB and Schrödinger Bridge Flow, and retinal and dehazing applications.The passage notes that GANs can suffer mode collapse and image hallucinations in low-level vision tasks.Source: B. Schr¨odinger Bridge based methods · p. 3
  • Di²CycleSB proposes cyclic Schrödinger Bridge training on unpaired data to suppress light effects under an adaptive Gaussian-like prior.The framework is motivated by adversarial Schrödinger Bridge objectives.Source: B. Schr¨odinger Bridge based methods · p. 3

III. SCHR ¨ODINGER BRIDGE PRELIMINARY

  • The Schrödinger Bridge problem (SBP) learns an optimal stochastic transport process connecting two arbitrary distributions under a reference measure and is also interpretable as entropy-regularized optimal transport, [41].The endpoint distributions π0 and π1 are on Rd. The path space Ω is on Rd, and the Wiener reference measure Wτ has variance τ.Source: §III · p. 3
  • Directly solving the bridge over path measures is intractable; this work injects cycle consistency into unpaired bridge training for prior-guided nighttime light-effect suppression.The passage notes tractable Schrödinger Bridge classes,,.Source: §III · p. 3

IV. METHOD · A. Di²CycleSB

Di²CycleSB uses unpaired forward and backward Schrödinger bridges with cycle consistency for nighttime light-effect suppression, followed by low-light enhancement. Its framework combines a light-effect estimator and prior-informed generator with bridge, cycle, and content-consistency losses.

IV. METHOD

  • The two-stage framework first suppresses light effects and then enhances low-light images, using a generator composed of Di²LE and a prior-informed Generator.Di²LE estimates a light-effect map, which guides suppression in the PI Generator; the PIT Block is a core component of that generator.Source: §IV · pp. 3–4, Figure 3 · p. 4

A. Di²CycleSB

  • The method formulates unpaired suppression as a Schrödinger Bridge problem between light-effect-affected and light-effect-free image distributions, adding a backward bridge and cycle constraints.The forward and backward Markov chains discretize the bridge into tractable transformations.Source: A. Di²CycleSB · pp. 4–5
  • Time-conditional PI Generators parameterize forward and backward transitions, which are iteratively sampled to progressively suppress light effects.The bridge is discretized into N = 5 time intervals in this work.Source: A. Di²CycleSB · pp. 4–5
  • Independent forward and backward time-step sampling establishes consistency across intermediate states and exposes generators to diverse input distributions, mitigating steganographic effects [44].This sampling also prevents degeneration into fixed stage-wise mappings and promotes semantically meaningful correspondences between the processes.Source: A. Di²CycleSB · p. 5
  • The training objective combines adversarial and Schrödinger Bridge losses with cycle, perceptual, and illumination consistency terms.Perceptual loss constrains texture and structure, while illumination consistency encourages illumination reconstruction and reduces local intensity fluctuations.Source: A. Di²CycleSB · p. 5
  • Jointly optimizing forward and backward bridges with cycle reconstruction constraints addresses adversarial instability that can cause artifacts and unnatural suppression results.The cycle loss requires outputs passed through both generators to reconstruct the input images at the same time step.Source: A. Di²CycleSB · p. 5

B. Di²LE

Di²LE addresses non-uniform nighttime light effects with spatially adaptive dynamic integral images that parameterize Gaussian-like priors, enabling estimation without hand-crafted regularization constraints.

B. Di²LE

  • The estimator predicts pixel-wise offsets and normalized weights for five anchor boxes, constructs dynamic integral images, and aggregates them into a light-effect map.The five empirically set anchor-box sizes are 5×5, 11×11, 21×21, 31×31, and 43×43.Source: B. Di²LE · pp. 5–6
  • The dynamic integral-image representations act as low-pass filters with spatially varying receptive fields, yielding a learnable glow-aware prior at each pixel.The input is first converted to an intensity image using a max-RGB operation.Source: B. Di²LE · p. 6
  • Di²LE uses spatially adaptive receptive fields and learnable aggregation to parameterize Gaussian-like priors for non-uniform light-effect estimation without hand-crafted regularization.This design targets the limitations of fixed, hand-crafted priors in real-world scenes.Source: B. Di²LE · pp. 5–6

C. Prior-Informed Generator

The Prior-Informed Generator uses a multi-scale U-shaped architecture to combine backbone features with light-effect cues, guiding suppression through time-conditional Transformer blocks. Channel-wise cross-attention enables the features to query the estimated light-effect map while limiting computational overhead.

C. Prior-Informed Generator

  • The PI Generator uses a four-layer U-shaped encoder-decoder with skip connections, a refinement block after the decoder, and resolution-adapted light-effect maps injected at each layer.The light-effect map is estimated by Di²LE and adapted using 4 × 4 convolutions with stride 2.Source: C. Prior-Informed Generator · p. 6
  • Each PIT block uses feature maps as queries and the light-effect map as keys and values in channel-wise multi-head cross-attention, integrating light-effect cues into feature representations.Treating each single-channel feature map as a token enables channel-wise affinity modeling; the transposed-attention strategy treats feature channels as tokens to reduce vanilla attention overhead.Source: C. Prior-Informed Generator · pp. 6–7
  • PIT blocks combine time-conditional RMSNorm with a Dual-Gated Feed-Forward Network to further refine features and suppress light effects.The RMSNorm scaling and bias parameters are predicted from the time step through separate linear projections.Source: C. Prior-Informed Generator · pp. 6–8
  • The multi-scale interaction between backbone features and the light-effect map captures non-local light-effect information to enable suppression.Source: C. Prior-Informed Generator · p. 8

D. Low-Light Image Enhancement

The second-stage module iteratively enhances illumination in light-effect-free night images, using the estimated light-effect map to avoid excessive enhancement of already well-lit regions.

D. Low-Light Image Enhancement

  • The module predicts a pixel-wise parameter map and uses the inverted light-effect estimate in a higher-order image-specific curve for iterative enhancement, avoiding excessive enhancement of well-lit regions.The input is the light-effect-free image produced by the first stage.Source: D. Low-Light Image Enhancement · p. 8
  • Enhancement uses three iterations, with the CNN architecture and loss functions following.The initial enhancement mapping is the identity.Source: D. Low-Light Image Enhancement · p. 8

V. EXPERIMENTS AND RESULTS · A. Datasets and Implementation Details · 1) Datasets:

Di²CycleSB is evaluated on the Light-Effects and NTIRE 2025 Challenge datasets, including an unpaired high-resolution training benchmark formed from NTIRE images.

1) Datasets:

  • The evaluation uses the Light-Effects dataset and NTIRE 2025 Challenge dataset.Source: 1) Datasets · p. 8
  • Light-Effects contains 501 real-world nighttime images with diverse light effects varying in number, shape, and color.Source: 1) Datasets · p. 8
  • NTIRE 2025 provides 1,000 paired 2k-resolution RAW images with and without light effects, captured at night using a Huawei smartphone and Sony camera.Source: 1) Datasets · p. 8
  • NTIRE RAW images are converted to RGB using the predefined processing pipeline in.Source: 1) Datasets · p. 8
  • The authors randomly select 500 light-effect images and 500 non-overlapping images without light effects from NTIRE to form an unpaired training dataset.Source: 1) Datasets · pp. 8–9
  • The resulting data is presented as a new unpaired high-resolution benchmark released with the code.Source: 1) Datasets · p. 9

2) Implementation Details: · B. Comparisons with State-of-the-Art Methods · C. Ablation Study

Di²CycleSB achieves the highest reported user-study and IQA scores, and the highest full-dataset LMM score; ablations indicate that its components contribute to performance. The method is trained with bidirectional Schrödinger Bridges and uses single-step forward inference cascaded with an LLIE module.

2) Implementation Details:

  • Training jointly optimizes forward and backward bridges for N = 5 time steps; inference uses a single forward step cascaded with LLIE.Training uses Markovian discriminators, Adam, specified loss weights, and data augmentation; 256 × 256 crops are trained on one NVIDIA 4090 GPU. The LLIE module is trained on generated light-effect-free images.Source: 2) Implementation Details · p. 9

B. Comparisons with State-of-the-Art Methods

  • Visual comparisons show Di²CycleSB suppressing light effects, enhancing visibility, and producing natural results, including on high-resolution images.The cited comparisons report color distortions or inadequate suppression for EnlightenGAN, NightEnhance, SFSNiD, SFN++, and DehazeSB, and artifacts for UNIE.Source: B. Comparisons with State-of-the-Art… · p. 9
  • Di²CycleSB achieves the highest scores in the user study on both the Light-Effects and NTIRE 2025 Challenge datasets.Twelve participants ranked eight methods on light-effect suppression, naturalness and visual consistency, and overall visibility enhancement.Source: B. Comparisons with State-of-the-Art… · p. 9
  • On full-dataset evaluation, Di²CycleSB achieves the highest GPT-5.6 Sol score, substantially outperforming the second-best method.On sampled images, GPT-5.6 Sol shows an evaluation trend consistent with human judgments.Source: B. Comparisons with State-of-the-Art… · p. 9
  • Di²CycleSB achieves top performance across all four IQA metrics on the Light-Effects and NTIRE 2025 Challenge datasets.For NTIRE 2025, LIQE evaluation resizes results to 512 × 512 while preserving aspect ratio because original-resolution scores were nearly identical across methods.Source: B. Comparisons with State-of-the-Art… · p. 9

C. Ablation Study

  • Removing any key component degrades evaluation performance; “w/o LLIE” ranks second-best across all reported metrics.Ablations remove cycle consistency, the Schrödinger Bridge formulation, Di²LE, the prior-informed generator, or LLIE; without LLIE, illumination enhancement is limited.Source: C. Ablation Study · p. 10
  • Ablations show distinct limitations: removing cycle consistency harms content consistency, while removing SB causes unnatural color reproduction.Without Di²LE, light-effect suppression is inaccurate; without the prior-informed generator, limited long-range modeling impedes content restoration in degraded regions.Source: C. Ablation Study · p. 10
  • The proposed Di²LE achieves the highest LMM-based score among six alternatives, including five fixed-kernel Gaussian filters and Max-RGB.All Di²LE variants degrade performance across IQA metrics, and none completely suppresses light effects in the illustrated comparison.Source: Table IV · p. 10, C. Ablation Study · p. 10, Figure 6 · p. 10

D. Computational Complexity Analysis · E. Visualization Analysis · VI. CONCLUSION

Di²CycleSB combines dynamic integral-image priors, a prior-informed generator, and cycle-consistent Schrödinger bridges for unsupervised nighttime light-effect suppression. The analyses report low parameter count and competitive inference speed, visualize adaptive light-effect estimation, and conclude that the method performs well on unpaired datasets.

D. Computational Complexity Analysis

  • Di²CycleSB has the fewest parameters among compared methods and ranks among the fastest recent approaches in inference efficiency.The parameter comparison indicates its performance gains do not come from increased model capacity.Source: D. Computational Complexity Analysis · p. 10

E. Visualization Analysis

  • In Fig. 7, regions more affected by light effects receive larger dynamic integral regions, while less affected areas receive smaller ones.The visualizations show the K = 5 dynamic integral regions and corresponding weighting coefficients.Source: Figure 7 · p. 11, E. Visualization Analysis · p. 11
  • Fig. 8 presents Di²LE-estimated light-effect maps alongside enhancement results produced by Di²CycleSB.The figure visualizes the estimated maps.Source: Figure 8 · p. 11, E. Visualization Analysis · p. 11
  • Di²LE uses dynamic integral-image representations to emulate spatially varying low-pass filters and parameterize Gaussian-like priors with varying receptive fields and variances.These priors support non-uniform light-effect estimation without hand-crafted constraints.Source: E. Visualization Analysis · pp. 10–11
  • Di²LE’s dynamically estimated coefficients spatially adapt the aggregation of integral-image representations to estimate pixel-wise Gaussian-like priors.The second row of Fig. 7 visualizes the weighting coefficients.Source: E. Visualization Analysis · p. 11, Figure 7 · p. 11

VI. CONCLUSION

  • The conclusion presents Di²CycleSB as an unsupervised framework that combines adaptive Gaussian-like glow priors, a prior-guided PI Generator, and cycle-consistent forward and backward Schrödinger bridge transport.The paper reports that extensive experiments on unpaired datasets demonstrate the method’s superiority.Source: §VI · p. 11

Related papers

Loading 2608.29043v1…