PaperlayerView exact revision on arXiv

Source-linked AI summary

Di$^2$CycleSB: Towards High-Quality Unsupervised Nighttime Visibility Enhancement via Schrödinger Bridge Transformer

Hanting Li, Xin Sun, Wei Ye, Jungong Han, Liang-jie Zhang

arXiv:2608.29043v1cs.CV

TL;DR

Nighttime visibility enhancement is hindered by non-uniform light effects and the limitations of hand-crafted priors and ill-posed decomposition. Di²CycleSB combines adaptive light-effect estimation, a prior-informed Transformer generator, and cycle Schrödinger Bridges for unsupervised suppression. Experiments on unpaired datasets demonstrate high-quality light-effect suppression and improved nighttime visibility without image decomposition or specialized training constraints.

  • Problem

    Nighttime images contain non-uniform light-effect contamination, while existing approaches may amplify effects or depend on hand-crafted priors and ill-posed image decomposition.

  • Method

    Di²CycleSB uses Di²LE for spatially adaptive Gaussian-like priors, a prior-informed Generator, and forward-backward Cycle Schrödinger Bridges for unsupervised suppression.

  • Results

    Extensive experiments on unpaired datasets demonstrate the superiority of Di²CycleSB for light-effect suppression and nighttime visibility enhancement.

  • Takeaways & Limitations

    The framework achieves end-to-end light-effect suppression without hand-crafted constraints, specialized training strategies, or image decomposition.

Abstract

from arXiv · show

Light-effect contamination poses a significant challenge to nighttime visibility enhancement. Most methods suppress light effects by estimating and decomposing them through prior-driven regularization, yet they are often limited by hand-crafted priors and ill-posed nature of decomposition. This work proposes Di$^2$CycleSB, a unsupervised Cycle Schrödinger Bridge Transformer framework guided by dynamic integral image priors, for high-quality unsupervised nighttime visibility enhancement. Specifically, a novel light-effect estimator is introduced to parameterize Gaussian-like adaptive priors by aggregating dynamic integral image representations for non-uniform glow estimation. Then, we propose a prior-informed Generator that exploits light-effect representations to guide long-range dependency modeling within our specific Transformer blocks. We formulate light-effect suppression as a Schrödinger bridge problem and construct forward and backward bridges with cycle consistency constraints to achieve visually pleasing enhancement. Extensive experiments on real-world datasets demonstrate the remarkable effectiveness of our Di$^2$CycleSB in enhancing nighttime visibility. In particular, it achieves effective end-to-end light-effect suppression without any regularization constraints and image decomposition. The code and models are available at https://github.com/LHTcode/Di2CycleSB.

I. INTRODUCTION

Di²CycleSB addresses nighttime visibility degradation from non-uniform light effects, which existing enhancement and suppression methods may amplify, over-clean, or model through constrained decomposition. It introduces adaptive light-effect priors, a prior-informed Transformer generator, and Cycle Schrödinger Bridges for unsupervised end-to-end suppression.

  • Night images suffer locally insufficient visibility and contamination from glare and floodlight artifacts under non-uniform illumination and multiple light sources.
  • Existing methods may amplify light effects, enforce global haze suppression, or rely on constrained layer decomposition, producing unnatural or ill-posed solutions.
  • Di²CycleSB formulates light-effect suppression as a Schrödinger bridge problem with forward and backward processes linked by cycle-consistency constraints.
  • Di²LE estimates spatially adaptive Gaussian-like light-effect priors by aggregating dynamic low-pass integral-image responses without loss-function constraints.
  • The PI Generator uses estimated light effects to guide Transformer long-range dependency modeling for end-to-end suppression.
  • Extensive experiments report improved nighttime visual perception and image quality, with performance surpassing existing approaches.

II. RELATED WORK

Nighttime light effects remain difficult to suppress because illumination is spatially non-uniform and existing approaches rely on paired data, decomposition, or fixed priors. Di²CycleSB instead uses an unpaired cyclic Schrödinger Bridge guided by adaptive light-effect estimation.

  • Night images suffer from locally insufficient visibility and contamination from glare and floodlight artifacts caused by non-uniform illumination and multiple light sources.
  • Low-light enhancement methods can amplify light effects, while nighttime dehazing methods may produce over-cleaned and visually unnatural appearances.
  • Existing light-effect suppression methods rely on paired data, prior-driven constraints, or globally consistent atmospheric point spread functions and ill-posed image decomposition.
  • Unpaired data are easier to acquire than paired real-world nighttime images, motivating unpaired training for high-quality light-effect suppression.
  • Di²CycleSB estimates spatially varying light effects with a dynamic integral-image prior and guides a Transformer-based Schrödinger Bridge without explicit decomposition or prior-driven losses.

IV. METHOD

Di²CycleSB treats unpaired nighttime light-effect suppression as a cyclic Schrödinger Bridge and combines adaptive light-effect estimation with prior-informed generators. Forward and backward bridges, cycle consistency, and content constraints jointly support iterative transport between affected and clear-image distributions.

  • The framework has two stages: light-effect suppression followed by low-light image enhancement, with a generator composed of Di²LE and the PI Generator.
  • Training jointly models forward and backward stochastic processes between light-effect-affected and light-effect-free image distributions using unpaired data.
  • Time-conditional PI Generators share parameters across time steps and iteratively sample states to gradually suppress light effects.
  • Cycle consistency reconstructs inputs through forward-backward generator compositions, while perceptual and illumination losses constrain content and illumination consistency.
  • The total objective combines adversarial, Schrödinger Bridge, cycle, perceptual, and illumination consistency losses.
  • The Schrödinger Bridge is discretized into a Markov chain so complex distribution mapping becomes a sequence of tractable transformations.

B. Di²LE

Di²LE estimates non-uniform light effects with spatially adaptive dynamic integral images rather than fixed hand-crafted priors. It aggregates multiple learned low-frequency representations into a glow-aware light-effect map for downstream suppression.

  • Di²LE addresses non-uniform light effects that fixed hand-crafted priors struggle to estimate in real-world scenes.
  • The estimator uses four stages: integral-image calculation, pixel-wise offset and weight prediction, dynamic integral-image construction, and weighted aggregation.
  • It applies max-RGB intensity extraction and integral images to construct spatially varying low-frequency representations.
  • Five anchor window sizes, from 5×5 to 43×43, initialize integration regions and guide learning of low-frequency Gaussian priors.
  • A lightweight attention layer predicts per-pixel offsets and K weighting coefficients whose sum is one for dynamic aggregation.
  • The aggregated output is a single-channel estimated light-effect map that provides a learnable glow-aware prior at each pixel.

C. Prior-Informed Generator

The PI Generator uses a U-shaped, multi-scale Transformer architecture that injects the estimated light-effect map into each resolution stage. Cross-attention lets image features query light-effect cues, supporting non-local suppression and feature refinement.

  • Architecture: The PI Generator adopts a U-shaped architecture with encoder downsampling, symmetric decoder upsampling, skip connections, and a refinement block.The light-effect map is adapted to multiple resolutions and injected into each layer.
  • Architecture: Each stage uses a time-conditional Prior-Informed Transformer block to guide feature processing with light-effect information.The blocks include time-conditional normalization, multi-head cross-attention, and feed-forward refinement.
  • Cross-attention: Channel-wise cross-attention treats feature channels as tokens, balancing long-range dependency modeling with computational efficiency.The feature map queries informative light-effect cues from the estimated light-effect map.
  • Cross-attention: The cross-attention mechanism enables multi-scale interaction between backbone features and the light-effect map for non-local light-effect suppression.The design is intended to capture light-effect information beyond local neighborhoods.

D. Low-Light Image Enhancement

The second-stage LLIE module further enhances illumination after light-effect suppression while using the estimated light-effect map to avoid excessive enhancement in already illuminated regions.

  • Light-enhancement pipeline: The LLIE module iteratively enhances illumination in light-effect-free night images produced by the suppression stage.The enhancement is guided by prior light-effect knowledge from the first stage.
  • Light-enhancement pipeline: Heavily light-effect-affected regions are not excessively enhanced because they are usually already well illuminated.The method uses the light-effect map to guide this selective enhancement behavior.
  • Enhancement rule: The enhancement update applies the inverted light-effect map through element-wise multiplication to a higher-order image-specific curve.The curve is used for iterative low-light image enhancement.
  • Enhancement rule: The iterative process uses M = 3 steps, with LLIE0(x) defined as the identity mapping.The CNN architecture and loss functions follow the cited prior work.

1) Datasets:

The evaluation uses a real-world unpaired Light-Effects dataset and a converted unpaired high-resolution dataset derived from paired NTIRE nighttime RAW images.

  • Light-Effects dataset: The Light-Effects dataset contains 501 real-world unpaired night images with diverse light effects, including varying numbers, shapes, and colors.It serves as a standard unpaired nighttime image dataset.
  • NTIRE 2025 dataset: The NTIRE 2025 Challenge dataset contains 1,000 paired 2K-resolution nighttime RAW images captured with a Huawei smartphone and a Sony camera.The images contain light-effect-affected and light-effect-free scenes.
  • NTIRE 2025 dataset: The NTIRE RAW images are converted to RGB using a predefined processing pipeline and then formed into an unpaired training dataset.The resulting data can serve as a high-resolution unpaired benchmark released with the code.

2) Implementation Details:

The study evaluates Di²CycleSB through visual comparisons, user studies, LMM assessment, IQA metrics, and ablations, while using single-step forward-bridge inference for efficiency. Results report strong visual and metric performance, and ablations associate each component with specific degradation patterns.

  • Implementation Details: Training jointly optimizes forward and backward Schrödinger bridges with N = 5 time steps and shared generator architecture.Training runs for 100 epochs before linear learning-rate decay over the remaining 50 epochs.
  • Implementation Details: Single-step forward Schrödinger-bridge inference is used to balance light-effect suppression performance and efficiency.The forward bridge and LLIE module are cascaded during nighttime visibility enhancement.
  • Comparisons: Visual comparisons report suppression of light effects, improved visibility, and visually natural results, including on high-resolution images.The comparisons cover the Light-Effects and NTIRE 2025 datasets.
  • Evaluation: Twelve participants rank eight methods using light-effect suppression, naturalness and visual consistency, and overall visibility enhancement criteria.The user studies sample 50 enhanced images from each comparison method on each dataset.
  • Ablation Study: Removing any key framework component degrades performance across all IQA metrics and LMM-based user-study evaluation.The ablations remove Cycle, SB, Di²LE, PI Generator, or LLIE components.
  • Ablation Study: The w/o Cycle variant suppresses light effects but has limited content consistency, while w/o SB produces unnatural color reproduction.The findings distinguish the effects of cycle consistency and Schrödinger-bridge formulation.
  • Ablation Study: Replacing Di²LE or the PI Generator weakens light-effect suppression or content restoration because fixed priors and limited long-range modeling are insufficient.Alternative Di²LE designs also degrade all IQA metrics and fail to completely suppress light effects.

D. Computational Complexity Analysis

Di²CycleSB combines low-complexity inference with spatially adaptive light-effect estimation. Its dynamic integral-image representations assign larger receptive regions to strongly affected areas and support pixel-wise Gaussian-like priors.

  • Computational efficiency: Di²CycleSB has the fewest parameters among compared unsupervised and semi-supervised methods while retaining competitive inference efficiency.The paper reports that it ranks among the fastest recent approaches, indicating that its performance gains are not due to increased model capacity.
  • Adaptive estimation: Di²LE parameterizes Gaussian-like priors with varying receptive fields and variances by aggregating dynamic integral-image representations.This design estimates non-uniform light effects using spatially adaptive representations.
  • Adaptive estimation: Regions heavily affected by light effects receive larger integral regions, whereas less affected areas receive smaller ones.The corresponding heatmaps show that pixels nearer light-effect regions activate larger low-pass filtering windows.
  • Adaptive estimation: Dynamically estimated weighting coefficients enable spatially adaptive aggregation of integral-image representations for pixel-wise prior estimation.The estimated light-effect maps and enhancement results are presented as further visual evidence of this process.
  • Framework design: The framework jointly optimizes forward and backward Schrödinger bridge transport processes with cycle consistency for high-quality light-effect suppression.The PI Generator uses the estimated light-effect priors to guide end-to-end suppression.
Loading 2608.29043v1…