Source-linked AI summary
Curricular Contrastive Regularization for Physics-aware Single Image Dehazing
Yu Zheng, Jiahui Zhan, Shengfeng He, Junyu Dong, Yong Du
TL;DR
Single-image dehazing is ill-posed, and existing contrastive negatives can be too distant from clear images while physics-aware feature interpretation remains limited. The paper introduces C2PNet, combining consensual curricular contrastive regularization with a physics-aware dual-branch unit. C2PNet is reported to outperform state-of-the-art methods, with PSNR boosts of 3.94dB on SOTS-indoor and 1.50dB on SOTS-outdoor.
Problem
Existing contrastive dehazing methods use non-consensual negatives that may be distant from clear images, while physics-aware interpretation in deep feature spaces remains underexplored.
Method
C2PNet uses hazy or restored images as consensual negatives, dynamically reweights easy, hard, and ultra-hard negatives, and models atmospheric light and transmission features in separate PDU branches.
Results
3.94dB and 1.50dB PSNR boosts are reported on SOTS-indoor and SOTS-outdoor, respectively, and the method outperforms state-of-the-art methods in synthetic and real-world scenarios.
Takeaways & Limitations
The approach establishes a consensual contrastive space and enforces physics-based priors in the feature space for single-image haze removal.
Takeaways & Limitations
Because existing methods often perform poorly in real-world scenarios, high-quality hard and ultra-hard negatives are difficult to collect, potentially limiting model capacity.
Abstract
from arXiv · showhide
Considering the ill-posed nature, contrastive regularization has been developed for single image dehazing, introducing the information from negative images as a lower bound. However, the contrastive samples are nonconsensual, as the negatives are usually represented distantly from the clear (i.e., positive) image, leaving the solution space still under-constricted. Moreover, the interpretability of deep dehazing models is underexplored towards the physics of the hazing process. In this paper, we propose a novel curricular contrastive regularization targeted at a consensual contrastive space as opposed to a non-consensual one. Our negatives, which provide better lower-bound constraints, can be assembled from 1) the hazy image, and 2) corresponding restorations by other existing methods. Further, due to the different similarities between the embeddings of the clear image and negatives, the learning difficulty of the multiple components is intrinsically imbalanced. To tackle this issue, we customize a curriculum learning strategy to reweight the importance of different negatives. In addition, to improve the interpretability in the feature space, we build a physics-aware dual-branch unit according to the atmospheric scattering model. With the unit, as well as curricular contrastive regularization, we establish our dehazing network, named C2PNet. Extensive experiments demonstrate that our C2PNet significantly outperforms state-of-the-art methods, with extreme PSNR boosts of 3.94dB and 1.50dB, respectively, on SOTS-indoor and SOTS-outdoor datasets.
1. Introduction
Single-image dehazing remains ill-posed because conventional contrastive negatives can be too distant from clear images, while physics-aware feature interpretation is underexplored. C2PNet addresses both issues with curricular consensual contrastive regularization and a physics-aware dual-branch unit.
- Haze degrades photographed images and limits subsequent high-level vision tasks, motivating effective single-image dehazing methods.
- Existing regularizations generally omit negative-image information, while non-consensual negatives may be too distant from the positive and leave the solution space under-constricted.
- Consensual negatives can be assembled from hazy inputs and restorations by existing methods, with diversity associated more with haze or haze residue than unrelated semantics.
- Single-negative experiments show hard negatives outperform easy and ultra-hard negatives, whereas combining multiple difficulty levels yields the best performance.
- Curricular contrastive regularization dynamically reweights easy, hard, and ultra-hard negatives as the anchor approaches the positive, promoting stable optimization in a more compact solution space.
- The physics-aware dual-branch unit separately approximates atmospheric-light and transmission-map features, and C2PNet integrates these units with curricular contrastive regularization.
- 3.94dB and 1.50dB PSNR boosts are reported on SOTS-indoor and SOTS-outdoor, respectively, over state-of-the-art methods.
2. Related Work
Related work spans atmospheric-model-based, data-driven, physics-aware, contrastive, and curriculum-learning approaches to image dehazing. The paper positions its method against limitations in raw-space physics modeling and distant contrastive negatives.
- Single Image Dehazing: Traditional dehazing methods use atmospheric-scattering priors, while early learning-based methods estimate transmission and atmospheric light to infer clear images.
- Single Image Dehazing: Data-driven methods such as FFANet and DeHamer improve restoration using attention and combined CNN-Transformer architectures but do not consider the physics of hazing.
- Contrastive Learning: Contrastive learning pulls anchors toward positives and away from negatives, but only a few low-level vision methods apply it to dehazing.
- Curriculum Learning: Curriculum learning arranges training samples by difficulty, motivating the paper’s strategy for handling consensual negatives with varying learning difficulty.
3. Method
C2PNet combines a physics-aware dual-branch feature unit with curricular contrastive regularization to improve interpretability and constrain single-image dehazing.
- Physics-aware Dual-branch Unit: C2PNet pursues interpretable feature-space dehazing using a physics-aware dual-branch unit derived from the atmospheric scattering model.The unit approximates atmospheric-light and transmission-map features in separate branches.
- Physics-aware Dual-branch Unit: The atmospheric scattering model represents hazy images as I(x) = T(x)J(x) + (1 − T(x))A, where T and A are unknown.The ill-posedness motivates imposing physics priors in feature space rather than directly estimating both factors in raw space.
- Physics-aware Dual-branch Unit: PDU synthesizes clear-image features as M⊙t̃ + Ã(1 − t̃), explicitly modeling the interaction between atmospheric-light and transmission features.The synergistic second term is identified as absent from FDU.
- Physics-aware Dual-branch Unit: The PDU uses global average pooling for homogeneous atmospheric-light features but convolutional layers for the non-homogeneous transmission-map features.This separates the branches according to the physical characteristics of their corresponding factors.
- Curricular Contrastive Regularization: Canonical contrastive regularization uses distant non-consensual negatives, which provide an unsatisfactory lower bound and leave the solution space under-constricted.The anchor is the recovered result, the positive is ground truth, and the regularizer minimizes anchor-positive distance while maximizing distance from negatives.
- Curricular Contrastive Regularization: The proposed regularization instead uses consensual negatives and curriculum learning, assigning difficulty-aware weights during training.The hazy input is always easy; other negatives are dynamically classified as hard or ultra-hard using network PSNR before each epoch.
4. Experiments
Experiments evaluate C2PNet on synthetic and real-world dehazing datasets, compare its components and regularization choices, and test curricular contrastive regularization across SOTA backbones.
- Results on Synthetic Datasets: C2PNet achieves the best reported PSNR and SSIM on SOTS-indoor and SOTS-outdoor against competing methods.It records 42.56dB PSNR and 0.9954 SSIM indoors, and 36.68dB PSNR and 0.9900 SSIM outdoors.
- Results on Synthetic Datasets: 3.94dB PSNR and 0.0045 SSIM separate C2PNet from UDN on SOTS-indoor.The comparison is against the second-best method, UDN.
- Results on Synthetic Datasets: 1.50dB PSNR and 0.0029 SSIM are the minimum reported gains of C2PNet on SOTS-outdoor.The passage reports at least these performance gains on the outdoor benchmark.
- Results on Real-world Datasets: C2PNet outperforms all competitors on Dense-Haze and NH-Haze2 in PSNR and SSIM.Visual comparisons also describe C2PNet as removing most haze and producing the most desired image among the compared reconstructions.
- Ablation Study: The PDU improves PSNR by 1.71dB over base+FDU and by 1.91dB over the base network.The ablation attributes these gains to disentangling atmospheric-light and transmission-related features.
- Ablation Study: Consensual CR improves PSNR by 3.79dB over base+PDU and 0.77dB over non-consensual CR, while reducing training time from 200 to 137 hours.The comparison uses seven consensual negatives versus ten non-consensual negatives.
- Generality Analysis for C2R: C2R adds 0.57dB PSNR over consensual CR without curriculum learning and improves five SOTA backbones by 0.70–3.30dB on average.It also exceeds non-consensual CR by average PSNR improvements of 0.93–2.10dB and exceeds self-paced curriculum learning by up to 0.71dB.
5. Discussion and Limitation
The method’s real-world performance may be limited by the difficulty of obtaining high-quality non-easy negatives, especially ultra-hard negatives.
- Poor real-world performance by existing dehazing methods makes high-quality non-easy negatives difficult to collect.The limitation is especially pronounced for ultra-hard negatives.
6. Conclusion
C2PNet combines consensual negatives, curricular contrastive regularization, and a physics-aware dual-branch unit for single image dehazing. Experiments demonstrate the proposed method’s validity and generality.
- C2PNet uses consensual negatives and curricular contrastive regularization to constrain a more compact solution space.The curriculum accounts for differences in negative difficulty.
- A physics-aware dual-branch unit makes feature representations conform to the hazing process to facilitate haze removal.The unit is based on the physics model.
- Extensive experiments demonstrate the validity and generality of C2PNet.