PaperlayerView exact revision on arXiv

Source-linked AI summary

UniPolar: Joint Image and Polarimetric Restoration under Compound Degradations via Physical Consistency Learning

Yang Jin, Guangyu Guo, Linghao Shen, Binglu Wang

arXiv:2610.23360v1cs.CVeess.IV

TL;DR

Compound degradations in CPFA imaging affect both image intensities and cross-angle polarization differences, while existing restoration approaches do not adequately address their joint effects and physical relationships. UniPolar couples image and Stokes streams and applies physical-consistency constraints. It reports improvements in image fidelity and polarimetric accuracy across normal, low-light, and haze conditions.

  • Problem

    Existing restoration methods often address device or environmental degradations separately, while general multi-degradation methods do not explicitly model cross-angle polarization differences or their physical relationships.

    Source: §I · pp. 1–2

  • Method

    UniPolar uses coupled image and Stokes streams, degradation-conditioned cross-domain interaction, and output-level physical-consistency constraints.

    Source: §I · p. 2

  • Results

    Experiments across normal, low-light, and haze conditions show improved image fidelity and polarimetric accuracy over representative task-specific and general multi-degradation methods, particularly in AoLP estimation.

    Source: §VI · p. 9

  • Takeaways & Limitations

    The reported results support considering image fidelity and polarimetric accuracy together when restoring compound-degraded CPFA observations.

    Source: §I · p. 1, §VI · p. 9

Abstract

from arXiv · show

Polarimetric image restoration aims to recover polarization angle images and reliable polarimetric quantities from degraded observations. Beyond visual fidelity, it also requires preserving the physical relationships underlying reliable polarimetric estimation. However, existing methods typically address either device- or environment-induced degradations in isolation, limiting their applicability to color polarization filter array (CPFA) imaging, where both often coexist. Such compound degradations jointly perturb image intensities and cross angle polarization differences. Consequently, improved image fidelity alone does not necessarily ensure accurate Stokes, DoLP, and AoLP recovery. To address this challenge, we propose UniPolar, a coupled Image Stokes framework for joint image polarimetric restoration from compound-degraded CPFA observations. Specifically, UniPolar employs two complementary streams to reconstruct four polarization-angle images and directly predict Stokes difference components. To coordinate these streams, Degradation Conditioned Cross Domain Interaction combines Dual Domain Frequency Interaction for frequency selective evidence exchange with Degradation Conditioned Modulation for condition adaptive feature correction. Furthermore, at the output level, Image Stokes Physical Consistency Learning aligns image derived and directly predicted Stokes components while regularizing angular fidelity and the physical feasibility of DoLP. On the compound-degradation benchmark, UniPolar achieves average image and DoLP PSNR of 31.522 dB and 24.097 dB, respectively, with an average AoLP error of 16.833 degree. These results underscore the importance of jointly considering visual fidelity and polarimetric reliability, offering a physically grounded perspective on learning based restoration under compound degradations.

I. INTRODUCTION

Compound device and environmental degradations jointly damage CPFA image quality and the cross-angle relationships needed for reliable polarimetric estimates. UniPolar addresses this gap with coupled image and Stokes streams and output-level physical consistency constraints.

A. Polarimetric Image Restoration

  • Polarimetric restoration must recover visually faithful angle images while preserving physical relationships needed for reliable AoLP and DoLP estimation.CPFA spatially multiplexes differently oriented polarized images, from which Stokes parameters, DoLP, and AoLP are estimated.Source: §I · p. 1

I. INTRODUCTION

  • Compound device and environmental degradations jointly reduce visual quality and disrupt cross-angle Stokes relationships, with errors propagating to derived polarization attributes.Device effects include directional undersampling, resolution loss, and noise; environmental effects include low illumination and haze.Source: §I · p. 1

B. Multi-Degradation Image Restoration

  • Prior multi-degradation restoration methods mainly target RGB images and do not explicitly model cross-angle polarization differences or the physical relationships required for reliable Stokes estimation.Polarimetric methods have often focused on individual degradation families, fixed pipelines, or degradation-specific training.Source: §I · p. 2, A. Polarimetric Image Restoration · p. 2

I. INTRODUCTION

  • UniPolar couples separate image and Stokes streams through frequency-selective exchange and degradation-conditioned correction, then aligns their outputs with component, angular, and feasibility constraints.The framework targets compound-degraded CPFA observations rather than treating image fidelity as sufficient for reliable polarimetric recovery.Source: §I · p. 2

III. METHODOLOGY

UniPolar formulates compound-degraded CPFA restoration as jointly recovering polarization-angle images and polarimetric quantities from one observation. Separate image and Stokes streams retain complementary evidence while representing two estimates of the same physically coupled state.

A. Problem Formulation and Design Principle

  • The task jointly restores four polarization-angle images and associated Stokes components, DoLP, and AoLP because the polarimetric quantities depend on cross-angle image differences.The formulation treats the image and polarimetric objectives as physically coupled, not independent.Source: A. Problem Formulation and Design… · pp. 2–3
  • UniPolar maps sparse measurements, sampling masks, degraded s1/s2 cues, and an environmental-degradation label to restored angle images and directly predicted Stokes difference components.The outputs are the four-image set ˆI and the directly predicted components ˜s∆ = {˜s1, ˜s2}.Source: A. Problem Formulation and Design… · p. 3

B. Coupled Image–Stokes Representation

  • Mask-aware locally filled observations provide coarse degraded Stokes cues that initialize a separate stream rather than serving as restored components.The cues inherit interpolation and compound-degradation errors.Source: B. Coupled Image–Stokes Representation · p. 3
  • The image stream preserves intensity, boundaries, and texture, while the Stokes stream models spatial variation in s1/s2; their separate pathways retain distinct restoration objectives.Their corresponding-level features can exchange complementary evidence because both describe the same scene.Source: B. Coupled Image–Stokes Representation · p. 3

C. Degradation-Conditioned Cross-Domain Interaction

Degradation-Conditioned Cross-Domain Interaction links separate image and Stokes representations by exchanging frequency-selective evidence and adapting features to the degradation condition. DDFI determines which evidence is shared, while DCM determines how it is adapted.

C. Degradation-Conditioned Cross-Domain Interaction

  • Low-frequency bands exchange global information, while difference-aware gates mediate cross-domain corrections in the high-frequency bands.The frequency decomposition separates global degradation correction from polarization-sensitive detail transfer.Source: C. Degradation-Conditioned… · p. 4
  • The inverse wavelet transform reconstructs interaction-enhanced features from updated subbands, retaining global degradation cues and polarization-sensitive details.The exchange is restricted to corresponding bands rather than collapsing the streams into an undifferentiated representation.Source: C. Degradation-Conditioned… · p. 4
  • Degradation-Conditioned Modulation projects the degradation label into channel-wise scaling and bias parameters that produce condition-specific feature corrections.Its modulation retains the shared representation while adapting a compact residual feature.Source: C. Degradation-Conditioned… · p. 4
  • DDFI exchanges evidence between image and Stokes streams, while DCM applies degradation-dependent correction without forcing identical responses.DDFI determines which evidence is shared; DCM determines how it is adapted.Source: C. Degradation-Conditioned… · p. 4

D. Physical Consistency Learning

Physical Consistency Learning uses image-derived Stokes components as a supervisory bridge to directly predicted components. It separately regularizes angular fidelity and DoLP feasibility.

D. Physical Consistency Learning

  • The first physical constraint aligns Stokes difference components computed from restored images with the directly predicted components.Feature-space coupling alone does not guarantee agreement between these output-level estimates.Source: D. Physical Consistency Learning · p. 4
  • A DoLP-weighted angular fidelity loss aligns predicted Stokes direction with the reference while reducing supervision where AoLP is poorly defined.The direction of the Stokes vector encodes twice the AoLP, and the weighting emphasizes reliable DoLP regions.Source: D. Physical Consistency Learning · p. 5
  • A separate feasibility term penalizes DoLP estimates above the physical upper bound of one without affecting feasible estimates.Component agreement alone does not enforce AoLP periodicity or DoLP feasibility.Source: D. Physical Consistency Learning · p. 5

E. Training Objective

The training objective combines direct supervision, Laplacian frequency preservation, and output-space reconciliation. Its weights balance reconstruction, Image–Stokes agreement, polarimetric constraints, and structural regularization.

E. Training Objective

  • Frequency supervision and physical constraints complement direct data fidelity.Restored images and predicted Stokes components are supervised.Source: E. Training Objective · p. 5
  • The overall objective combines direct supervision, Laplacian frequency preservation, and output-space reconciliation.Its weights balance direct reconstruction, Image–Stokes agreement, polarimetric constraints, and structural regularization; each output is anchored to its reference while the predictions are coupled through shared physical relations.Source: E. Training Objective · p. 5
  • The objective couples the two predictions through the physical relations they must share.The objective weights balance Image–Stokes agreement alongside direct reconstruction and structural regularization.Source: E. Training Objective · p. 5

IV. EXPERIMENTS

UniPolar is evaluated on compound-degraded CPFA data and individual degradation benchmarks, with image restoration and polarimetric recovery measured separately. It leads the compound-degradation comparison on average image and DoLP PSNR and AoLP error, while results on individual tasks vary by metric.

1) Dataset:

  • The benchmark contains 528 training and 114 test samples across normal, low-light, and haze conditions, with registered four-angle images and synthesized CPFA mosaics.Haze is synthesized because real hazy polarimetric data are scarce; the test split includes 70 normal, 30 low-light, and 14 haze samples.Source: 1) Dataset · p. 5

2) Evaluation Metrics:

  • Evaluation reports mean image PSNR/SSIM, s0/2 and DoLP metrics per RGB channel, Image–Stokes agreement, and periodic AoLP error.AoLP error uses the shortest angular difference under its π-periodicity.Source: 2) Evaluation Metrics · p. 5

3) Implementation Details:

  • The model takes sparse CPFA measurements, masks, and observation-derived degraded s1/s2 cues, and outputs four polarized images plus Stokes s1/s2.The task-matched degradation condition is provided during both training and inference; comparison methods use matched data splits and input-matched cues where specified.Source: 3) Implementation Details · p. 5, B. Comparison with State-of-the-Art · p. 6

B. Comparison with State-of-the-Art

  • UniPolar achieves the best average image and DoLP PSNRs and lowest average AoLP error among compared methods on compound-degraded CPFA observations.Against input-matched DFPIR*, it improves average image PSNR/SSIM by 1.830 dB/0.018 and DoLP PSNR/SSIM by 1.001 dB/0.056, while reducing AoLP error by 1.543°.Source: Table I · p. 6, B. Comparison with State-of-the-Art · p. 6
  • On PIDSR, UniPolar ranks first across reported metrics, including a mean image score of 44.7447/0.9852 and an AoLP error of 11.4149°.On PLIE, PLIE has higher s0 and DoLP PSNR, while UniPolar has higher DoLP SSIM (0.6765) and lowers AoLP error from 15.9859° to 14.0609°; mean image score is unavailable for PLIE.Source: Table II · p. 7, Table III · p. 7, B. Comparison with State-of-the-Art · p. 7
  • On Synthetic MQ, UniPolar improves mean image PSNR over PUGDiff by 1.1316 dB; on Synthetic PIDSR, it leads mean image and s0 reconstruction, while PUGDiff retains the best DoLP result.On Synthetic PIDSR, its mean image and s0 PSNR gains over PUGDiff are 1.4442 and 4.7762 dB, respectively.Source: Table III · p. 7, B. Comparison with State-of-the-Art · p. 7

C. Ablation Study and Model Analysis

The ablations indicate complementary contributions from separate image and Stokes representations, frequency interaction, condition adaptation, and physical consistency. Frequency-branch analysis further supports combining low- and high-frequency exchanges and adding a frequency loss.

1) Ablation Study:

  • The full model improves mean image PSNR/SSIM by 3.8140 dB/0.0765 and DoLP PSNR/SSIM by 2.0743 dB/0.1177, reducing AoLP error by 7.8197°.Progressive integration shows distinct contributions from dual-domain representations, DDFI, DCM, and PCL; DCM's image PSNR gain is largest at 1.4139 dB, while its DoLP PSNR decreases by 0.1285 dB.Source: 1) Ablation Study · p. 7

2) Analysis of Dual-Domain Frequency Interaction:

  • The low-frequency branch improves mean-image and DoLP PSNR over high-frequency interaction alone by 0.1545 and 0.5611 dB, while combining both branches reduces AoLP error to 17.3958°.Adding Lfreq further improves mean-image and DoLP PSNR by 0.6412 and 0.6497 dB and reduces AoLP error by 0.5624°; Fig. 4 reports subband MAE reductions.Source: 2) Analysis of Dual-Domain Frequency… · pp. 7–8, Figure 4 · p. 8

3) Analysis of Degradation-Conditioned Modulation:

Physical consistency constraints improve angular and polarization-magnitude measures, while qualitative comparisons span normal, low-light, and haze conditions. The reported failure cases cluster around weak polarization, low signal-to-noise ratios, and spatially varying scattering.

4) Analysis of Physical Consistency Learning:

VI. CONCLUSION

  • UniPolar combines separate image and Stokes streams with frequency-selective exchange, degradation-conditioned adaptation, and physical-consistency constraints.Experiments across normal, low-light, and haze conditions report improvements in image fidelity and polarimetric accuracy over representative task-specific and general multi-degradation methods, particularly in AoLP estimation under challenging conditions.Source: §VI · pp. 8–9

V. QUALITATIVE RESULTS

  • Across normal, low-light, and haze scenes, UniPolar is reported to preserve sharper structures and more coherent polarization patterns than competing methods.The comparisons assess restored s0, DoLP, and AoLP; the cited failure cases associate errors with weak polarization, low signal-to-noise ratios, and spatially varying scattering.Source: §V · p. 8, Figure 6 · p. 9, Figure 7 · p. 9

APPENDIX A DATASET GENERATION

The dataset pairs aligned clean polarization-angle images with simulated CPFA acquisition, sensor noise, and haze to represent compound device- and environment-induced degradations. It contains 642 samples, with 528 for training and 114 for testing.

A. Dataset Composition

  • 642 samples comprise 528 training and 114 test examples spanning normal, low-light, and haze conditions.The training split contains 359 normal, 100 low-light, and 69 haze samples; the test split contains 70, 30, and 14, respectively.Source: A. Dataset Composition · p. 9
  • Four registered images at 0°, 45°, 90°, and 135° serve as clean targets, with shared downsampling and cropping preserving cross-angle alignment.The aligned targets support reliable computation of Stokes parameters, DoLP, and AoLP.Source: A. Dataset Composition · pp. 9–10

B. CPFA Mosaic Sampling

  • CPFA sampling decomposes each angle image into RGB channels and uses binary masks to retain only the color-angle samples observed at each pixel.The model receives 12 sparse observation channels and 12 corresponding mask channels, marking observed samples and locations requiring reconstruction.Source: B. CPFA Mosaic Sampling · p. 10

C. Sensor Noise Simulation

  • Sensor corruption uses a heteroscedastic Gaussian approximation to Poisson–Gaussian noise, combining intensity-dependent shot noise with approximately intensity-independent readout noise.After noise simulation, the images are converted into sparse CPFA observations using the fixed sampling pattern.Source: C. Sensor Noise Simulation · p. 10

D. Haze Degradation

Haze is synthesized on dense polarized images using a shared scene-level relative-depth prior and a physics-based polarimetric haze model. CPFA sampling and sensor noise are applied online rather than during the offline haze-synthesis stage.

D. Haze Degradation

  • A pretrained Depth Anything model estimates relative inverse depth from a representative RGB intensity image, and the scene-level prior is shared across all four polarizer angles.The depth map is normalized within each image because the predictor supplies relative rather than metric depth.Source: D. Haze Degradation · p. 11
  • The normalized depth map is used to compute transmission with an atmospheric scattering model.The scattering coefficient is denoted β_scat.Source: D. Haze Degradation · p. 11
  • A physics-based polarimetric haze model combines attenuated scene radiance with partially polarized airlight to produce direction-dependent haze responses.The implementation fixes P_A = 0.15, P_T = 0.08, and ϕ_ref = 0°; CPFA sampling and sensor noise are applied online during training, not in offline haze synthesis.Source: D. Haze Degradation · p. 11

APPENDIX B PHYSICAL CONSISTENCY VISUALIZATION

The appendix visualizes reduced disagreement between image-derived and directly predicted Stokes components after physical consistency learning. Its hyperparameter analyses identify settings that balance image fidelity and polarimetric reliability across compound degradations.

APPENDIX B PHYSICAL CONSISTENCY VISUALIZATION

  • PCL visibly reduces structured residuals between image-derived and directly predicted s1 and s2, including around boundaries and textured regions across normal, low-light, and haze conditions.The inconsistency maps use lower values to indicate better physical consistency.Source: Appendix B · p. 11, Figure 9 · p. 11

APPENDIX C HYPERPARAMETER ANALYSIS

  • λ1 = 0.2 gives the best mean-image, s0, and DoLP performance, while λ2 = 0.03 gives the best mean-image, s0, and DoLP results among the reported settings.Increasing λ2 to 0.05 lowers AoLP error to 16.290° but noticeably degrades image fidelity and DoLP estimation.Source: Appendix C · pp. 11–12
  • The thresholds (τl, τh) = (0.05, 0.20) achieve the lowest angular error over all pixels and evaluated DoLP ranges.The selected thresholds balance suppression of uncertain AoLP estimates with retention of informative polarization cues.Source: Table XI · p. 11, Appendix C · p. 12
  • λ3 = 0.01 yields DoLP values of 24.097/0.6076 and an AoLP error of 16.833°; larger feasibility-loss weights degrade both metrics.The authors note that excessive feasibility regularization may over-constrain polarization recovery.Source: Appendix C · p. 12, Figure 10 · p. 12
  • Initializing modulation scale α to 0.01 improves mean-image PSNR by 1.238, 0.336, and 0.813 dB under normal, low-light, and haze conditions, with a best average gain of 0.811 dB.Larger initial scales produce less stable gains.Source: Appendix C · p. 12, Figure 11 · p. 12

Related papers

Loading 2610.23360v1…