Source-linked AI summary
DensePol: Dense-Angle Polarization Dataset for Learning-Based Polarimetric Vision
Param Sangani, Ahmad Moori, Erik Blasch, Guna Seetharaman, Hadi Aliakbarpour
TL;DR
DensePol targets the limited angular redundancy and structured errors of conventional four-angle DoFP supervision for RGB-to-polarization prediction. It builds a 180-angle full-resolution DoT dataset and a deterministic diffusion-based predictor with circular AoLP handling and local DoLP refinement. Dense sampling reduces circular AoLP deviation from 13.36° to 2.21°, while experiments report improved prediction over evaluated baselines.
Problem
Four-angle DoFP datasets provide limited angular redundancy and introduce calibration, interpolation, and instantaneous-field-of-view errors that constrain polarization supervision fidelity.
Method
DensePol captures paired RGB–polarization data with 180 full-resolution DoT measurements and uses a deterministic diffusion predictor with circular outputs and a local DoLP refiner.
Results
13.36° to 2.21° circular AoLP deviation accompanies improved polarization prediction over evaluated diffusion and feed-forward baselines.
Takeaways & Limitations
Dense angular sampling provides high-redundancy polarization supervision, while the proposed framework improves RGB-to-polarization prediction and downstream surface-normal estimation.
Takeaways & Limitations
Dense DoT references remain vulnerable to scene motion, mechanical error, and systematics at the retained harmonic, so they are not proof of absolute accuracy.
Abstract
from arXiv · showhide
Polarimetric vision is gaining increasing attention because it provides physical cues about scene shape, material, and reflection that are difficult to recover from RGB alone. Recent work has therefore explored predicting polarization directly from conventional RGB images; however, the fidelity of these methods strongly depends on the polarization supervision used for training. Most existing datasets rely on Division-of-Focal-Plane (DoFP) cameras with four spatially interleaved analyzer orientations, which provide limited angular redundancy and introduce interpolation and instantaneous-field-of-view errors. We introduce DensePol, a high-redundancy RGB--polarization dataset based on Division-of-Time (DoT) acquisition, capturing 180 full-resolution analyzer orientations at $1^\circ$ intervals. DensePol contains 2,018 paired RGB--polarization images with the angular measurements and fitting residuals retained. Dense angular sampling substantially improves polarization stability, reducing AoLP deviation from $13.36^\circ$ to $2.21^\circ$. We further introduce a deterministic diffusion-based RGB-to-polarization framework with cyclic AoLP representation and a local DoLP refiner. Experiments demonstrate improved polarization prediction and downstream surface-normal estimation. The dataset and code will be publicly available.
1. Introduction
DensePol addresses the limited and error-prone supervision produced by four-angle DoFP polarization datasets by using dense DoT acquisition. It releases 2,018 paired RGB–polarization images and studies high-redundancy supervision for RGB-to-polarization prediction.
- Polarization supplies surface, reflection, and material cues that conventional RGB images weakly represent, motivating RGB-to-polarization prediction.The fidelity of such prediction depends on the polarization labels used for training.
- Four spatially interleaved DoFP orientations introduce calibration, interpolation, and instantaneous-field-of-view errors into polarization supervision.These structured errors can become part of the target learned by RGB-to-polarization models.
- Dense angular sampling serves as a high-redundancy reference through overdetermination, noise averaging, and residual analysis rather than proof of absolute accuracy.The acquisition remains vulnerable to scene motion, mechanical error, and retained-harmonic systematics.
- DensePol provides 2,018 paired RGB–polarization images using 180-angle dense DoT acquisition and retains angular stacks and harmonic residuals.The dataset supports both polarization-reference analysis and single-image polarization synthesis.
- DensePol combines a diffusion-based predictor with circular polarization outputs and a DoLP refiner for RGB-to-polarization synthesis.The framework predicts bounded DoLP and a circular doubled-angle representation.
2. Related Work
Polarimetric vision provides cues for shape, material, and reflection-related tasks, while RGB-to-polarization synthesis seeks to recover such cues from conventional images. Its usefulness therefore depends on the fidelity of the acquisition targets.
- Polarization provides complementary cues for surface orientation, material identity, and reflection separation across several vision applications.These applications include learned shape recovery, road-scene analysis, material segmentation, and glass segmentation.
- RGB-to-polarization synthesis aims to recover useful polarimetric cues from conventional imagery, making prediction quality dependent on acquisition-target fidelity.
3. The DensePol Dataset
DensePol constructs paired RGB–polarization data from full-resolution, 180-angle DoT sweeps and fits polarization through an overdetermined harmonic model. The released dataset retains measurement-derived references, residual diagnostics, and split-level metadata.
- 3.1. Dense Division-of-Time Acquisition: 180 full-resolution analyzer orientations are captured from 0° to 179° at 1° increments, followed by a separate polarizer-removed RGB observation.Each released pair is partitioned into four non-overlapping 1250 × 1250 quadrants, while angular analyses use the original full frame.
- 3.2. Harmonic Estimation and Residual Analysis: The 180-view sweep is fit with an overdetermined least-squares model using DC, cos 2θ, and sin 2θ components of the linear-polarization signal.Uniform angular sampling makes the basis columns orthogonal, allowing an efficient DFT implementation.
- 3.2. Harmonic Estimation and Residual Analysis: 180 samples provide 177 residual degrees of freedom, reduce independent-noise variance, and expose non-model angular structure that can alias into four-angle estimates.The residual angular spectrum is retained as a per-pixel diagnostic of model mismatch.
- 3.2. Harmonic Estimation and Residual Analysis: The resulting 180-view Stokes maps serve as measurement-derived ground truth for training and evaluation, while angular-count stability is evaluated separately.The authors explicitly distinguish this reference from absolute accuracy.
- 3. The DensePol Dataset: DensePol contains 2,018 RGB–polarization pairs, including 1,988 real pairs and 30 physically based polarized renderings.The real pairs include tabletop, multi-pose-object, and outdoor collections, with acquisition groups assigned exclusively to one partition.
4. Method
The method estimates polarization from a single RGB image using a two-stage framework: one-step diffusion prediction of a circularly encoded map followed by local DoLP refinement. Timestep-conditioned feature alignment transfers multi-stage diffusion representations without iterative denoising, while the refiner preserves Stage I AoLP and corrects local DoLP errors.
- Polarization representation: The three-channel representation encodes DoLP in one channel and AoLP through normalized double-angle cosine and sine components.Double-angle encoding addresses AoLP periodicity over [0, π).
- Framework overview: A two-stage framework predicts a coarse encoded polarization map, then applies lightweight local refinement to DoLP while preserving Stage I AoLP.The representation is [ρ, cos(2ϕ), sin(2ϕ)], which avoids direct regression across the periodic AoLP boundary.
- Stage I: Diffusion-based prediction: Stage I performs single-pass diffusion prediction: a VAE encodes RGB, a trainable U-Net predicts the polarization latent at a learned timestep, and the decoder produces the coarse map.Unlike conventional diffusion generation, inference uses one forward pass without iterative denoising.
- Stage I: Feature alignment: A frozen diffusion teacher supplies multi-timestep features, while timestep-conditioned adapters align the student features with those teacher representations during training.The student and teacher begin from shared pretrained SD2.1 weights, but only the student predictor is optimized; the prediction branch is retained at inference.
- Stage II: Local DoLP refinement: Stage II fuses RGB with the encoded Stage I prediction and estimates a gated residual correction for fine local DoLP structures.Full-resolution residual blocks use varied dilation rates, and the final correction is trained separately against ground-truth DoLP.
- Training objectives: Training combines reconstruction, DoLP, circular AoLP, unit-vector, and gradient-preservation losses, while Stage II separately minimizes refined DoLP discrepancy.The AoLP loss uses the double-angle representation, and the refinement stage focuses on local DoLP errors without changing the angular prediction.
5. Experiments
Experiments show that denser angular references improve polarization stability and evaluation consistency, while DensePol achieves strong RGB-to-polarization and downstream surface-normal results. Ablations further identify a compact DoLP refiner and unweighted cyclic representation as effective design choices.
- Angular-count convergence: 86% lower DoLP deviation and 83% lower circular AoLP deviation occur when increasing the analyzer budget from four to 90 angles, while held-out intensity NRMSE falls from 0.1026 to 0.0801.
- Reference angular density: 2.60° lower common-mask AoLP MAE, 15% lower DoLP MAE, 1.70 dB higher PSNR, and 0.221 higher SSIM result when evaluation references increase from four to 90 views.
- Refinement and saturation: 7.4% lower DoLP MAE, 0.63 dB higher PSNR, and 0.141 higher SSIM result from the selected six-block specialist, while AoLP remains unchanged.
- Downstream surface-normal transfer: 23.40° mean normal error makes DensePol first among six predictions, ahead of GenPolar at 23.52° and retrained PolarAnything at 30.67°.
- RGB-to-polarization synthesis: 11.15° lower AoLP error than the strongest feed-forward baseline and 13.78° lower than GenPolar establish DensePol’s strongest reported RGB-to-polarization comparisons.
- Target representation ablation: The unweighted circular representation is retained because DoLP-weighted angular losses improve AoLP by only 0.105° while degrading all three DoLP metrics, whereas normalized Stokes is 0.373° worse in AoLP.
6. Conclusion
DensePol combines dense full-resolution DoT supervision with a deterministic diffusion-based RGB-to-polarization framework and a lightweight DoLP refiner. Experiments report more stable references, stronger prediction than evaluated baselines, improved DoLP refinement without changing AoLP, and useful downstream surface-normal cues.
- DensePol provides 180 full-resolution analyzer measurements per capture as high-redundancy supervision for single-image polarization estimation.
- The framework combines a deterministic diffusion-based predictor with multi-timestep feature alignment and a lightweight local DoLP refiner.
- Dense angular sampling improves polarization-reference stability and reduces sensitivity to sparse-angle measurements, while the model outperforms evaluated diffusion and feed-forward baselines.
- The local refiner improves DoLP magnitude and structure without altering predicted AoLP, and synthesized polarization preserves useful physical cues for downstream surface-normal estimation.