PolarScale: A Physics-Grounded Benchmark for Radiometrically Consistent RGB-to-Stokes Estimation

1 Introduction

• Dedicated polarization hardware raises cost, can reduce spatial resolution, complicates calibration, and limits integration; existing RGB-like methods omit scale, preventing absolute Stokes reconstruction needed by radiance-level applications.Examples include diffuse/specular separation and glare assessment.Source: §1 · pp. 1–2
• PolarScale predicts normalized Stokes components, AoLP, DoLP, DoCP, and per-scene scale from scene-referred linear s0, then reconstructs full Stokes as Sn = σsn.Because σ is divided out of the input, it is not physically identifiable and is evaluated as dataset-conditioned semantic scale estimation.Source: §1 · pp. 2–3, Figure 1 · p. 2
• Restoration models beat the constant-scale control and nearly satisfy physical bounds, while two generative baselines collapse to near-zero scale; downstream probes find full-Stokes representations useful but learned-scale gains mixed.Scale accuracy remains the open problem the benchmark is designed to track.Source: §1 · pp. 2–3
• Scale estimation is scored against a constant-scale control, alongside wrap-aware angular error, Stokes self-consistency, and physical-bound violations.The benchmark package includes preprocessed splits, scale values, derived targets, evaluation scripts, and a datasheet.Source: §1 · p. 3
• Seven restoration-based and generative backbones are compared under direct, joint, and decoupled prediction strategies; explicit descriptor supervision and decoupled decoding help, with scale-collapse failures characterized.The strategies model scale-dependent components and scale-independent descriptors in different ways.Source: §1 · pp. 2–3

2 Related work

• The Spectral-Polarization dataset provides trichromatic and hyperspectral full-Stokes measurements across more than 2,000 real-world scenes, retaining radiometric scale for constructing PolarScale.Earlier datasets were often small and controlled, while Fan et al. cover 64 planar objects.Source: §2 · p. 3
• Prior RGB-based benchmarks and synthesis methods predict normalized Stokes components or relative descriptors, whereas PolarScale reuses existing measurements and changes what is predicted and evaluated.Restoration methods use deterministic pixel-wise prediction; generative methods use conditional synthesis.Source: §2 · p. 3
• Polarization supports material, reflection, geometry, segmentation, and appearance tasks; tasks operating on polarized radiance, such as reflection separation, need absolute rather than normalized Stokes information.Polarization also supports sparse sensing and wearable robotics.Source: §2 · pp. 3–4
• The benchmark compares direct Stokes estimation, joint estimation with a unified decoder, and decoupled multi-decoder estimation of scale-dependent components and scale-independent descriptors.All three strategies take normalized total-intensity image s0 as input.Source: Figure 2 · p. 4

3.1 Preliminaries: Stokes representation

• The Stokes vector comprises total intensity and three linear or circular polarization differences, while AoLP, DoLP, and DoCP are scale-invariant descriptors.Source: §3.1 · p. 4

3.2 Task formulation

• Prior RGB-based methods predict normalized Stokes components or relative descriptors, removing the scale needed to assess recovery of absolute Stokes signals.Source: §3.2 · pp. 4–5
• Input is globally normalized scene-referred linear s0, not consumer sRGB, and predictions combine normalized Stokes components, a scale map, and three descriptors.Source: §3.2 · p. 5
• Because σ is divided out of s0, it cannot be recovered by physical inversion; learned scale is dataset-conditioned semantic estimation and is evaluated against a constant-scale control.Source: §3.2 · p. 5, §3.4 · p. 6

3.3 Benchmark construction

• The benchmark uses the trichromatic Spectral-Polarization subset and a fixed 1,000/200 train/test split; the training-set median σ is 4.85 and gives 5.7% test mean relative error as a constant prediction.Source: §3.3 · p. 5
• Three strategies model scale-dependent and scale-independent quantities: direct Stokes estimation, joint estimation, and decoupled multi-decoder estimation; the package includes splits, targets, scripts, and baseline code.Source: §3.3 · p. 6

3.4 Evaluation protocol

• Evaluation pairs learned scale with a training-median constant control and reports relative and log-scale errors, wrap-aware AoLP error, descriptor self-consistency, physical-bound violations, and image-quality metrics.Source: §3.4 · p. 6

4.1 Implementation details

• Seven restoration-based and generative backbones were evaluated under three prediction strategies; RealFill and I2ITurbo use task prompts and separate component passes to preserve pretrained weights.All baselines were trained to convergence on the same server, with about one day per restoration model and two days per diffusion-based model.Source: §4.1 · pp. 6–7

4.2 Main results

• 3.6% and 4.3% mean relative error for MAE and Uformer beat the 5.7% constant-scale floor; restoration bound violations stay below 0.25% of pixels, versus 3.6–58% for generative models.DiT and WDiff collapse toward near-zero scale, while models regress toward the prior on scenes whose scale departs strongly from it.Source: Table 3 · p. 7, §4.2 · p. 7

4.3 Analysis

• 18.88 to 23.54/23.66 dB descriptor PSNR for MAE accompanies joint/decoupled estimation; joint estimation can lower I2ITurbo normalized-Stokes PSNR from 38.89 to 34.98 dB.The authors recommend explicit descriptor supervision with decoupled decoding; even direct supervision’s best AoLP angular error is 26.1°.Source: §4.2 · p. 7, Table 4 · p. 8, §4.3 · p. 8

4.2 Main results

• Full-Stokes PSNR is not a scale measure: replacing MAE’s learned scale with the training median changes PSNR from 21.08 to 21.13 dB, while WDiff’s drops from 18.30 to 2.65 dB.A fixed peak value lets PSNR reward shrinking reconstructions, so the paper uses scale metrics as the primary scale measures.Source: §4.2 · p. 8

4.3 Analysis

• Under the evaluated adaptation protocols, restoration backbones outperform generative ones on normalized-Stokes and descriptor accuracy, scale estimation, and physical consistency; I2ITurbo is the scale exception but trails on physical consistency.The comparison is limited to the evaluated architectures and adaptation protocols.Source: §4.3 · p. 8

4.4 Qualitative evaluation

• MAE alone recovers the polarized sky in scene 1043, but predicts the wrong sky S1 sign in scene 1050; all models struggle with fine structures and abrupt transitions.Figure 3 shows the scene 1043 comparison: Uformer recovers only faint edges, while DiT and I2ITurbo predict nearly flat maps.Source: Figure 3 · p. 9, §4.4 · p. 9

4.5 Downstream validation

• Predicted full-Stokes improves diffuse/specular AbsRel from 0.190/0.508 to 0.118/0.445, material mIoU from 0.330 to 0.374, and glare accuracy from 48.5% to 63.0%.For diffuse/specular separation, learned scale is on par with the constant control (0.118/0.445 versus 0.118/0.448), while ground-truth scale gives 0.100/0.440.Source: Table 5 · p. 9, §4.5 · pp. 9–10

5 Limitations and broader impacts

• Because σ is divided out of the input, scale estimates are dataset-conditioned and models regress toward the training prior when scene scales depart strongly from it.Predicted scales and reconstructed Stokes vectors are not calibrated radiometry.Source: §5 · p. 10
• All data come from one dataset captured with one polarimetric camera, and its narrow scale distribution lets a constant predictor reach 5.7% mean relative error.The best learned models reach 3.6%, leaving only a small margin; scale results may not transfer to other sensors, exposures, or scene distributions.Source: §5 · p. 10
• The benchmark uses normalized linear images captured through polarimetric optics; consumer sRGB images introduce ISP nonlinearities and polarization-dependent sensor responses.No dataset pairs consumer RGB with absolute-scale Stokes ground truth, so this setting cannot yet be evaluated quantitatively; low light, strong specular highlights, and adverse weather are not covered.Source: §5 · p. 10
• The restoration-versus-generative comparison is limited to evaluated architectures and adaptation protocols, and each configuration has results from one training run on a fixed split.Downstream probes are small-scale, two lack full scale controls, scale attribution is mixed, and all models still fail on fine structures and abrupt polarization transitions.Source: §5 · p. 10
• In safety-critical settings such as driver assistance, inferred polarization risks over-reliance because predicted scales are dataset-conditioned estimates rather than measurements.The paper calls for validation under the target sensor, lighting, and domain before high-stakes use.Source: §5 · p. 10

6 Conclusion

• The strongest evaluated restoration models beat the dataset prior and nearly satisfy physical bounds, while two generative baselines collapse the scale; explicit descriptor supervision with decoupled decoding gives the most balanced predictions.Predicted full-Stokes representations benefit three downstream tasks, but the learned scale is not yet accurate enough to consistently beat the prior.Source: §6 · p. 10

A Notation

• Table 6 lists the notation used in the paper.Source: Table 6 · p. 15

B.1 Source dataset

• The source dataset contains 2,022 trichromatic and 311 hyperspectral Stokes images, but PolarScale uses only the trichromatic subset.Trichromatic captures are single-shot measurements from a division-of-focal-plane RGB full-Stokes polarimeter; hyperspectral captures use a scanning filter and rotating quarter-wave plate.Source: §B.1 · p. 15

B.2 Split and package contents

• The fixed 1,000/200 split follows Lin et al. [22], and the package provides split scene IDs, per-scene σ values, and derived targets for reproducing the reported numbers.Source: §B.2 · p. 15

B.3 Scale distribution

• A constant predictor using training-set median σ = 4.85 or mean σ = 4.25 gives test-split mean/median relative errors of 5.7%/4.5% or 16.2%/16.0%, respectively.The mean below the median indicates a tail toward small scales.Source: §B.3 · p. 15

C Implementation details

• Baselines were trained to convergence on eight NVIDIA RTX A5000 GPUs, taking about one day per restoration model and two days per diffusion-based model.Source: C Implementation details · p. 15
• Direct estimation predicts s1, s2, s3, and scale; joint estimation adds AoLP, DoLP, and DoCP to the output.Restoration methods, WDiff, and DiT use widened output layers, while RealFill and I2ITurbo use task-specific prompts.Source: C Implementation details · p. 15
• For decoupled estimation, restoration methods, WDiff, and DiT use separate decoder branches for scale-dependent and scale-independent parameters; RealFill and I2ITurbo instead use independent LoRA parameter sets.The LoRA choice avoids the parameter redundancy and overhead of extra frozen decoder layers.Source: C Implementation details · p. 16
• WDiff and DiT use native multi-channel regression heads, whereas RealFill and I2ITurbo use prompts to preserve pretrained output dimensionality and predict components in separate inference passes.The protocols match task-level outputs so every model predicts the same components exactly once.Source: C Implementation details · p. 16
• For pretrained generative baselines, native conditioning was the weaker adaptation: the text encoder was frozen with an empty prompt, and all seven outputs were predicted jointly in a single forward pass.Source: C Implementation details · p. 16

D Constant-scale control on full-Stokes reconstruction

• For well-behaved models, constant scale matches or exceeds learned scale in full-Stokes PSNR, while near-zero predicted scales inflate learned-scale PSNR for WDiff and DiT.The reported constant scales are the training median 4.85 and mean 4.25.Source: Table 8 · p. 16, D Constant-scale control on… · p. 16

E Downstream protocols

• Diffuse/specular separation compares absolute-radiance references with polarization-based outputs composed from predicted normalized Stokes and a scale, including shuffled-scale and RGB-direct controls.The shuffled-scale control permutes learned scales across test scenes.Source: E Downstream protocols · p. 16
• Material-segmentation settings share the network, training data, schedule, and initialization, differing only in the supplied polarization representation.Source: E Downstream protocols · p. 16
• Glare classification uses glare levels computed from ground-truth polarization for the same 200 held-out scenes, with Wilson 95% score intervals and no seed or split-resampling variation.Source: E Downstream protocols · p. 17

• PolarScale targets a question unavailable in benchmarks that assess only normalized or relative polarization: whether models estimate radiometrically scaled Stokes vectors from intensity images.The package is documented following Datasheets for Datasets.Source: F Datasheet · p. 17
• The benchmark contains 1,200 real-world scenes from the trichromatic subset of [16], split into 1,000 training and 200 test instances, with polarization measurements and inherited scene labels.The package provides s0, scale σ, normalized Stokes components s1–s3, and AoLP, DoLP, and DoCP; it contains no people annotations.Source: F Datasheet · p. 17
• No new data were captured; all instances derive from [16] through deterministic preprocessing, with σ defined as each scene’s maximum S0 and normalized Stokes components computed as Sk/σ.Preprocessing also computes AoLP, DoLP, and DoCP using Eqs. (2) and (3).Source: F Datasheet · p. 17
• PolarScale is intended for RGB-to-Stokes benchmarking, not calibrated radiometry, safety-critical decisions without target-sensor validation, or identifying people.Source: F Datasheet · p. 17
• The derived data remain subject to the source dataset’s license and terms of use [16].The package is distributed through the code repository identified in Appendix H.Source: F Datasheet · p. 17

G Additional qualitative results

• Across Figures 4–10 and 11–17, inputs are normalized total-intensity images s0, with outputs shown for DiT [35], Uformer [32], I2ITurbo [37], and MAE [33].Source: G Additional qualitative results · p. 17

H Ethics statement and computational resources

• The work involves no human subjects, and the data contain no annotations of people or other sensitive personal data, though outdoor scenes may incidentally show vehicles or bystanders.All data are used solely for research purposes.Source: H Ethics statement and computational… · p. 17
• Experiments used eight NVIDIA RTX A5000 GPUs; training took about one day per restoration model and two days per diffusion-based model.Code, checkpoints, configuration files, and the benchmark package are publicly available.Source: H Ethics statement and computational… · p. 17

G Additional qualitative results

• For test scene 1050, MAE recovers S2 and S3 in the sky’s strong negative-S2 region but predicts the wrong S1 sign; other models produce nearly flat maps.Source: Figure 4 · p. 18
• For street-intersection scene 1158, models capture global layout but miss abrupt polarization transitions; DiT has noisy AoLP, I2ITurbo saturates DoCP, and MAE is closest to ground truth.Source: Figure 11 · p. 20