PaperlayerView exact revision on arXiv

Source-linked AI summary

NIDAR: NIR-Guided Intrinsic Decomposition for Scalable Scene-Agnostic LiDAR Intensity Reconstruction

Junjie Zhang, Jie Yin, Kefei Qian, Jie Li, Mengpei Jia, Yajuan Dun, Wenbo Chu, Guofa Li

arXiv:2609.36878v1cs.RO

TL;DR

LiDAR simulation often omits intensity or reconstructs it using real intensity supervision and per-scene fitting. NIDAR synthesizes intensity-like observations from RGB and geometry through pseudo-NIR translation, intrinsic decomposition, and calibration. Reported comparisons show competitive pixel-wise accuracy and favorable structural and perceptual fidelity against evaluated reconstruction baselines.

  • Problem

    LiDAR simulation often omits intensity or uses reconstruction methods requiring real intensity supervision and per-scene optimization, limiting scalability across scenes.

    Source: §I · p. 1

  • Method

    NIDAR combines pseudo-NIR translation, intrinsic decomposition, geometry-aware modulation, and source-domain calibration to map RGB appearance and geometry to intensity-like observations without per-scene fitting.

    Source: §I · p. 1

  • Results

    NIDAR shows competitive pixel-wise accuracy and favorable structural and perceptual fidelity against evaluated reconstruction baselines, while pseudo-NIR is most beneficial through the reflectance-calibration route.

    Source: C. Ablation Studies · p. 6, §V · p. 8

  • Takeaways & Limitations

    NIDAR provides a reusable intensity-synthesis interface for the evaluated scenes and simulation settings without target-scene gradient-based fitting.

    Source: §I · p. 1

    Waymo evaluation uses only the roof-mounted top LiDAR, and unisolated sensor and environment differences preclude interpreting cross-dataset gaps as sensor-independent radiometric performance.

    Source: B. Results on Public Benchmarks · p. 5

Abstract

from arXiv · show

LiDAR return intensity provides complementary surface-response cues for robotic perception and state estimation, yet many simulation pipelines omit it or reproduce it using reconstruction methods that require real intensity supervision and per-scene optimization. These requirements increase data-collection and fitting costs and limit reuse across simulated scenes. We present NIDAR, a feed-forward framework that synthesizes dense intensity-like observations from RGB appearance and simulator geometry. NIDAR combines pretrained pseudo-NIR translation, hierarchical intrinsic decomposition, geometry-aware modulation, and source-domain distribution calibration to transfer reflectance-related image cues to simulated point clouds. Its learned components are trained offline using Waymo data; their weights and calibration remain fixed during evaluation on Waymo and nuScenes. Deployment therefore requires neither target-scene intensity labels nor target-scene gradient-based fitting. The reported comparisons show competitive pixel-wise accuracy and favorable structural and perceptual fidelity against the evaluated reconstruction baselines. A controlled pseudo-NIR-versus-RGB diagnostic further shows that the pseudo-NIR prior is most beneficial when used through the paper-aligned reflectance-and-remapping route, rather than as a simple direct intensity regressor. We further integrate NIDAR with Unreal Engine 5, Isaac Sim, and a generative LiDAR pipeline. Two intensity-aware SLAM systems evaluated in two simulated indoor scenes suggest potential downstream utility, but do not constitute real-robot validation. NIDAR therefore offers a scalable intensity-synthesis interface for the evaluated settings; cross-wavelength, camera-configuration, embedded, and real-sensor validation remain future work.

I. INTRODUCTION

LiDAR intensity carries surface-response cues useful for robotic perception, but simulation often omits it or relies on methods requiring real intensity supervision and per-scene fitting. NIDAR proposes a reusable, feed-forward synthesis interface combining spectral, intrinsic, geometric, and calibration components.

I. INTRODUCTION

  • LiDAR intensity reflects surface response and provides complementary material and physical cues that can benefit detection, segmentation, and SLAM.These uses motivate more faithful intensity modeling for robotic perception and simulation.Source: §I · p. 1
  • Existing simulators often omit or heuristically approximate intensity, while reconstruction methods may require real intensity supervision and per-scene optimization.Those requirements limit scalability when many scenes must be simulated.Source: §I · p. 1
  • NIDAR synthesizes pseudo-NIR from RGB, decomposes it into reflectance-related and shading components, then maps learned cues to 3D points using geometry-aware modulation and distribution calibration.Its output is intensity-like, not a sensor-independent reconstruction of absolute returned power.Source: §I · pp. 1–2
  • The framework is designed for reuse without scene-specific network fitting in evaluated settings, while retaining offline training and source-domain calibration.The learned weights remain fixed during evaluation.Source: §I · p. 1
  • Evaluations span Waymo and nuScenes, three simulation pipelines, a controlled pseudo-NIR-versus-RGB diagnostic, and two intensity-aware SLAM systems in simulated indoor scenes.The reported SLAM cases indicate potential downstream utility, not real-robot validation.Source: §I · p. 2, Figure 1 · p. 2

II. RELATED WORK

Prior LiDAR simulation work spans graphics engines, reconstruction from real scans, and generative data models. These approaches address geometry, ray behavior, or data generation, while intensity modeling and reuse across scenes remain limitations in the described methods.

A. Graphics Engine-Based LiDAR Simulation

  • Engine-based simulators focus mainly on geometric fidelity and ray behavior, while omitting intensity or approximating it heuristically.Graphics engines simulate scanning through ray casting and may partially model noise and raydrop effects.Source: A. Graphics Engine-Based LiDAR… · p. 2

B. LiDAR Reconstruction from Real-world Data

  • Reconstruction-based approaches include neural implicit representations and Gaussian rendering, but many rely on real intensity measurements or per-scene optimization.This dependence limits scalability beyond captured scenes.Source: B. LiDAR Reconstruction from… · p. 2

C. LiDAR Simulation Based on Data Generation

  • Generative LiDAR methods use GANs, VAEs, and diffusion models, but often treat intensity as a learned feature rather than decomposing reflectance-, geometry-, and sensor-related factors.NIDAR is presented as using RGB appearance with available geometry and no per-scene network-parameter optimization, while its components and calibration are trained offline on source-domain intensity.Source: C. LiDAR Simulation Based on Data… · p. 2
  • NIDAR’s pipeline combines pseudo-NIR synthesis, intrinsic and dense-intensity prediction under sparse projected-LiDAR supervision, then calibrated projection, geometry-aware modulation, and quantile remapping.The overview assigns intensity values to visible 3D points.Source: Figure 2 · p. 3

III. METHODOLOGY

NIDAR formulates intensity synthesis from RGB imagery and 3D geometry as transferring reflectance-related image cues to visible points. The formulation distinguishes that latent cue from calibrated LiDAR return intensity and accounts for additional variation through geometry-aware modulation and calibration.

A. Problem Formulation

  • The task takes RGB images with camera calibration and a 3D point cloud with camera–LiDAR calibration, and outputs visible points augmented with intensity-like values.NIDAR synthesizes pseudo-NIR, decomposes it into reflectance-related and shading components, then transfers a learned intensity representation through calibrated projection and geometry-aware modulation.Source: A. Problem Formulation · p. 3
  • In the intrinsic image model, pseudo-NIR appearance is represented by reflectance-related latent R and shading S, but R is not calibrated LiDAR return intensity.Measured returns also depend on sensor gain, material and wavelength, incidence angle, geometric attenuation, and measurement noise.Source: A. Problem Formulation · p. 3
  • NIDAR uses R as a cue and applies geometry-aware modulation and calibration to account for part of the remaining variation, without claiming sensor-independent absolute-return reconstruction.Source: A. Problem Formulation · p. 3

B. NIDAR Framework Overview

NIDAR uses offline-learned components to transfer pseudo-NIR appearance cues into pointwise intensity-like values, with weights fixed and no scene-specific gradient optimization during evaluation.

B. NIDAR Framework Overview

  • NIDAR combines pretrained pseudo-NIR synthesis, intrinsic decomposition and dense intensity prediction, then calibrated projection, geometry-aware modulation, and quantile remapping.The learned components and source-domain calibration are obtained offline; weights remain fixed during evaluation.Source: B. NIDAR Framework Overview · p. 3

C. Cross-spectral Pseudo-NIR Prior

  • The pretrained Spectral Translation Network converts visible RGB appearance into a single-channel pseudo-NIR image to reduce the spectral gap to LiDAR returns.The STN is used without fine-tuning in NIDAR.Source: C. Cross-spectral Pseudo-NIR Prior · p. 3

D. IRNet: Intrinsic Reflectance Network

  • IRNet estimates reflectance R and shading S from pseudo-NIR, using intrinsic-image constraints, edge-aware reflectance smoothness, and a low-frequency shading prior.A mean regularizer stabilizes R's global scale, while a DeepLabV3-style head predicts dense intensity under sparse projected-LiDAR supervision.Source: D. IRNet: Intrinsic Reflectance Network · pp. 3–4

E. Distribution Calibration via Quantile Remapping

NIDAR calibrates reflectance-related point values with distance-dependent modulation and a source-domain quantile mapping held fixed during scene evaluation.

E. Distribution Calibration via Quantile Remapping

  • Calibration first applies distance-dependent linear modulation, then maps the reflectance-like R branch through source-domain quantile remapping before point assignment.The mapping is learned offline and applied without scene-specific gradient optimization; unobserved points remain zero.Source: E. Distribution Calibration via… · p. 4
  • Returns with branch value i at or below threshold τ are treated as diffuse with Rmin = 0; higher values are treated as retroreflective with Rmin = rretro.The modulated value is clipped to [0,1] before quantile mapping.Source: E. Distribution Calibration via… · p. 4

F. Point Cloud Intensity Assignment

Point assignment projects LiDAR-frame points into camera images using the dataset-specific calibration and samples image outputs at the projected locations.

F. Point Cloud Intensity Assignment

  • The projection uses camera-to-ego extrinsics and a pinhole camera model expressed in the Waymo coordinate convention, with camera intrinsics determining image coordinates.Dataset-specific extrinsics are applied before projection, without adding a learnable component.Source: F. Point Cloud Intensity Assignment · p. 4
  • The nuScenes and Waymo tables report quantitative comparisons with reconstruction methods, but the supplied passages do not provide their cell values.The table captions identify dataset and comparison scope only.Source: Table II · p. 5, Table III · p. 5

A. Experimental Setup

The experiments evaluate fixed-weight NIDAR on nuScenes and Waymo against reconstruction baselines, alongside a matched pseudo-NIR-versus-RGB diagnostic.

A. Experimental Setup

  • IRNet and source-domain quantile remapping are trained on 17 Waymo scenarios, while all learned weights and calibration remain fixed during scene evaluation.The pretrained pseudo-NIR stage is not fine-tuned.Source: A. Experimental Setup · p. 5
  • Evaluation uses nuScenes and Waymo intensity ground truth, compares three reconstruction baselines, and reports MedAE, RMSE, PSNR, SSIM, and LPIPS.NIDAR performs no gradient-based fitting on evaluation scenes, whereas the scene-specific baselines use consecutive frames or point clouds for optimization.Source: A. Experimental Setup · p. 5
  • A matched diagnostic compares pseudo-NIR-input and RGB-input IRNet variants on held-out frames, measuring direct intensity and reflectance outputs before and after remapping.Remapping is applied only to R, because it was designed for reflectance-like outputs rather than the direct intensity branch.Source: A. Experimental Setup · p. 5

B. Results on Public Benchmarks

On nuScenes, NIDAR-Full shows clearer object-aligned transitions and more coherent local contrast than reconstruction baselines in the reported examples. Ablations favor a hierarchical U-Net+DeepLabV3 architecture and show pseudo-NIR is most beneficial through the reflectance-and-calibration route.

B. Results on Public Benchmarks

  • NIDAR-Full shows clearer object-aligned transitions and more coherent local contrast than reconstruction baselines in the reported nuScenes examples.The examples compare corresponding spatial locations across methods.Source: B. Results on Public Benchmarks · p. 5, Figure 3 · p. 6

C. Ablation Studies

  • U-Net+DeepLabV3 achieves the best overall performance among the tested intrinsic-backbone and intensity-head configurations.The combinatorial ablation varies the backbone, head, and convolutional-head depth, using three held-out Waymo test scenes.Source: C. Ablation Studies · p. 6, Table IV · p. 6
  • 0.02044 vs. 0.02049: RGB and pseudo-NIR have nearly identical sparse-intensity validation losses, with RGB slightly lower.The controlled variants share architecture, objective, optimization settings, and frame-camera pairs.Source: C. Ablation Studies · p. 6, Table V · p. 6
  • Pseudo-NIR benefits the reflectance-decomposition route rather than serving as a universal shortcut for direct intensity regression.For the reflectance-related representation, pseudo-NIR improves MAE, RMSE, and SSIM before remapping and yields the strongest structural score after remapping while retaining lower MAE/RMSE than the RGB route.Source: C. Ablation Studies · p. 6

D. Applications

NIDAR augments simulated point clouds across graphics-engine and generative LiDAR pipelines, with the reported simulated SLAM runs favoring NIDAR intensity among successful runs. These applications demonstrate compatibility and potential utility, not physical sensor fidelity or real-robot gains.

D. Applications

  • UE5 and Isaac Sim point clouds are augmented with NIDAR intensity patterns, shown as normalized intensity magnitude rather than semantic classes.The rendered-scene example also shows the raw simulated cloud before augmentation.Source: D. Applications · p. 6, Figure 5 · p. 7, Figure 6 · p. 7
  • Fig. 4 compares IRNet and IIW-CRF on MIT Intrinsic using gradient error, where lower values indicate better texture and edge reconstruction.Gradient error is the absolute difference between predicted and ground-truth gradient magnitudes.Source: Figure 4 · p. 7, Table VI · p. 7
  • The rendered-scene examples demonstrate pipeline compatibility rather than quantitative evidence of absolute physical or sensor fidelity.Matched real-LiDAR ground truth is unavailable for these scenes.Source: D. Applications · pp. 6–7
  • NIDAR post-processes LiDAR-Diffusion outputs by assigning RGB-derived intensity to camera-visible generated points through calibrated projection, without per-scene NIDAR optimization.The described integration preserves the generated geometry.Source: D. Applications · p. 7, Figure 7 · p. 7

E. Downstream SLAM Evaluation

  • Among successful simulated runs, NIDAR has the lowest ATE for both backends in both scenes; Hospital ISC-LOAM falls 3.2578 to 1.9187 m, versus Intensity-SLAM 0.1100 to 0.1085 m.In Warehouse, NIDAR enables Intensity-SLAM to complete and reduces ISC-LOAM ATE from 4.1218 to 2.3504 m.Source: E. Downstream SLAM Evaluation · p. 7, Table VII · p. 8

V. CONCLUSION

NIDAR synthesizes dense LiDAR intensity-like observations from RGB appearance and simulator geometry without per-scene network optimization in the evaluated settings. Results support cross-scene transfer within the evaluated datasets and simulators, while the conclusion identifies broader validation as future work.

V. CONCLUSION

  • NIDAR combines spectral translation, intrinsic decomposition, geometry-aware modulation, and distribution calibration to synthesize intensity-like observations from RGB appearance and simulator geometry.The evaluated settings require no per-scene network optimization.Source: §V · p. 8
  • Experiments on Waymo and nuScenes show competitive pixel-wise accuracy and favorable structural and perceptual fidelity in the reported comparisons.The controlled diagnostic finds pseudo-NIR most beneficial through the reflectance-calibration route, even when direct RGB and pseudo-NIR heads fit sparse intensity supervision similarly.Source: §V · p. 8
  • The evidence supports cross-scene transfer within evaluated datasets and simulators, not universal sensor-independent radiometric reconstruction; future work includes camera and wavelength robustness, real-sensor validation, embedded deployment, and broader downstream evaluation.Source: §V · p. 8

Related papers

Loading 2609.36878v1…