PaperlayerView exact revision on arXiv

Source-linked AI summary

RangeNav: Task-Aligned LiDAR Range-Image Pretraining for Robot Navigation

Samuel Foo Enze, Manthan Patel, Fan Yang, Linus Kramer, Pascal Roth, Marco Hutter, Cesar Cadena

arXiv:2610.29845v2cs.ROcs.LG

TL;DR

Navigation in crowds and clutter calls for full-surround perception, but pretrained LiDAR representations are underexplored and large point-cloud encoders are costly for massively parallel RL. RangeNav pretrains a compact LiDAR range-image encoder with navigation-aligned objectives and freezes it for policy learning. It improves simulated success rates over reconstruction-only pretraining and demonstrates zero-shot navigation in challenging real-world settings.

  • Problem

    LiDAR offers full-surround perception, but pretrained LiDAR representations remain underexplored and point-cloud encoders are costly for massively parallel RL.

    Source: §I · p. 1

  • Method

    RangeNav pretrains a compact LiDAR range-image encoder on synthetic data with range reconstruction, semantic segmentation, and BEV prediction, then freezes it for policy learning.

    Source: §I · p. 2

  • Results

    32.03 and 23.73 percentage-point average success-rate gains over a reconstruction-only baseline in procedural training and out-of-distribution test environments, respectively.

    Source: §I · p. 2

  • Takeaways & Limitations

    Zero-shot deployment on a Unitree B2-W navigates dense crowds, low-light settings, and construction zones where limited-field-of-view depth policies struggle.

    Source: §I · p. 2

    Sparse sampling, projection, occlusions, and the deployed platform’s rear payload mask can leave fine, thin, or low obstacles unseen; dynamic obstacles are handled reactively without explicit tracking or motion prediction.

    Source: H. Limitations · pp. 7–8

Abstract

from arXiv · show

Autonomous mobile robots operating in dense crowds and cluttered sites require $360^\circ$ perception to react to surrounding hazards. While panoramic LiDAR offers full-surround coverage, learning representations within reinforcement learning (RL) is sample-inefficient under sparse rewards, and existing 3D pretrained point-cloud encoders are too memory-heavy for massively parallel RL simulation. We present RangeNav, a compact (183k parameters) LiDAR range-image encoder pretrained entirely on synthetic data. RangeNav uses anisotropic convolutions and circular padding to handle the extreme aspect ratio and azimuthal seam of range images. We pretrain the network using a navigation-aligned objective combining range reconstruction, semantic segmentation, and bird's-eye-view (BEV) prediction, then freeze it for policy learning. In simulation, adding semantic and BEV supervision increases success rates over a reconstruction-only baseline by an average of 32.03 and 23.73 percentage points (pp) in procedurally generated training and out-of-distribution realistic test environments, respectively. Zero-shot real-world deployments on a Unitree B2-W quadruped demonstrate robust navigation through dense crowds, low-light settings, and construction zones where limited-field-of-view depth policies struggle. Code and models will be open-sourced. Project website: https://rangenav.github.io/.

I. INTRODUCTION

RangeNav addresses the need for full-surround, efficient LiDAR representations for navigation by combining a compact range-image encoder with task-aligned pretraining. Its simulations and zero-shot deployments report advantages in settings where limited-field-of-view depth policies struggle.

I. INTRODUCTION

  • LiDAR offers 360° coverage, longer metric range, and illumination invariance, while pretrained LiDAR representations remain underexplored and large point-cloud encoders are costly for massively parallel RL.The desired encoder should be lightweight, reduce perception learning under sparse rewards, and generalize across scenes and to the real world.Source: §I · p. 1
  • RangeNav projects scans into panoramic range images and uses anisotropic kernels and circular convolutions, with reconstruction, semantic segmentation, and BEV prediction objectives.The frozen latent feeds an RL navigation policy.Source: §I · p. 2
  • 32.03 and 23.73 pp average success-rate gains over reconstruction-only pretraining in procedural training and out-of-distribution test environments, respectively.The LiDAR policy also surpasses a depth-based policy in crowded and densely cluttered environments.Source: §I · p. 2

II. RELATED WORK

  • Prior pretrained RGB and depth navigation representations target perspective images, so panoramic LiDAR requires handling its large aspect ratio, structured sparsity, and azimuthal wraparound.Camera-based auxiliary supervision is established, but complementary geometric and semantic objectives for LiDAR navigation have received little attention.Source: §II · p. 2
  • General-purpose point-cloud encoders operate on irregular 3D data, while projection-based methods make scans efficient 2D arrays; pretrained range encoders remain largely unexplored for RL navigation.Existing range-encoding systems are optimized and benchmarked for semantic perception.Source: §II · p. 2

III. METHOD

The method generates synthetic LiDAR range images, pretrains an encoder with reconstruction and navigation-relevant targets, then supplies its frozen spatial latents to a recurrent navigation policy.

III. METHOD

  • The three-stage pipeline projects synthetic depth observations into range images, pretrains a deterministic autoencoder with reconstruction, semantic, and BEV objectives, then trains a recurrent actor–critic policy on frozen encoder latents.Source: §III · p. 2

A. Range-Image Generation

  • Six depth images are fused in 3D and mapped into target LiDAR azimuth/elevation bins, with collisions resolved by minimum pooling, producing fixed-shape tensors for efficient batching.The data-processing overview shows the projection onto the spherical grid.Source: A. Range-Image Generation · p. 2, Figure 2 · p. 3

B. Range Preprocessing and Augmentation

  • Pretraining applies circular horizontal rolls and corruptions including scan-ring bias, range- and edge-dependent noise, dropout, and contiguous masking; the clean rolled scan is the reconstruction target.Ranges are clipped and log-transformed, emphasizing nearby geometry while compressing distant returns; corrupted inputs are paired with clean targets.Source: B. Range Preprocessing and Augmentation · pp. 2–3

C. Task-Aligned Pretraining

RangeNav pretrains a range-image encoder with geometric and semantic targets, using an architecture suited to panoramic scans. Three decoder heads provide complementary supervision during pretraining, while only the encoder is retained for policy learning.

C. Task-Aligned Pretraining

  • The encoder uses circular convolutions to preserve the 360° azimuthal seam, depthwise-separable residual blocks to reduce computation, and anisotropic kernels for wider horizontal receptive fields.Layer normalization is applied independently at each spatial location.Source: C. Task-Aligned Pretraining · p. 3
  • Three decoder heads predict clean range images, per-return semantic labels, and BEV occupancy and ground height from the encoder’s spatial latent.A fully connected bottleneck maps the polar latent to a Cartesian grid before BEV deconvolutions; the heads are discarded after pretraining.Source: Figure 3 · p. 3, C. Task-Aligned Pretraining · pp. 3–4
  • The BEV head predicts occupancy and ground height on a Cartesian grid, with mean absolute error measuring predicted versus target height over known cells.All three decoder heads serve only during pretraining; the encoder alone is retained for the RL policy.Source: C. Task-Aligned Pretraining · pp. 3–4
  • The range reconstruction objective combines an edge-weighted masked Huber loss with a gradient-matching loss.The gradient term compares adjacent horizontal and vertical differences where both target pixels are valid.Source: C. Task-Aligned Pretraining · p. 3
  • Semantic supervision uses pixel-wise cross-entropy and Lovász–Softmax on valid, nonself-occluded returns.The raw labels are collapsed into eight classes, and Lovász–Softmax is a differentiable surrogate for mIoU.Source: C. Task-Aligned Pretraining · pp. 3–4

D. Recurrent Navigation Policy

During policy learning, the pretrained LiDAR encoder remains frozen. Its spatial latent is combined with proprioception and the relative goal as input to a recurrent actor–critic policy.

D. Recurrent Navigation Policy

  • The frozen encoder’s spatial latent is combined with noisy proprioception and the relative goal before entering recurrent memory.The actor predicts control actions from the recurrent state, while an asymmetric critic may use privileged geometric observations during training.Source: D. Recurrent Navigation Policy · p. 4
  • The actor–critic is trained with MDPO in massively parallel Isaac Lab environments randomized across terrain, dynamics, sensing, actuation, delays, and disturbances.Episodes end on goal reach, timeout, unsafe contact, excessive roll or pitch, or a fall.Source: D. Recurrent Navigation Policy · p. 4
  • The reward combines sparse goal-reaching rewards with dense shaping for goal progress, smooth motion, and path efficiency.The policy uses recurrent memory to process its observations over time.Source: D. Recurrent Navigation Policy · p. 4

IV. EXPERIMENTS

The experiments assess pretraining objectives, compare panoramic LiDAR with front-facing depth, and compare the range-image encoder with general-purpose point-cloud encoders. They also ablate encoder design choices and include real-world deployments.

IV. EXPERIMENTS

  • The experiments evaluate how pretraining objectives affect navigation success rate.This question is addressed in Sec. IV-C.Source: §IV · p. 4
  • A separate comparison tests panoramic LiDAR against front-facing depth and the range-image encoder against general-purpose point-cloud encoders.These questions are assigned to Secs. IV-D and IV-E, respectively.Source: §IV · p. 4
  • The evaluation also ablates encoder design choices and performs real-world deployments.These are covered in Secs. IV-F and IV-G.Source: §IV · p. 4

A. Implementation Details

Implementation uses synthetic LiDAR range images, a compact encoder pretrained on a large frame set, and an actor–critic trained in parallel environments. Additional masking and stick-noise augmentation model structured sensor artifacts.

A. Implementation Details

  • Six fused TartanGround depth views simulate a 32-beam LiDAR with 70° vertical and 360° horizontal fields of view, producing 32 × 900 range images that are pooled to 32 × 180.Minimum pooling along azimuth preserves closest returns while reducing horizontal redundancy.Source: A. Implementation Details · p. 4
  • The encoder maps 1 × 32 × 180 inputs to 64 × 4 × 12 latents and is pretrained on about 144,000 frames for 5,000 epochs.Training uses Adam with batch size 1,024; the BEV target spans 40 × 40 m on a 256 × 256 grid.Source: A. Implementation Details · p. 4
  • The MDPO actor–critic trains in 1,920 parallel environments, with policies in Table I trained for 15,000 iterations and ablations for 5,000.The policy uses a one-layer 512-unit recurrent state followed by actor and critic MLPs.Source: A. Implementation Details · p. 4
  • Random range-image masking simulates robot-body or payload occlusions, while stick noise models persistent returns from thin poles, bars, and ring-like structures.These structured artifacts are not captured by independent Gaussian noise or dropout.Source: A. Implementation Details · p. 4

B. Evaluation Protocol

Evaluation spans training and held-out environments, while the pretraining comparison tests whether adding semantic segmentation and BEV prediction improves navigation over reconstruction alone.

B. Evaluation Protocol

  • The base mixture of mazes, pillar fields, stairs, and pits is used for RL training; TartanGround scenes and side-wall and dynamic-obstacle settings are held out for evaluation.Each evaluation uses 1,024 fixed start–goal scenarios per environment; 13 pedestrians move along random paths in the dynamic-obstacle setting.Source: B. Evaluation Protocol · p. 4

C. Task-Aligned Pretraining Objectives

  • The comparison tests reconstruction alone, reconstruction plus semantic segmentation, and adding BEV prediction as cumulative LiDAR pretraining objectives.Table I checks indicate which pretraining heads each encoder uses.Source: C. Task-Aligned Pretraining Objectives · pp. 4–5
  • The encoder supervised by segmentation and BEV objectives is the strongest LiDAR variant in every column, improving Pits success from 26.76% to 92.09% and AbandonedCable from 51.46% to 81.45%.The passage attributes the gains to semantic supervision distinguishing returns with different navigation implications and BEV supervision organizing features around free space and terrain layout.Source: C. Task-Aligned Pretraining Objectives · p. 5

D. Panoramic LiDAR versus Front-Facing Depth

Panoramic LiDAR slightly exceeds front-facing depth in average success across both evaluation sets, with particular benefits in settings where relevant geometry lies outside the forward view. The compact range-image encoder also trains at high throughput and outperforms variants distilled from general-purpose point-cloud encoders.

D. Panoramic LiDAR versus Front-Facing Depth

  • LiDAR has higher success in five of eight base-mixture and TartanGround environments, while depth leads on stairs, pillars, and Cable.The passage suggests denser depth observations may help with step geometry and thin or low-lying obstacles missed by sparse LiDAR beams.Source: D. Panoramic LiDAR versus Front-Facing… · p. 5, Table I · p. 6
  • LiDAR achieves slightly higher average success than depth on both evaluation sets despite sparser observations, and improves success from 79.98% to 88.87% in side-wall tests and from 77.64% to 79.49% with dynamic obstacles.Across base-mixture environments, LiDAR also scores 0.58 pp higher SPL and reaches goals 1.38 s faster on average.Source: D. Panoramic LiDAR versus Front-Facing… · p. 5, Table I · p. 6, Table II · p. 6

E. Comparison with General-Purpose Point-Cloud Encoders

  • The range-image encoder reaches 1,469 steps/s with 1,920 parallel environments, while point-cloud encoders reach only 19–75 steps/s during RL training.The comparison measures GPU-memory parallel capacity and training throughput on one NVIDIA RTX 4090.Source: E. Comparison with General-Purpose… · p. 6, Table III · p. 6
  • On base-mixture episodes, the task-aligned pretraining objective exceeds the best Sonata- and Utonia-distilled variants by 7.32 pp and 11.40 pp, respectively.The comparison distills teacher features into the same range-image encoder architecture.Source: E. Comparison with General-Purpose… · p. 6, Table IV · p. 6

F. Encoder Architecture and Ablations

The ablations compare encoder width, stem-kernel anisotropy, and architecture. They find that navigation performance favors the 3 × 9 stem and is not captured reliably by reconstruction loss alone.

F. Encoder Architecture and Ablations

  • Uniformly scaling the encoder’s six channel widths does not improve navigation SR.The tested widths are 32/48/64/96/128/160.Source: F. Encoder Architecture and Ablations · p. 6
  • The 3 × 9 stem performs best, supporting anisotropic kernels for range images with many more azimuthal than elevation pixels.The variants retain the same range-image representation and are evaluated on the base mixture.Source: F. Encoder Architecture and Ablations · p. 6
  • RegNet-FPN achieves lower reconstruction loss (0.02809 vs. 0.04685), but the smaller encoder obtains much higher SR, so reconstruction loss is not a reliable proxy for navigation.Source: F. Encoder Architecture and Ablations · p. 6

G. Real-World Deployments

The authors evaluate the simulation-trained navigation pipeline on a Unitree B2-W without real-world fine-tuning. Deployments span low-light scenes, crowds, and construction sites, demonstrating zero-shot sim-to-real transfer.

G. Real-World Deployments

  • Figure 6 shows scene photographs, point clouds and range images, alongside decoder outputs for range reconstruction, semantic segmentation, and BEV prediction.Its examples include pedestrians, a fence, and vehicles.Source: Figure 6 · p. 7, G. Real-World Deployments · p. 7
  • The deployed B2-W uses a RoboSense Helios 32-beam LiDAR and IMU, with the rear 120° sector masked because the payload box occludes the scan.BIEVR-LIO estimates pose from LiDAR and inertial measurements; the stack runs onboard an NVIDIA Jetson AGX Orin.Source: G. Real-World Deployments · p. 7
  • Successful rollouts in dark scenes, dynamic crowds, and cluttered construction sites demonstrate zero-shot transfer using the same perception–policy interface as in simulation.The mapless recurrent policy balances pedestrian avoidance with longer-horizon progress and avoids repeated attempts at impassable routes.Source: G. Real-World Deployments · pp. 6–7

H. Limitations

The limitations concern sensor visibility and scan-geometry dependence, as well as reactive handling of dynamic obstacles. The conclusion presents task-aligned range-image pretraining as a practical foundation for LiDAR navigation.

H. Limitations

  • Sparse sampling, projection, downsampling, and occlusions can hide fine structure or thin and low obstacles; the deployed rear mask creates a substantial blind sector.The fixed-size input is tied to the target LiDAR’s channel count, field of view, and angular resolution.Source: H. Limitations · p. 7
  • Dynamic obstacles are handled reactively without explicit tracking or motion prediction, making sudden interactions and dense crowds potential failure cases.The stated future work addresses cross-sensor adaptation and explicit motion prediction.Source: H. Limitations · p. 8

V. CONCLUSION

  • The conclusion presents compact, task-aligned range-image pretraining as a practical foundation for LiDAR navigation, with zero-shot transfer demonstrated in dark, crowded, and construction environments.It also reports better path efficiency, faster goal-reaching, and better performance in cluttered and dynamic-obstacle settings than a front-facing depth policy.Source: §V · p. 8

Related papers

Loading 2610.29845v2…