PaperlayerView exact revision on arXiv

Source-linked AI summary

Memory-Aware Multi-Sensor Perception for Efficient and Safe Navigation in Dynamic Environments

Jingshuo Li, Yifan Xue, Yifei Li, Shubhodeep Shiv Aditya, Nadia Figueroa

arXiv:2609.30495v1cs.RO

TL;DR

Perception-based navigation can depend on prior maps or discard observed geometry beyond the local sensing horizon. The paper combines a persistent multi-sensor representation with stage-adaptive MCBF-QP control; in the reported comparison, its method reached a 100% success rate while maintaining a comparable human collision rate.

  • Problem

    Perception-based methods may rely on prior maps or discard previously observed geometry beyond the local sensing horizon, limiting use of explored structure.

    Source: §I · p. 1

  • Method

    The framework persistently represents static infrastructure, tracks dynamic obstacles, and adapts MCBF-QP safety constraints and geodesic guidance to local conditions.

    Source: §I · p. 2, E. Perception-Based Adaptive MCBF-QP · p. 5, 2) Stage Selection · p. 5

  • Results

    The proposed MCBF-QP achieved a 100% success rate while maintaining a comparable human collision rate; memory-unaware CE-CBF and MPC-CBF had success rates below 12.5%.

    Source: 4) Result Analysis · p. 7

  • Takeaways & Limitations

    The reported results indicate that persistent environmental information alone is insufficient for reliable target convergence, while the proposed method combines it with adapted guidance and parameters for efficient convergence.

    Source: 4) Result Analysis · p. 7

Abstract

from arXiv · show

Autonomous navigation in previously unseen environments requires effective perception, persistent environmental representation, and collision avoidance while maintaining progress toward a goal. Existing perception-based methods often rely on prior maps or short-horizon observations, limiting their ability to exploit previously observed structure. We propose a memory-aware multi-sensor navigation framework that integrates LiDAR and RGB perception, online distance-field representation learning, and a stage-adaptive Modulated Control Barrier Function Quadratic Program (MCBF-QP). The framework persistently represents static infrastructure while tracking dynamic obstacles, enabling the MCBF-QP controller to exploit previously observed geometry for obstacle circumvention and adapt its safety constraints and guidance to local conditions. Experiments in complex indoor and outdoor environments demonstrate improved navigation efficiency and goal-reaching performance while maintaining collision avoidance in narrow passages and around dynamic obstacles.

I. INTRODUCTION

Navigation in unseen environments must balance map-free sensing with the ability to use previously observed structure and keep progressing toward a goal. The paper proposes persistent multi-sensor representation and liveness-aware control, alongside an online distance-field representation.

I. INTRODUCTION

  • Persistent local sensing can discard geometry beyond the sensing horizon, while prior-map methods depend on maps that may be unavailable or unreliable after environmental change.Repeatedly reacting to local obstacles can lead to undesirable equilibria or revisiting explored regions; persistent static-infrastructure representations can inform current and subsequent tasks.Source: §I · p. 1
  • LiDAR-camera fusion combines LiDAR’s long-range geometry with RGB’s semantic information, but prior fusion work had not addressed complex indoor navigation involving multiple rooms and dynamic agents.The paper also notes that common perception-based controllers do not explicitly guarantee convergence without undesirable equilibria or dysfunctional circulation; MCBF-QPs target collision avoidance and navigation liveness.Source: §I · p. 1
  • The paper proposes map-free navigation that retains observed environmental structure and combines it with liveness-aware collision avoidance.Its contributions include a persistent online distance-field representation of static infrastructure that distinguishes dynamic obstacles, integrated with MCBF-QP control for obstacle circumvention.Source: §I · p. 2

A. Assumptions

  • The robot is modeled with control-affine dynamics, where x is the state, u the control action, and f and g are Lipschitz continuous.Source: A. Assumptions · p. 2

B. Gaussian Process Distance Field

  • The Gaussian Process Distance Field (GPDF) turns observed obstacle boundaries into a continuous, differentiable distance representation with an analytically computable gradient.The method applies an inverse transformation to the GP posterior mean; differentiability makes the resulting field suitable for constructing differentiable controller barrier functions.Source: B. Gaussian Process Distance Field · p. 2

C. Modulated Control Barrier Functions

The MCBF-QP combines collision avoidance with tangent guidance along an obstacle-boundary geodesic. For multiple nearby obstacles, the paper uses individual barrier functions to impose independent safety constraints.

C. Modulated Control Barrier Functions

  • The MCBF-QP adds tangent guidance along an estimated obstacle-boundary geodesic, requiring positive velocity in that direction to steer around obstacles and mitigate undesirable equilibria and dysfunctional circulation.The paper describes this controller as providing proven safety and convergence guarantees.Source: C. Modulated Control Barrier Functions · p. 2
  • For multiple nearby obstacles, the modified MCBF-QP replaces a unified barrier function with individual obstacle barrier functions, yielding independent constraints and a more conservative representation that better preserves clearance.Source: C. Modulated Control Barrier Functions · pp. 2–3
  • The formulation uses obstacle state and velocity, a class-K function, and a user-defined positive minimum velocity component along the guidance vector.Source: C. Modulated Control Barrier Functions · p. 3

III. METHODOLOGY

The framework places state estimation, multi-sensor perception, online representation learning, and stage-adaptive MCBF control in a shared reference frame. Its LiDAR and RGB branches work together to identify and track dynamic obstacles.

III. METHODOLOGY

  • The framework transforms observations into an online distance-field representation that supports collision-avoidance constraints and obstacle-circumventing guidance, with controller behavior adapted to dynamic obstacles, narrow passages, and open space.Source: §III · p. 3

A. Onboard State Estimation

  • FAST-LIO2 LiDAR-inertial pose and wheel velocity are fused with an extended Kalman filter, providing the shared-frame state and transforming sensor observations for persistent mapping.Wheel odometry supplies complementary short-term motion information when LiDAR scan registration is degraded.Source: A. Onboard State Estimation · p. 3

B. Dynamic Obstacle Detection

  • LiDAR geometry and RGB instance segmentation are cross-validated to reject static lookalikes, recover missed dynamic obstacles, and produce tracked obstacle positions and velocities.The LiDAR branch treats line-like clusters as likely static infrastructure and circle-like clusters as candidate dynamic obstacles; projected LiDAR points associate geometry with RGB instance masks.Source: B. Dynamic Obstacle Detection · pp. 3–4
  • The RGB mask centroid selects the projected LiDAR point used for an instance because a mask may include nearby objects at substantially different image depths.Source: B. Dynamic Obstacle Detection · p. 3

C. Occupancy Grid Map Construction

The occupancy grid uses LiDAR ray tracing for free and occupied space, then uses tracked dynamic obstacles to remove cells they have vacated.

C. Occupancy Grid Map Construction

  • Tracked dynamic-obstacle states clear vacated occupancy cells, preventing stale occupied cells from unnecessarily restricting control and potentially causing infeasible or suboptimal actions.Source: C. Occupancy Grid Map Construction · p. 4
  • LiDAR points are filtered and projected onto the horizontal plane, discretized into 360° beams, and ray-traced to update free and endpoint-cell log odds.Log odds are bounded, and cells above the occupancy threshold are classified as occupied.Source: C. Occupancy Grid Map Construction · p. 4
  • Cells covered by a tracked obstacle’s mask in the previous cycle but not the current one are reset to the minimum log odds, while standard updates remain in place for static obstacles.Source: C. Occupancy Grid Map Construction · p. 4

D. Online Environment Representation Learning

The module turns clustered occupancy observations into separate infrastructure and dynamic-obstacle representations, then converts them into distance fields for controller use.

1) Obstacle Clustering:

  • Occupied cells near the robot are clustered with DBSCAN, and marching-squares contours retain obstacle geometry while removing redundant interior cells.Source: 1) Obstacle Clustering · p. 4

2) Dynamic Obstacle Separation:

  • Tracked dynamic-obstacle states separate boundary points into dynamic and infrastructure sets by proximity, and undersized clusters are discarded as noise.Source: 2) Dynamic Obstacle Separation · p. 4

3) Distance Field Learning:

  • Fixed-cardinality point-set resampling supports reuse of JIT-compiled GPDF routines; oversized sets are uniformly sampled and undersized sets are augmented with Gaussian-noise perturbations.Source: 3) Distance Field Learning · p. 4
  • Infrastructure obstacles and sufficiently extensive dynamic obstacles receive separate GPDF representations, while small-scale dynamic obstacles are modeled as circles using estimated centers and radii.Source: 3) Distance Field Learning · pp. 4–5
  • The learned latent field is transformed into a distance field, with a small offset for numerical stability near obstacle boundaries.Source: 3) Distance Field Learning · p. 5

E. Perception-Based Adaptive MCBF-QP

The MCBF controller derives barrier constraints and geodesic guidance from the perceived environment, then adapts both to local navigation conditions.

  • The controller selects its collision-avoidance stage and corresponding parameters according to local conditions, adapting both safety constraints and geodesic guidance.Source: E. Perception-Based Adaptive MCBF-QP · p. 5

1) Perception-Based MCBF Formulation:

The perception-based formulation uses obstacle distance fields to construct barrier constraints and boundary-following guidance for obstacle circumvention.

1) Perception-Based MCBF Formulation:

  • The barrier-function derivative is constrained by ˙h(x,u) ≥ −α(h(x)), with f_other(x) accounting for state-dependent terms outside the distance field and δ denoting the safety margin.Source: 1) Perception-Based MCBF Formulation · p. 5
  • For dynamic obstacles, the barrier derivative includes relative obstacle velocity, while the system terms use the components governing position dynamics.Source: 1) Perception-Based MCBF Formulation · p. 5
  • Geodesic tangential guidance follows an isoline around the closest infrastructure cluster blocking the direct route, selecting among candidate directions by accumulated potential.Source: 1) Perception-Based MCBF Formulation · p. 5

2) Stage Selection:

The controller selects among dynamic-obstacle, narrow-space, and normal stages, adapting its constraints and guidance to local geometry and motion.

2) Stage Selection:

  • The stage adaptation addresses a trade-off: conservative barrier settings improve dynamic-obstacle clearance but can impede narrow passages, while boundary guidance can poorly represent the robot state in open space.Source: 2) Stage Selection · p. 5
  • A nearby dynamic obstacle approaching above the closing-speed threshold triggers the dynamic-obstacle stage.Source: 2) Stage Selection · p. 5
  • If that criterion is unmet, narrow-space selection uses infrastructure-cluster separation or a separate confinement test, excluding transient dynamic-obstacle geometry.Source: 2) Stage Selection · pp. 5–6
  • Confinement is detected when at least 90% of bearing bins are blocked; for a confined robot, a temporary target at the farthest detected opening facilitates departure.Source: 2) Stage Selection · p. 6

3) Stage-Dependent MCBF Adaptation:

The selected stage adjusts both CBF conservativeness and geodesic tangential guidance. In narrow spaces, the robot is projected onto the zero isoline to initialize the guidance.

3) Stage-Dependent MCBF Adaptation:

  • Dynamic-obstacle, normal, and narrow-space stages use progressively less conservative CBF settings through α(h) and the safety margin δ.The specified ordering is αdyn(h) ≤ αnormal(h) ≤ αnarrow(h), with δnarrow ≤ δnormal ≤ δdyn.Source: 3) Stage-Dependent MCBF Adaptation · p. 6
  • Dynamic-obstacle and normal stages evaluate geodesic guidance on the isoline corresponding to the robot’s current position.In narrow spaces, that isoline may exclude passages narrower than approximately 2hφ(xrob), so the robot state is projected onto the zero isoline and used as the initial state.Source: 3) Stage-Dependent MCBF Adaptation · p. 6
  • Stage-dependent adaptations reduce CBF conservativeness in confined regions while preserving obstacle-circumventing guidance and increased clearance from rapidly approaching dynamic obstacles.Source: 3) Stage-Dependent MCBF Adaptation · p. 6

IV. EXPERIMENTS

The experiments evaluate the proposed framework in simulation and on hardware, with simulation comparisons covering an ablation and three baseline methods. The design uses hospital and warehouse scenarios with dynamic human actors, and reports success, human collision rate, and trajectory tortuosity.

1) Experiment Design:

  • Six hospital scenarios and two smaller, less cluttered warehouse scenarios use 20 and three human actors per trial, respectively.Each scenario has fixed start and goal poses; each human follows an independent waypoint sequence without reacting to the robot, and trials randomize the human's initial position along that sequence.Source: 1) Experiment Design · p. 6

2) Ablation and Baselines:

  • The ablation retains the perception, obstacle representation, and adaptation pipeline but replaces MCBF-QP with standard CBF-QP.The baselines are MPC-CBF, DWA, and CE-CBF; MPC-CBF and CE-CBF do not retain previously observed environmental information, while DWA uses a memory-aware occupancy map.Source: 2) Ablation and Baselines · p. 6

IV. EXPERIMENTS

  • The evaluation compares the memory-aware MCBF method, an ablated CBF variant, and three baselines on target-reaching success, human collision rate, and trajectory tortuosity.Table I summarizes these measures; MPC-CBF was infeasible at the initial position, so its tortuosity is undefined.Source: 2) Ablation and Baselines · p. 6, Table I · p. 7

3) Metrics:

  • Figure 3 presents representative trajectories for the evaluated controllers in hospital Scenario 6.Source: Figure 3 · p. 7
  • Success rate measures goal-reaching trials, human collision rate measures collisions per human encounter, and tortuosity is path length divided by straight-line distance.An encounter is determined by ground-truth robot–human distance within three meters; higher tortuosity indicates more oscillation and detouring.Source: 3) Metrics · p. 7

4) Result Analysis:

The comparison indicates that persistent environmental information and liveness enhancement are complementary: MCBF-QP achieves a 100% success rate with comparable human collision rate, while approaches lacking either feature retain convergence weaknesses. Hardware trials show online mapping and goal-reaching in two unmapped settings; larger maps and dynamic-obstacle prediction remain future challenges.

4) Result Analysis:

  • MCBF-QP is the only compared method with a 100% success rate while maintaining a comparable human collision rate.The comparison also reports success rates below 12.5% for memory-unaware CE-CBF and MPC-CBF.Source: 4) Result Analysis · p. 7
  • Memory-aware Adaptive CBF and DWA achieve higher success rates (30% −50%), but without liveness enhancement remain susceptible to undesirable equilibria and limit cycles.The paper concludes that persistent environmental information alone is insufficient for reliable target convergence.Source: 4) Result Analysis · p. 7
  • Dynamic obstacles can push robots away from targets, causing long detours and tortuous trajectories; the proposed controller adapts guidance and parameters using mapped infrastructure.The framework combines persistent infrastructure mapping with modified MCBF-QP control to support efficient convergence.Source: 4) Result Analysis · p. 7

2) Navigation Experiment:

  • In two previously unmapped indoor and outdoor environments, online mapping accompanied successful goal-reaching amid narrow passages, moving pedestrians, and irregular terrain or structures.Fig. 4 overlays onboard-estimated trajectories on the resulting occupancy maps.Source: Figure 4 · p. 7, 2) Navigation Experiment · p. 7

1) Platform and Computing Architecture:

V. DISCUSSIONS AND FUTURE WORK

  • Larger occupancy maps may slow controller execution under limited onboard computing resources; future work includes adaptive memory management and more accurate obstacle-motion prediction.Source: §V · p. 8

Related papers

Loading 2609.30495v1…