PaperlayerView exact revision on arXiv

Source-linked AI summary

FIRE-LIVWO: Robust LiDAR-Inertial-Visual-Wheel Odometry via Failure-Immune mmWave Radar Enhancement

Kun Hu, Menggang Li, Kaidi Wu, Zhiwen Jin, Yingjie Zhao, Chaoquan Tang, Eryi Hu, Gongbo Zhou

arXiv:2609.05325v1cs.RO

TL;DR

Underground mine SLAM is challenged by smoke-induced visual loss, degraded LiDAR structure, and drift in repetitive corridors. FIRE-LIVWO combines tightly coupled radar, LiDAR, vision, inertial, and wheel constraints with observability-based switching, achieving a 5.677 m average localization error in real mines.

  • Problem

    Smoke, dust, poor illumination, and self-similar corridors degrade visual and LiDAR constraints, causing odometry drift and failures in underground mine SLAM.

  • Method

    FIRE-LIVWO tightly couples LiDAR–radar and visual residuals in a unified VoxelMap, adds radar Doppler and wheel constraints, and switches modalities using online observability analysis.

  • Results

    5.677 m average localization error was achieved by FIRE-Full, outperforming all evaluated baselines.

  • Takeaways & Limitations

    Online switching down-weights unreliable vision in dense smoke, uses radar-dominant observations, and reactivates vision when image quality recovers.

Abstract

from arXiv · show

Achieving robust SLAM in large-scale underground coal mines with complex structures and severe degeneracies remains highly challenging. Dense smoke and dust cause substantial loss of visual information and degrade LiDAR point-cloud features, while long, self-similar corridors induce geometric degeneration, leading to pronounced odometry drift. To address these issues, we propose FIRE-LIVWO: Failure-Immune mmWave Radar-Enhanced LiDAR-Inertial-Visual-Wheel Odometry, a tightly coupled multi-modal odometry framework based on an iterated error-state Kalman filter (IESKF). The framework fuses 4D mmWave radar, LiDAR, and visual features within a unified VoxelMap and jointly constructs LiDAR-radar point-to-plane residuals and sparse visual photometric residuals. In smoke-filled environments, we exploit the strong penetration of 4D mmWave radar and introduce pointwise Doppler velocity constraints to preserve state observability. In geometrically degenerate corridors, we tightly couple wheel odometry using non-holonomic constraints (NHC) and online lever-arm compensation to reduce drift. Our central contribution is a degeneration detection and adaptive fusion model switching strategy grounded in geometric and visual observability analysis, which quantifies observability online and dynamically adjusts modality weights. Real-world experiments in underground coal mines demonstrate that FIRE-LIVWO accurately identifies failure boundaries, enabling reliable modality switching under extreme conditions. Compared with baselines, it achieves superior accuracy and robustness (average localization error of 5.677m). We open source our code on Github to benefit the robotics community.

I. INTRODUCTION

FIRE-LIVWO addresses heterogeneous visual and geometric degradation in underground mines with tightly coupled multimodal sensing, wheel constraints, and observability-based adaptive fusion. The paper reports real-world validation against existing approaches under challenging conditions.

  • Motivation: Underground mines combine smoke, dust, poor illumination, repetitive layouts, and weak corridor geometry that destabilize conventional LVI odometry.Visual degradation disrupts photometric consistency, while long straight corridors provide insufficient LiDAR structure and cause drift.
  • Method: FIRE-LIVWO jointly optimizes LiDAR–radar geometric and visual photometric residuals within a unified VoxelMap.The framework also integrates radar Doppler velocity constraints to improve robustness in smoke-filled environments.
  • Method: Wheel odometry uses non-holonomic constraints and adaptive online lever-arm compensation to reduce drift in long underground corridors.This module supplies proprioceptive motion information when exteroceptive geometry is degenerate.
  • Adaptive fusion: Online geometric and visual observability scores detect smoke interference and geometric underconstraints, enabling adaptive modality switching.The strategy is designed to maintain continuous state estimation across scenarios rather than relying on fixed fusion.
  • Validation: Real-world underground coal-mine experiments report superior accuracy and reliability against state-of-the-art approaches under challenging conditions.The paper also states that its code is publicly available on GitHub.

III. PRELIMINARY

The preliminary section defines the variable notation and discrete-time state-transition model used for the estimator. It assumes known sensor timing relationships and rigid sensor mounting.

  • Assumptions: LiDAR, IMU, camera, wheel odometer, and 4D mmWave radar are assumed to have known temporal offsets.The IMU frame is the body frame, and the global frame is defined at the total-station origin.
  • Assumptions: All sensors are rigidly mounted, while LiDAR, camera, and IMU are hardware-synchronized.
  • Notation: The state x is an 18-dimensional quantity, with generalized addition and subtraction operators used for state updates.The notation is expressed using x ∈ R18 and ⊞/⊟ operators.
  • State model: The estimator uses a discrete-time state-transition model evaluated at each IMU measurement.The input, process noise, and transition function follow prior definitions.

B. Error-State Iterated Kalman Filter Update

The IESKF update combines propagated state information with radar, LiDAR, visual, and wheel-odometry measurements in a MAP formulation. The system architecture organizes these constraints with adaptive degradation detection and fusion-mode switching for robust estimation.

  • The propagated state and covariance impose a prior distribution on the current state estimate.
  • The MAP update jointly incorporates radar Doppler, radar geometric, LiDAR, visual, and wheel-odometry measurements.
  • The residual formulation assigns covariance matrices and measurement counts to each sensor modality before iterative optimization.
  • The system architecture includes radar-enhanced LiDAR-visual-inertial constraints, vehicle kinematic constraints, and adaptive fusion-mode switching.
  • Visual and geometric observability scores identify heterogeneous degradation boundaries and adjust active residual terms in the IESKF.

V. METHODOLOGY

The methodology unifies LiDAR and radar geometric information with radar Doppler constraints and sparse visual map associations. These measurements are formulated as residuals for tightly coupled state estimation in challenging environments.

  • An adaptive octree voxel map stores planar features, LiDAR points, and visual map points sourced from LiDAR or radar.The root voxel size is 0.5 m, and visual points retain reference image patches.
  • Radar and LiDAR points are projected into the image plane to support sparse direct photometric errors and stable alignment under LiDAR degeneracy.
  • Radar and LiDAR points are jointly fused through geometric constraints and radar Doppler-velocity constraints.Radar geometric constraints support data association when LiDAR features degrade, while Doppler residuals provide complementary motion information.
  • LiDAR measurements are motion-compensated and modeled with point-to-plane residuals against matched map planes.The residual is zero when the transformed point agrees with its matched plane under the true state.
  • Radar feature points use analogous point-to-plane geometry with radar-to-IMU extrinsic calibration and radar measurement noise.The formulation defines the radar geometric residual alongside the LiDAR residual.
  • Radar points additionally produce Doppler-velocity residuals using Doppler measurements, angular velocity, radar–IMU extrinsics, and unit direction vectors.

3) Sparse Direct Photometric Residuals:

The sparse direct photometric model compares reference and current image patches after projecting map points into the current image. Its formulation accounts for patch offsets, camera projection, affine transformation, and image noise.

  • Visual map points are transformed into the current image, where the photometric error between reference and current patches is modeled as zero under the true state.
  • The model uses projections in the current and reference images together with the camera projection function and pixel offsets within each patch.
  • An affine transformation matrix models image-patch appearance changes, while image noise is included in the measurement model.

B. Vehicle Kinematic-Constraint Module

The vehicle kinematic module adds wheel-odometry information to address under-constrained motion in long, self-similar corridors. FIRE-LIVWO detects visual and geometric degradation and switches fusion modes by introducing complementary sensor constraints.

  • B. Vehicle Kinematic-Constraint Module: Wheel odometry supplies proprioceptive motion information independent of external features when LiDAR or visual constraints become under-constrained.This targets accumulated drift in long, self-similar tunnels.
  • B. Vehicle Kinematic-Constraint Module: The wheel constraint expresses velocity measurements through IMU and wheel frames with wheel-to-IMU extrinsics, angular velocity, and measurement noise.
  • C. Degradation Detection and Adaptive Fusion Mode Switching Module: The adaptive strategy introduces additional sensor constraints and switches fusion modes online to improve SLAM robustness.
  • C. Degradation Detection and Adaptive Fusion Mode Switching Module: Smoke and dust reduce image contrast and obscure texture information, constituting visual degradation.
  • C. Degradation Detection and Adaptive Fusion Mode Switching Module: LIV is used in nominal scenes, LIVR adds radar Doppler constraints under visual degradation, and LIVW adds wheel constraints under geometric degeneration.
  • C. Degradation Detection and Adaptive Fusion Mode Switching Module: The visual detector suppresses visual photometric constraints and strengthens radar constraints during smoke-induced low visibility.

1) Visual Degradation Detector:

The method detects visual and geometric degradation by quantifying image transmittance and point-cloud observability, then suppresses unreliable updates when constraints become under-constrained.

  • Visual degradation: Smoke degradation is modeled through transmittance, whose reduction lowers image contrast and usable visual information.Denser smoke yields smaller transmittance and fewer visible details.
  • Visual degradation: The visual observability score OV is the average estimated transmittance over sampled pixels, with larger values indicating more reliable photometric constraints.The score summarizes current-frame visual information quality.
  • Visual degradation: When visual degradation is detected, the system down-weights or disables photometric updates and relies on Doppler-velocity constraints for state estimation.Contrast loss and scattering-induced attenuation make sparse direct photometric residuals less reliable.
  • Geometric degradation: Geometric degradation is identified from weak Hessian eigen-directions, which reveal registration under-constraint and unreliable pose updates.The method analyzes rotational and translational Hessian submatrices derived from LiDAR and radar point-to-plane residuals.
  • Geometric degradation: The geometric observability score OG uses relative eigenvalue strength, while the smallest-eigenvalue eigenvectors identify degenerate translation or rotation directions.Typical cases include translation along a corridor or rotation about a specific axis.

3) Adaptive Fusion-Mode Switching:

The system switches fusion modes according to visual and geometric observability, activating radar Doppler and wheel-odometry constraints when degradation removes reliable information.

  • Adaptive fusion: Visual and geometric detectors compute OV and OG online to identify degradation type and severity before changing the active fusion constraints.The switching mechanism responds when an observability score crosses its corresponding threshold.
  • LIV mode: In nominal LIV mode, the estimator fuses LiDAR, visual, and IMU measurements when neither visual nor geometric degradation is detected.This mode applies in feature-rich corridor scenes with well-conditioned translational geometry.
  • LIVR mode: Under visual degradation, the system activates radar Doppler constraints and switches to LIVR mode.Radar supplies an additional velocity constraint when visual observations become unreliable.
  • LIVW mode: Under geometric degradation, the system activates wheel-odometry velocity constraints and switches to LIVW mode.Wheel measurements compensate for missing constraints along degenerate directions.
  • LIVRW mode: When visual and geometric degradation occur together, both radar and wheel constraints are activated in LIVRW mode.This mode is intended for long, straight corridors filled with dense smoke.

VI. EXPERIMENT

Real underground-mine experiments evaluate trajectory accuracy and adaptive switching across tunnels with smoke, sparse features, and repetitive geometry, comparing FIRE variants with established baselines.

  • Experimental setup: The platform is a Husky A200 equipped with LiDAR, camera, 4D mmWave radar, IMU, and wheel odometry, with total-station ground truth.Experiments use calibrated and time-synchronized sensors on an industrial PC.
  • Mapping results: Only FIRE-Full successfully reaches the end of Tunnel3; other methods fail or drift under smoke-induced visual degradation and geometric degeneration.The comparison includes substantial y- and z-axis drift for 4DRadarSLAM.
  • Experimental setup: The evaluation compares FIRE-Base, FIRE-LIVR, FIRE-LIVW, and FIRE-Full against FAST-LIVO2, R3LIVE, GaRLIO, and 4DRadarSLAM.The variants isolate the contributions of radar, wheel odometry, and their combination.
  • Experimental scenarios: Tunnel1 combines relatively rich features with severe smoke and dust, while Tunnel2 is smoke-free but feature-sparse and Tunnel3 is highly repetitive and extremely feature-sparse.These environments target visual degradation, weak LiDAR geometry, and repetitive-structure failure modes.

1) Trajectory Accuracy Analysis:

FIRE-Full maintains complete, accurate trajectories across smoke-induced visual degradation and geometrically degenerate corridors by detecting observability loss and adaptively switching fusion modalities. It achieves the strongest reported localization performance while avoiding the failures and drift observed in baseline methods.

  • Trajectory Accuracy Analysis:: FIRE-Full reaches the tunnel end by using radar to overcome smoke-induced visual failures and wheel constraints to suppress geometric degeneration.Baseline systems either fail after entering smoke or drift in the repetitive Tunnel3 corridor.
  • Trajectory Accuracy Analysis:: 5.677 m AvgErr: FIRE-Full substantially outperforms all baselines in average localization error.FIRE-LIVR and FIRE-Base report 15.420 m and 23.962 m, respectively; FIRE-LIVW and FAST-LIVO2 fail during the sequence.
  • Analysis of Degradation Detection and Adaptive Fusion-Mode Switching:: 114.2 geometric observability score: OG peaks at x = 143 m in the most severe corridor degeneration, where point-cloud registration is severely under-constrained.The smallest eigenvalue of Hpp becomes extremely small, and competing methods exhibit pronounced x-axis drift.
  • Analysis of Degradation Detection and Adaptive Fusion-Mode Switching:: 0.02 visual observability score: OV reaches its minimum at x = 25 m in densest smoke, making camera observations nearly invalid.FIRE-Full down-weights vision and switches to radar-dominant observations, then reactivates vision when OV recovers to 0.66 at x = 53 m.
  • Analysis of Degradation Detection and Adaptive Fusion-Mode Switching:: OG and OV reliably identify geometric under-constraints and visual failures, enabling two online fusion switches under mixed degradation conditions.The reported sequences are LIV→LIVR→LIV for visual degradation and LIV→LIVW→LIV for geometric degradation.

VII. CONCLUSION

FIRE-LIVWO is a tightly coupled multi-modal odometry framework for underground coal-mine environments with smoke, dust, and long-tunnel geometric degeneration. It unifies LiDAR, radar, vision, and wheel constraints with online observability-based fusion switching to maintain continuous state estimation.

  • VII. CONCLUSION: FIRE-LIVWO jointly fuses LiDAR–radar geometric residuals and sparse visual photometric residuals within a unified VoxelMap.The estimator uses a single filter update and is based on an IESKF.
  • VII. CONCLUSION: 4D mmWave radar Doppler-velocity constraints improve observability and resilience to visual failure in smoke-filled environments.The radar modality is integrated alongside LiDAR and vision in the tightly coupled formulation.
  • VII. CONCLUSION: Wheel odometry with non-holonomic constraints and online lever-arm compensation reduces drift caused by long, straight corridor degeneracy.The wheel module is tightly coupled into the estimator.
  • VII. CONCLUSION: Observability-based degradation detection dynamically switches fusion modes to maintain continuous and stable state estimation across scenarios.The strategy identifies smoke interference and geometric underconstraints online.
Loading 2609.05325v1…