PaperlayerView exact revision on arXiv

Source-linked AI summary

DynaVINS: A Visual-Inertial SLAM for Dynamic Environments

Seungwon Song, Hyungtae Lim, Alex Junho Lee, Hyun Myung

arXiv:2208.11500v1cs.RO

TL;DR

Dominant dynamic objects and temporarily static objects challenge visual-inertial SLAM, whose motion priors alone may not handle severe occlusion and whose loop closures can be persistently false. DynaVINS combines robust bundle adjustment with grouped, multi-hypothesis loop-closure constraints. Experiments report better performance than other algorithms in simulations and actual environments with various dynamic objects.

  • Problem

    Dominant dynamic objects can defeat motion-prior-only handling, while temporarily static objects can cause persistent false-positive loop closures.

    Source: §I · p. 1

  • Method

    DynaVINS uses robust bundle adjustment to reject dynamic features against an IMU motion prior and groups loop-closure constraints into multiple hypotheses.

    Source: §I · p. 2

  • Results

    Experiments corroborated that DynaVINS works better than other algorithms in simulations and actual environments with various dynamic objects.

    Source: §VI · p. 8

  • Takeaways & Limitations

    The paper presents DynaVINS as robust against both dynamic and temporarily static objects.

    Source: §I · p. 2

Abstract

from arXiv · show

Visual inertial odometry and SLAM algorithms are widely used in various fields, such as service robots, drones, and autonomous vehicles. Most of the SLAM algorithms are based on assumption that landmarks are static. However, in the real-world, various dynamic objects exist, and they degrade the pose estimation accuracy. In addition, temporarily static objects, which are static during observation but move when they are out of sight, trigger false positive loop closings. To overcome these problems, we propose a novel visual-inertial SLAM framework, called DynaVINS, which is robust against both dynamic objects and temporarily static objects. In our framework, we first present a robust bundle adjustment that could reject the features from dynamic objects by leveraging pose priors estimated by the IMU preintegration. Then, a keyframe grouping and a multi-hypothesis-based constraints grouping methods are proposed to reduce the effect of temporarily static objects in the loop closing. Subsequently, we evaluated our method in a public dataset that contains numerous dynamic objects. Finally, the experimental results corroborate that our DynaVINS has promising performance compared with other state-of-the-art methods by successfully rejecting the effect of dynamic and temporarily static objects. Our code is available at https://github.com/url-kaist/dynaVINS.

I. INTRODUCTION

Visual SLAM commonly assumes static landmarks, while dominant dynamic objects and temporarily static objects challenge visual-inertial systems in different ways. DynaVINS addresses both with robust bundle adjustment and multi-hypothesis loop-closure constraints.

I. INTRODUCTION

  • Dominant dynamic objects can occlude most of the view, a case that visual-inertial motion priors alone cannot solve.The introduction also notes that geometry-based methods require accurate poses and deep-learning methods are limited to predefined object types.Source: §I · p. 1
  • Temporarily static objects can produce persistent false-positive loop closures, beyond the instantaneous false positives targeted by prior robust back ends.They appear static while observed but move when out of sight.Source: §I · p. 1
  • DynaVINS combines robust bundle adjustment with selective global optimization to handle dynamic and temporarily static objects.Its contributions include rejecting dynamic-object features that deviate from an IMU motion prior and grouping loop-closure constraints into multiple hypotheses.Source: §I · p. 2

A. Visual-inertial SLAM

  • Visual-inertial SLAM integrates IMU data with visual sensing, but existing methods still have potential limitations with dominant dynamic and temporarily static objects.The introduction situates visual sensors as relatively low-cost and lightweight options that provide rich information for SLAM.Source: §I · p. 1, A. Visual-inertial SLAM · p. 2

B. Dynamic Objects Rejection in Visual and VI SLAM

Prior approaches address dynamic features and false loop closures with distinct assumptions and limits. DynaVINS applies Black-Rangarajan duality in both bundle adjustment and graph optimization, using IMU priors and feature tracking information.

B. Dynamic Objects Rejection in Visual and VI SLAM

  • Geometry-based dynamic-object rejection depends on accurate camera poses, while deep-learning methods require predefined object types and may miss partially visible objects.Wheel-encoder approaches also depend strongly on the encoder, limiting their applicability.Source: B. Dynamic Objects Rejection in Visual… · p. 2

C. Robust Back-End

  • Robust kernels for loop closures are not suited to persistent false positives from temporarily static objects, and prior B-R methods do not address repeated false closures or VI-SLAM bundle adjustment.Max-mixture and switchable constraints target incorrect loop closures; cited B-R approaches address outliers in registration and pose-graph optimization.Source: C. Robust Back-End · p. 2
  • DynaVINS adopts Black-Rangarajan duality in both bundle adjustment and graph optimization, reflecting IMU priors and feature-tracking information.The pipeline tracks features in mono or stereo images, preintegrates IMU data, rejects dynamic features, groups keyframes, and clusters loop closures into weighted hypotheses for selective optimization.Source: C. Robust Back-End · p. 2, Figure 2 · p. 3

A. Notation

  • The notation defines the moving-window feature set and, for each tracked feature, the set of camera frames in which it appears.The full state vector includes keyframe poses and velocities, IMU biases, and estimated feature depths.Source: A. Notation · p. 3

B. Conventional Bundle Adjustment

Conventional visual-inertial bundle adjustment combines marginalization, IMU, and visual reprojection residuals, but conventional robust losses have distinct outlier limitations. The proposed B-R-inspired regularization assigns feature weights and uses alternating optimization to reduce the influence of high-error features.

B. Conventional Bundle Adjustment

  • Huber loss struggles when outliers are numerous, while redescending losses can permanently reject features that actually come from static objects.The latter occurs when those features are treated as outliers and the loss has zero gradients beyond a range.Source: B. Conventional Bundle Adjustment · p. 3

C. Regularization Factor

  • The proposed B-R-inspired loss weights visual reprojection residuals, using a regularization factor to reduce the influence of features with high reprojection error.The formulation replaces the Huber norm in the visual reprojection term; the feature weights lie in [0, 1].Source: C. Regularization Factor · p. 3
  • Alternating optimization first updates feature weights with the state fixed, then optimizes the state with the weights fixed.Each feature weight can be optimized independently.Source: C. Regularization Factor · p. 3
  • The adaptive factor draws on both Huber and Geman-McClure losses, filtering outliers without entirely ignoring them at the outset.During optimization, outlier-feature weights fall and their losses flatten toward zero gradient.Source: C. Regularization Factor · p. 4

D. Weight Momentum Factor

The weight momentum factor addresses cases where aggressive motion makes IMU preintegration imprecise and feature weights unreliable. It uses previously estimated weights to make current weights tend to persist, increasingly so for features tracked longer.

D. Weight Momentum Factor

  • Aggressive motion can make IMU preintegration imprecise, causing static-feature reprojection residuals to rise so regularization may ignore them despite previously high weights.Simply increasing λw can instead admit dynamic-object features with high reprojection residuals.Source: D. Weight Momentum Factor · p. 4
  • The momentum factor makes current feature weights tend toward their previous values, with stronger persistence as a feature is tracked more times.A tracked outlier with a previous weight of zero has less effect on bundle adjustment as its track length increases.Source: D. Weight Momentum Factor · pp. 4–5
  • For stereo, weights apply to movement-affected reprojections, while the same-keyframe stereo reprojection is used only for depth estimation.The text also distinguishes reprojections in another keyframe from the stereo reprojection.Source: D. Weight Momentum Factor · p. 5

IV. SELECTIVE GLOBAL OPTIMIZATION

Moving-window optimization causes cumulative trajectory drift, making loop closure necessary to optimize all trajectories. Temporarily static objects can produce false-positive loop closures that risk SLAM failure, motivating a method that rejects false closures while retaining true ones.

IV. SELECTIVE GLOBAL OPTIMIZATION

  • Because VIO optimizes only within a moving window, trajectory drift accumulates and loop closure is needed to optimize all trajectories.The passage gives DBoW2 as an example of loop-closure detection.Source: §IV · p. 5
  • Temporarily static objects can create false-positive loop closures, which may cause the SLAM framework to fail.These objects are static while observed but move when unobserved.Source: §IV · p. 5
  • The method aims to eliminate false-positive loop closures while preserving true ones, including when both temporarily static and static-object features occur in one keyframe.The two feature types may coexist in that keyframe.Source: §IV · p. 5

A. Keyframe Grouping

Keyframe grouping first joins adjacent frames that share at least a minimum number of tracked features. Loop closures from the same features are then grouped across keyframes, using one weight per group to speed optimization.

A. Keyframe Grouping

  • Loop closures from the same features are grouped even when they originate in different keyframes, with one weight per group for faster optimization.Source: A. Keyframe Grouping · p. 5
  • Adjacent keyframes are grouped when they share at least a minimum number of tracked features.Source: A. Keyframe Grouping · p. 5
  • A group starting at frame C_i consists of features tracked from C_i through C_k, with the feature-count threshold determining its extent.The text denotes this group as G_i.Source: A. Keyframe Grouping · p. 5

B. Multiple Hypotheses Clustering

The method groups related keyframes, generates candidate loop-closure matches, and clusters their estimated poses into competing hypotheses. This reduces the influence of temporarily static features whose old positions can produce inaccurate poses.

B. Multiple Hypotheses Clustering

  • Up to three similar earlier keyframes are matched to each keyframe in the current group to estimate relative poses.Keyframes without a similar match are skipped.Source: B. Multiple Hypotheses Clustering · p. 5
  • Loop closures with similar estimated poses are clustered by Euclidean distance, forming hypotheses that may yield different optimized trajectories.The pose estimates can be inaccurate when matching relies on a temporarily static feature at its previous position.Source: B. Multiple Hypotheses Clustering · p. 5, Figure 5 · p. 5
  • The two hypotheses with the most loop closures are retained, but this selection does not yet distinguish true from false positives.The method defers that distinction to a later step.Source: B. Multiple Hypotheses Clustering · p. 5

C. Selective Optimization for Constraint Groups

Selective optimization represents adjacent-keyframe links and loop closures as graph constraints, then assigns weights to competing hypotheses. It adopts a high-weight hypothesis and can suppress the loop-closure contribution when all hypotheses are false positives.

C. Selective Optimization for Constraint Groups

  • The graph optimization includes local edges between adjacent keyframes and loop-closure edges, with covariances specified for both pose types.The graph notation defines sets of keyframes, poses, loop closures, and weights.Source: C. Selective Optimization for… · pp. 5–6
  • For each group, the method represents two hypotheses with weights and applies Black-Rangarajan duality to the optimization.The loop-closure regularization includes a constant parameter, and weights are divided by each hypothesis's cardinality.Source: C. Selective Optimization for… · p. 6
  • Only a high-weight hypothesis is adopted; if all hypotheses are false positives, their weights can approach zero and prevent that failure from affecting optimization.Because keyframe poses change after optimization, hypothesis clustering is repeated for all groups before the next optimization.Source: C. Selective Optimization for… · p. 6

V. EXPERIMENTAL RESULTS

The evaluation compares DynaVINS with VINS-Fusion, ORB-SLAM3, and DynaSLAM across camera and inertial configurations. The authors note that DynaSLAM lacks an IMU and frame the comparison as a way to stress the value of an IMU in dynamic environments.

V. EXPERIMENTAL RESULTS

  • The evaluation compares DynaVINS with VINS-Fusion, ORB-SLAM3, and DynaSLAM.Source: §V · p. 6
  • The tested configurations include mono-inertial and stereo-inertial modes; DynaSLAM is tested only in stereo mode because it does not use an IMU.Source: §V · p. 6
  • The authors describe the DynaSLAM comparison as somewhat unfair, but use it to stress the necessity of an IMU in dynamic environments.Source: §V · p. 6

A. Dataset

The evaluation combines simulated dynamic-object sequences with sequences designed to test temporarily static objects, measuring trajectory error and degradation as dynamics increase. DynaVINS shows less performance degradation than other methods as dynamic objects increase, while the reported comparisons also describe failures under occlusion and false loop closures.

A. Dataset

  • The VIODE dataset is simulated and contains many moving objects, including scenes with dominant-object occlusion; the authors also collected four sequences for temporarily static-object loop closures.The E-shape sequence moves a checkerboard while unobserved, producing false-positive loop closures at the trajectory's three end vertices.Source: A. Dataset · p. 6

B. Error Metrics

  • Absolute Trajectory Error measures the difference between ground-truth points and the aligned estimated trajectory; VIODE degradation rate is ATE_high/ATE_none.Source: B. Error Metrics · p. 6

C. Evaluation on the VIODE Dataset

  • In the parking_lot high ablation, the authors report that adding the momentum factor to regularization improves results over VINS-Fusion and reduces time relative to using regularization alone.They say the increased BA time from added optimization steps remains sufficient for high-level robot control.Source: C. Evaluation on the VIODE Dataset · p. 6, Table I · p. 6
  • DynaSLAM tends to diverge or show large ATE as dynamic objects increase, with overall occlusion linked to segmentation failure and loss of static features.Source: C. Evaluation on the VIODE Dataset · p. 6
  • In city_day sequences, the reported ATE comparison shows less performance degradation for DynaVINS than for other state-of-the-art methods.Figure 6 presents ATE results with a logarithmic y-axis.Source: Figure 6
  • Across scenes and both mono-inertial and stereo-inertial modes, DynaVINS shows small performance degradation as dynamic objects increase and stable pose estimation where other algorithms are affected.The authors attribute this robustness to adaptive feature weighting in the robust bundle adjustment.Source: Table III · p. 7, C. Evaluation on the VIODE Dataset · p. 7, Figure 7 · p. 7

D. Evaluation on Our Dataset

DynaVINS performed better than other algorithms in simulations and actual environments with dynamic objects. On the authors’ dataset, it handled cases involving occlusion and persistent false loop closures where other methods faltered.

D. Evaluation on Our Dataset

  • In the static case, all algorithms had low ATE values, validating the dataset sequence.Source: D. Evaluation on Our Dataset · p. 8
  • In Dynamic follow, other algorithms tracked the occluding object, triggering bundle-adjustment failures and false-positive loop closures; their ATEs were higher than DynaVINS’s.Source: D. Evaluation on Our Dataset · p. 8
  • In Temporal static, VINS-Fusion with Switchable Constraints could reject false-positive loop closures, but DynaVINS performed better.ORB-SLAM3 and VINS-Fusion rejected them in the stereo-inertial case but not the mono-inertial case, where inaccurate depth estimation was cited.Source: D. Evaluation on Our Dataset · p. 8
  • In E-shape, other algorithms failed to optimize the trajectory because of false-positive loop closures, while DynaVINS rejected them by optimizing hypothesis weights.This rejection did not depend on the number of false-positive loop closures.Source: D. Evaluation on Our Dataset · p. 8

VI. CONCLUSIONS

  • DynaVINS worked better than other algorithms in simulations and actual environments with various dynamic objects.Source: §VI · p. 8
  • The authors plan to improve DynaVINS’s speed and performance and apply its concept to LiDAR-Visual-Inertial SLAM.Source: §VI · p. 8
Loading 2208.11500v1…