PaperlayerView exact revision on arXiv

Source-linked AI summary

EAO-SLAM: Monocular Semi-Dense Object SLAM Based on Ensemble Data Association

Yanmin Wu, Yunzhou Zhang, Delong Zhu, Yonghui Feng, Sonya Coleman, Dermot Kerr

arXiv:2004.12730v2cs.ROcs.CV

TL;DR

Object-level data association and pose estimation in semantic SLAM lack robust, accurate algorithms. EAO-SLAM combines parametric and nonparametric association with outlier-robust pose estimation and monocular object mapping; the authors report improved association and accurate object-oriented maps in experiments.

  • Problem

    Object-level data association and pose estimation in semantic SLAM lack robust and accurate algorithms.

    Source: §I · p. 1

  • Method

    EAO-SLAM ensembles parametric and nonparametric association tests and uses iForest-based outlier-robust object pose estimation and initialization.

    Source: §I · p. 1, §V · pp. 4–5

  • Results

    EAO-SLAM significantly outperforms in association, with almost twice as many successfully associated objects on the TUM dataset, while the advantage is not obvious on Microsoft RGB-D and Scenes V2.

    Source: B. Ensemble Data Association Experiments · p. 6

  • Takeaways & Limitations

    Experiments show that EAO-SLAM builds accurate object-oriented maps with object poses and scales accurately registered.

    Source: §VII · p. 7

Abstract

from arXiv · show

Object-level data association and pose estimation play a fundamental role in semantic SLAM, which remain unsolved due to the lack of robust and accurate algorithms. In this work, we propose an ensemble data associate strategy for integrating the parametric and nonparametric statistic tests. By exploiting the nature of different statistics, our method can effectively aggregate the information of different measurements, and thus significantly improve the robustness and accuracy of data association. We then present an accurate object pose estimation framework, in which an outliers-robust centroid and scale estimation algorithm and an object pose initialization algorithm are developed to help improve the optimality of pose estimation results. Furthermore, we build a SLAM system that can generate semi-dense or lightweight object-oriented maps with a monocular camera. Extensive experiments are conducted on three publicly available datasets and a real scenario. The results show that our approach significantly outperforms state-of-the-art techniques in accuracy and robustness. The source code is available on: https://github.com/yanmin-wu/EAO-SLAM.

I. INTRODUCTION

Semantic SLAM enriches conventional localization and mapping with environment labels and objects to support higher-level applications. EAO-SLAM addresses object association and pose-estimation challenges with an ensemble strategy and an outlier-robust pose framework, then builds lightweight object-oriented maps.

I. INTRODUCTION

  • Existing object-SLAM association methods struggle with complex scenes containing multiple instances, while monocular pose estimates remain inaccurate and prior improvements rely on strict assumptions.Those assumptions are described as difficult to fulfill in real-world applications.Source: §I · p. 1
  • The ensemble association strategy aggregates different object measurements to improve association accuracy.It integrates parametric and nonparametric statistical tests with an IoU-based method.Source: §I · p. 1
  • The iForest-based pose framework robustly estimates object locations, poses, and scales despite outliers.Its centroid-and-scale estimation and pose initialization are intended to improve pose-estimation optimality.Source: §I · pp. 1–2
  • EAO-SLAM builds lightweight object-oriented maps in which cubes or quadrics represent objects with registered locations, orientations, and scales.The paper presents this as an extension of object SLAM’s mapping capabilities.Source: §I · pp. 1–2

II. RELATED WORK

Prior data-association approaches trade off accuracy, generality, and efficiency, motivating an ensemble of parametric and nonparametric tests. Object-SLAM research uses varied object models and camera–object optimization strategies, while EAO-SLAM adds a novel initialization method.

A. Data Association

  • Probabilistic association can be accurate but handles few instances and incurs costly EM optimization, whereas tracking is efficient but can produce incorrect priors in complex environments.Feature- and descriptor-based approaches can also fail when their features are insufficiently general or robust.Source: A. Data Association · p. 2
  • Nonparametric tests handle non-Gaussian statistics, but do not effectively address Gaussian statistics; EAO-SLAM combines them with parametric methods.The paper motivates the ensemble as a way to exploit different measurements in complex scenes with multiple object categories.Source: A. Data Association · p. 2

B. Object SLAM

  • Object-SLAM models include instance-level, category-specific, and general representations, with general geometric models such as cubes and quadrics commonly used.Instance-level models require known object databases, limiting their application scenarios.Source: B. Object SLAM · p. 2
  • Prior joint camera–object optimization methods use object centroids or points, or projected quadrics and bounding-box scale errors; this work adds a novel initialization method.The paper states that its initialization can significantly improve solution optimality.Source: B. Object SLAM · p. 2

III. SYSTEM OVERVIEW

EAO-SLAM extends ORB-SLAM2 with semantic detection and an ensemble-association tracking thread, then uses iForest initialization and joint optimization to build object-oriented and semi-dense semantic maps. The following section’s notation defines point-cloud, rank, centroid, and statistical-function terms.

III. SYSTEM OVERVIEW

  • The system adds a YOLOv3 semantic thread to ORB-SLAM2 and combines bounding boxes, semantic labels, and point clouds for ensemble association in the tracking thread.iForest removes outliers to initialize joint object-pose and scale optimization with camera pose.Source: §III · p. 2
  • The optimized object map is combined with a semi-dense map to produce a semi-dense semantic map.The architecture figure highlights the system’s main contributions in red.Source: §III · p. 2, Figure 2 · p. 3

IV. ENSEMBLE DATA ASSOCIATION

  • The ensemble-association section defines notation for object point clouds, data-point rank, observed and historical centroids, and probability, mean, and variance functions.These terms establish the quantities used in the statistical tests.Source: §IV · p. 3

A. Nonparametric Test

The nonparametric test compares object point-cloud distributions, using the Wilcoxon Rank-Sum test to assess whether two clouds come from the same object. Its Mann-Whitney statistic is evaluated against a Gaussian confidence region.

A. Nonparametric Test

  • The Wilcoxon Rank-Sum test checks whether two object point clouds follow the same distribution, the null hypothesis for association.The method applies this nonparametric test because the point clouds are reported to follow a non-Gaussian distribution.Source: A. Nonparametric Test · p. 3
  • The test concatenates the clouds and sorts their points in each of three dimensions before forming rank statistics.The passage defines the concatenated cloud as X = [P|Q].Source: A. Nonparametric Test · p. 3
  • The Mann-Whitney statistic W is the minimum of the two rank statistics and is asymptotically Gaussian despite the non-Gaussian input clouds.The method calculates W's mean and variance for the subsequent test.Source: A. Nonparametric Test · p. 3
  • Association succeeds when W satisfies Eq. (4), placing it within a confidence region determined by the significance level α.The region is approximately [m − s√σ, m + s√σ], with confidence level 1 − α.Source: A. Nonparametric Test · p. 3

B. Single-sample and Double-sample T-test

The single-sample t-test compares observed object centroids across frames, while the double-sample test checks whether separately tracked objects should be merged. Both use t-statistics to test whether the centroids are consistent with belonging to the same object.

B. Single-sample and Double-sample T-test

  • The single-sample t-test checks whether an observed centroid c and historical centroid set C come from the same object.It is applied to centroids observed in different frames, which typically follow a Gaussian distribution.Source: B. Single-sample and Double-sample… · p. 3
  • For the null hypothesis to stand, t should satisfy condition (6); if the t statistic satisfies (6), c and C come from the same object.Here, tα/2,v is the upper α/2 quantile of the t-distribution with v degrees of freedom, and v = |C| − 1.Source: B. Single-sample and Double-sample… · p. 3
  • The double-sample t-test addresses objects mistakenly recognized as new by testing whether their historical centroids indicate the same object.When the test satisfies the stated condition, the two objects are merged; the degrees of freedom are |C1| + |C2| − 2.Source: B. Single-sample and Double-sample… · p. 3

V. OBJECT SLAM

The system represents objects with cubes or quadrics and estimates their position, scale, and orientation from monocular observations. It rejects point-cloud outliers for centroid and scale estimation, initializes yaw by line alignment, then jointly optimizes object and camera poses.

Object Representation

  • Objects use cubes for regular shapes and quadrics for non-regular shapes without an explicit direction; assuming ground-parallel placement reduces the estimated pose to yaw, translation, and scale for cubes, and translation and scale for quadrics.The representations are expressed in the object frame and registered in the global frame using translation and orientation.Source: §V · p. 4

Estimate t and s

  • iForest removes points with scores above 0.6 before computing the object centroid and scale from the remaining cloud.The algorithm identifies easily isolated points as outliers; the retained points determine the mean location and scale.Source: §V · pp. 4–5

Estimate θy

  • Yaw initialization samples 30 angles over [−π/2, π/2] and selects the angle with the highest score from line-to-edge alignment errors.The score counts line segments with error below five degrees and accounts for their average error.Source: §V · p. 5, Figure 5 · p. 5

Joint Optimization

  • Joint optimization combines object pose and scale errors with the camera reprojection error.The scale error measures distances between projected cube edges and their nearest parallel LSD segments.Source: §V · p. 5

A. Distributions of Different Statistics

The tested object point-cloud positions and centroid errors follow different distribution patterns. This supports using different statistical tests for the two measurements.

B. Ensemble Data Association Experiments

The ensemble method is more robust than the compared association approaches, with stronger results than on TUM. The experiments also show where that advantage is less evident.

  • Figure 6 depicts point-cloud position distributions in three directions and centroid distance-error distributions as statistics used in data association.Source: Figure 6 · p. 6
  • Across other sequences, the final number of objects in the map is closer to ground truth with the proposed method.The evaluation uses the number of objects finally present in the map; GT denotes the ground-truth object number.Source: B. Ensemble Data Association Experiments · p. 6, Table I · p. 6
  • In the TUM fr3 long office sequence, IoU, nonparametric-test, and t-test approaches leave some objects incorrectly associated, while the ensemble handles occlusion-related unassociated objects more robustly.Source: B. Ensemble Data Association Experiments · pp. 5–6
  • On TUM, EAO-SLAM associates almost twice as many objects successfully as, while its advantage is not obvious on Microsoft RGB-D and Scenes V2, where object counts are limited.Source: B. Ensemble Data Association Experiments · p. 6

C. Qualitative Assessment of Object Pose Estimation

The pose-estimation assessment shows improvement through initialization and joint optimization, alongside qualitative examples across dataset sequences. The visualizations include an initial scale error and the later estimated poses.

D. Object-Oriented Map Building

EAO-SLAM builds semi-dense and object-oriented maps, with object poses and scales registered. The reported results include varied scenes and a real scenario, while noting an inaccurate estimate in one sequence.

D. Object-Oriented Map Building

  • The examples show semi-dense semantic and object-oriented maps, which express the environment better than ORB-SLAM2's sparse map; the object-oriented map is reported to outperform the semi-dense map in environment understanding.Source: D. Object-Oriented Map Building · p. 7
  • EAO-SLAM processes multiple object classes at different scales and orientations in complex environments, although a fast-moving camera yields an inaccurate chair estimate in the fire sequence.Source: D. Object-Oriented Map Building · p. 7
  • In a real scenario, objects are accurately estimated despite occlusion.Source: D. Object-Oriented Map Building · p. 7

VII. CONCLUSION

  • The conclusion reports that EAO-SLAM builds accurate object-oriented maps with object poses and scales registered, and says the methods will facilitate navigation, mobile manipulation, and human-robot interaction research.Source: §VII · p. 7

Related papers

Loading 2004.12730v2…