PaperlayerView exact revision on arXiv

Source-linked AI summary

Pop-up SLAM: Semantic Monocular Plane SLAM for Low-texture Environments

Shichao Yang, Yu Song, Michael Kaess, Sebastian Scherer

arXiv:1703.07334v1cs.CVcs.RO

TL;DR

Monocular SLAM in low-texture environments needs robust state estimates and dense maps, while plane SLAM can be under-constrained. Pop-up Plane SLAM uses single-image plane measurements and combines planes with point-based SLAM; the paper reports dense mapping on TUM and low loop-closure error on a 60 m dataset.

  • Problem

    Low-texture environments challenge monocular SLAM, and planar SLAM can be under-constrained, including along corridors with parallel walls.

    Source: §I · p. 1, §V · p. 4

  • Method

    Pop-up Plane SLAM derives plane measurements from single-image scene layout and combines plane SLAM with point-based LSD SLAM in two configurations.

    Source: §VII · p. 7

  • Results

    6.2 cm depth error and 3.9% state estimates error on the public TUM dataset; 0.67% loop closure error for LSD Pop-up SLAM on a 60 m dataset.

    Source: §VII · p. 7

  • Takeaways & Limitations

    The method outperforms existing methods in some low-texture environments and is demonstrated on large datasets with loops.

    Source: §I · p. 1

Abstract

from arXiv · show

Existing simultaneous localization and mapping (SLAM) algorithms are not robust in challenging low-texture environments because there are only few salient features. The resulting sparse or semi-dense map also conveys little information for motion planning. Though some work utilize plane or scene layout for dense map regularization, they require decent state estimation from other sources. In this paper, we propose real-time monocular plane SLAM to demonstrate that scene understanding could improve both state estimation and dense mapping especially in low-texture environments. The plane measurements come from a pop-up 3D plane model applied to each single image. We also combine planes with point based SLAM to improve robustness. On a public TUM dataset, our algorithm generates a dense semantic 3D model with pixel depth error of 6.2 cm while existing SLAM algorithms fail. On a 60 m long dataset with loops, our method creates a much better 3D model with state estimation error of 0.67%.

I. INTRODUCTION

The paper combines monocular scene understanding with visual SLAM to improve state estimation and dense mapping, particularly in low-texture environments. It presents a plane-based system that also uses point-based SLAM for robustness.

I. INTRODUCTION

  • Monocular cameras are small, low-cost sensors suited to weight-constrained micro aerial vehicles, motivating pose and map estimation from monocular images.Source: §I · p. 1
  • Point-based visual SLAM tracks features across frames but works poorly in low-texture scenes and typically produces sparse or semi-dense maps.The paper notes that such maps convey little information for motion planning.Source: §I · p. 1
  • Pop-up plane measurements, plane association, and loop closing jointly optimize scene layout and poses; point-based LSD SLAM adds robustness when plane SLAM is under-constrained.The system is described as real-time and monocular.Source: §I · p. 1
  • The method outperforms existing methods in some low-texture environments and is demonstrated on large datasets with loops.The introduction states the qualitative comparison and scope of the demonstrations.Source: §I · p. 1

II. RELATED WORK

  • The paper frames its approach as combining single-image scene understanding with multiple-image visual SLAM.Source: §I · p. 1, §II · p. 2

A. Single Image

Prior work addresses single-image layout estimation, point- and plane-based SLAM, and multi-image scene understanding, but the paper identifies limitations for low-texture mapping and state estimation. It extends a single-image pop-up plane model to produce more consistent geometry for use in SLAM.

A. Single Image

  • Prior single-image approaches include room-box and fixed-building models; the authors’ earlier pop-up model combined CNNs with geometry modeling and handled varied corridor configurations and lighting.Source: A. Single Image · p. 2

1) v-SLAM using points:

  • Point-based methods such as LSD SLAM and ORB SLAM [2] track features across frames, but perform poorly when low-texture scenes lack visual and geometric features.Source: 1) v-SLAM using points · p. 2

2) v-SLAM using planes:

  • Plane- or superpixel-based mapping – can densify low-texture maps, but assumes poses from other sources; room-layout depth priors do not track and update layout and are limited to small workspaces.Source: 2) v-SLAM using planes · p. 2

3) Scene understanding:

  • Multi-image scene-understanding methods – do not use a plane world to constrain state estimation, so they do not solve low-texture visual SLAM.Source: 3) Scene understanding · p. 2

III. SINGLE IMAGE PLANE POP-UP

A. Pop-up 3D Model

  • Fitted ground-region polylines may miss true wall-ground edges, creating invalid models and frame-varying planes that violate SLAM’s landmark-invariance requirement.The paper gives a right-turning hallway as an example.Source: A. Pop-up 3D Model · p. 2
  • The earlier model assumes zero camera rotation, which is often unmet; different rotations can produce different pop-up models.Source: A. Pop-up 3D Model · p. 2
  • The improved pipeline detects line segments, selects ground-wall edges, and estimates camera pose; better edge detection and pose estimation yield a more accurate 3D model.Source: Figure 2 · p. 2, A. Pop-up 3D Model · p. 2

B. Optimal Boundary Detection

The method selects and merges detected line segments into wall–ground boundaries by optimizing edge coverage under proximity and overlap constraints. Its objective and constraints are designed for varied environments rather than only Manhattan-world scenes.

B. Optimal Boundary Detection

  • The method selects and merges line segments as wall–ground boundaries, addressing detector noise such as a straight line split into disconnected segments.It optimizes a subset of detected edges using a score function and feasibility constraints designed for varied scene structures, not only Manhattan-world scenes.Source: B. Optimal Boundary Detection · pp. 2–3
  • Edges must stay near the CNN-detected boundary curve, within a threshold.This proximity condition is one of two constraints on the selected edge set.Source: B. Optimal Boundary Detection · p. 3
  • Selected edges are constrained against excessive horizontal overlap; the algorithm can still select most ground edges in unsatisfactory configurations.The figure contrasts desired corridor layouts, where all ground edges can be selected, with layouts where overlap may cause some edges to be missed.Source: B. Optimal Boundary Detection · p. 3, Figure 3 · p. 3
  • The score maximizes the horizontal image coverage of the selected edge set, and a greedy algorithm adds feasible edges by marginal gain.After selection, post-processing removes tiny edges and merges adjacent ones.Source: B. Optimal Boundary Detection · p. 3

C. Pop-up World from an Arbitrary Pose

The pop-up model constructs 3D planes in a global frame from image boundaries and plane geometry. It assumes walls are vertical to the ground and uses the camera pose to transform local planes into world coordinates.

C. Pop-up World from an Arbitrary Pose

  • A local plane is represented by its normal and distance, and transformed to the global frame using the local-to-global camera pose.The notation uses w for the world frame, c for the camera frame, and gnd for the ground plane.Source: C. Pop-up World from an Arbitrary Pose · p. 3

1) Create 3D model:

  • A pixel on a plane is back-projected through the camera calibration matrix to obtain its 3D pop-up point at the ray–plane intersection.The calibration matrix is denoted K.Source: 1) Create 3D model · p. 3
  • Given two ground-edge boundary pixels, the model obtains their 3D points and uses the assumption that walls are vertical to the ground to compute a wall-plane normal.The wall-plane distance is then computed from the condition that both points lie on the wall.Source: 1) Create 3D model · p. 3

2) Camera pose estimation:

The paper estimates camera rotation from Manhattan-world vanishing points and extends plane SLAM to monocular images using pop-up plane measurements. Its factor graph jointly represents camera poses and labeled plane landmarks for optimization.

2) Camera pose estimation:

  • Three orthogonal Manhattan directions correspond to vanishing points, whose constraints recover the camera’s 3-DoF rotation for initialization.The camera attitude estimate is presented as a single-image method.Source: 2) Camera pose estimation · p. 3

IV. POP-UP PLANE SLAM

  • The monocular extension uses pop-up plane measurements to perform plane SLAM from images.Plane SLAM had previously been addressed with RGB-D measurements in the cited work.Source: §IV · p. 3

A. Planar SLAM Formulation

  • The factor graph estimates 6-DoF camera poses and ground-or-wall plane landmarks from plane measurements, odometry, and an initial pose constraint.The ground-plane landmark connects to every pose; planes use a unit-quaternion minimal representation to support optimization.Source: A. Planar SLAM Formulation · p. 3, Figure 4 · p. 4

B. Plane Measurement

The system derives plane measurements from a single-image pop-up model, associates planes using geometric cues, and handles loop closure by consolidating duplicate plane landmarks. It combines plane and point-based SLAM to address under-constrained motion and uses pop-up depths to enhance LSD SLAM.

B. Plane Measurement

  • Plane measurements come from the single-image pop-up model and are recomputed after pose optimization; updating hundreds takes less than 1 ms.The pop-up process depends on camera rotation and height, but not its local x, y position.Source: B. Plane Measurement · p. 4

C. Data Association

  • Plane matching filters candidates by normal difference, distance, and projected overlap, then selects the best match using a weighted sum of those cues.The projected polygon comes from the pop-up process.Source: C. Data Association · p. 4, Figure 5 · p. 4

D. Loop Closure

  • Bag-of-words place recognition detects loops; after repeated observations identify the same plane, its factors are transferred to the retained landmark and the duplicate is removed.The paper reports that keeping bag-of-words visual words for each plane is not robust in texture-less images.Source: D. Loop Closure · p. 4, Figure 6 · p. 4

V. POINT-PLANE SLAM FUSION

  • Plane SLAM can leave motion along a corridor unconstrained, so the proposed fusion adds point-based LSD SLAM photometric odometry constraints along that direction.The paper identifies parallel corridor walls as an example of this under-constrained direction.Source: §V · p. 4

A. Depth Enhanced LSD SLAM

  • Pop-up depth estimates replace missing or high-variance LSD depths and are fused with propagated estimates otherwise, improving depth quality at initialization and in low-parallax scenes.The paper says the pop-up depth uncertainty is propagated from image-pixel uncertainty.Source: A. Depth Enhanced LSD SLAM · pp. 4–5

B. LSD Pop-up SLAM

LSD Pop-up SLAM runs Depth Enhanced LSD SLAM first, then uses its pose output as odometry constraints for plane SLAM. Those measurements constrain motion and capture fine movements that plane-only SLAM may miss.

B. LSD Pop-up SLAM

  • LSD Pop-up SLAM uses pose output from Depth Enhanced LSD SLAM as odometry constraints in a subsequent plane SLAM stage.The frame-to-frame photometric tracking provides constraints along plane SLAM’s unconstrained direction and captures fine movements.Source: B. LSD Pop-up SLAM · p. 5
  • The proposed two-stage approach first runs Depth Enhanced LSD SLAM and then uses its pose output to run plane SLAM.Source: B. LSD Pop-up SLAM · p. 5
  • Unlike Pop-up Plane SLAM, LSD Pop-up SLAM adds odometry measurements; Pop-up Plane SLAM usually uses a constant-velocity assumption.Figure 8 distinguishes the three proposed methods: Pop-up Plane SLAM, Depth Enhanced LSD SLAM, and LSD Pop-up SLAM.Source: B. LSD Pop-up SLAM · p. 5, Figure 8 · p. 5

VI. EXPERIMENTS AND RESULTS

The experiments compare monocular methods on a low-texture TUM scene, using RGB images and depth images for evaluation. Pop-up Plane SLAM reconstructs depth while LSD SLAM and ORB SLAM fail in this setting.

A. TUM SLAM dataset

  • The TUM fr3/structure notexture far scene consists of five white walls and a ground plane, and Pop-up Plane SLAM uses ground-truth pose for initialization.The provided initial truth height gives the pop-up model an absolute scale.Source: A. TUM SLAM dataset · p. 5, 1) Qualitative Comparison · p. 5

1) Qualitative Comparison:

2) Quantitative Comparison:

  • 0.18 ± 0.07 m mean positioning error and 0.10 m endpoint error for Pop-up Plane SLAM on the 4.58 m TUM trajectory.The method captures overall movement but misses a small jerk movement in the middle; the paper attributes this mainly to few plane landmarks and lack of frame-to-frame odometry tracking.Source: 2) Quantitative Comparison · p. 5, Figure 9 · p. 5
  • On the TUM dataset, the depth-reconstruction evaluation reports a mean pixel depth error of 6.2 cm, with 86.8% of pixels’ depth error within 0.1 m.Source: 2) Quantitative Comparison · p. 6, Figure 10 · p. 6, Table I · p. 6

B. Large Indoor Environment

The corridor experiments assess qualitative mapping and loop-closure error without ground-truth depth or pose. LSD Pop-up SLAM produces a dense corridor model and, on the looped dataset, the best map and smallest reported loop-closure error; the system runs in real time.

1) Corridor dataset I:

  • LSD Pop-up SLAM generates a dense 3D model with distinct doors and pillars, building on the passageway and turning visible in Depth Enhanced LSD SLAM’s semi-dense map.The first corridor comparison reports that original LSD SLAM does not perform well and ORB SLAM often fails to initialize or track.Source: 1) Corridor dataset I · p. 6, Figure 11 · p. 6

2) Corridor dataset II:

  • ORB SLAM produces a better map than LSD SLAM on the second corridor dataset, but does not start tracking until reaching a large open space with more features and sufficient parallax.This dataset is a 60 m square corridor containing a large loop.Source: 2) Corridor dataset II · p. 6, Figure 12 · p. 6
  • 0.4 m loop-closure positioning error over the 60 m trajectory, with LSD Pop-up SLAM generating the best 3D map and smallest loop-closure error.Figure 13 reports the loop-closure positioning error as 0.67%; its grid dimension is 1 m.Source: 2) Corridor dataset II · p. 6, Figure 13 · p. 7

C. Runtime Analysis

  • The plane SLAM algorithm runs at over 20 Hz in single-threaded programming; CNN segmentation, edge detection, and selection take 30 ms.iSAM incremental update takes 17.43 ms and batch optimization takes 45.35 ms; batch optimization is used to re-factor the information matrix when a loop closes.Source: C. Runtime Analysis · p. 6, Table II · p. 7

D. Discussion

Pop-up Plane SLAM combines scene-layout planes with point-based LSD SLAM, and the paper reports improved mapping and loop closure on its evaluated datasets. Its scale depends on camera height, while visible ground planes affect graph complexity and occluded ground-wall edges remain an open challenge.

1) Height effect on map scale:

  • Plane measurements and map scale depend on camera height, so inaccurate or unavailable external information can allow scale and height to drift in plane-only SLAM.The experiments did not encounter scale drift because camera height stayed nearly constant.Source: 1) Height effect on map scale · p. 7

2) Ground effect on graph complexity:

  • The ground-plane landmark connects to all camera poses and can increase graph fill-in, but variable reordering limits the measured increase to about 10%.The paper cites COLAMD as an example of reordering that moves this variable toward the last block column.Source: 2) Ground effect on graph complexity · p. 7

VII. CONCLUSIONS

  • The method combines point-based LSD SLAM with plane SLAM in two ways: integrating pop-up pixel depth into LSD depth estimation, or using LSD poses as plane-SLAM odometry constraints.The conclusion identifies plane SLAM as easily under-constrained in some environments.Source: §VII · p. 7
  • 0.67% loop-closure error for LSD Pop-up SLAM on a 60 m dataset; on TUM, Pop-up Plane SLAM achieved 6.2 cm depth error and 3.9% state-estimate error while LSD and ORB SLAM failed.The runtime analysis reports operation over 10Hz.Source: §VII · p. 7
  • The conclusion leaves unified point, edge, and plane landmarks for future work and identifies cluttered corridors with potentially occluded ground-wall boundaries as needing further study.The authors also want to test the algorithm on robots.Source: §VII · p. 7

APPENDIX

The appendix analyzes edge selection as a submodular set-selection problem with matroid constraints. It explains the score's monotonicity, diminishing marginal gains, the constraint construction, and the greedy algorithm's guarantee.

A. Submodular Edge Selection

  • The edge-selection problem is characterized as submodular set selection with matroid constraints.The appendix presents this as an optimality analysis of the edge-selection method.Source: A. Submodular Edge Selection · p. 8

1) Monotonicity:

  • Adding edges cannot decrease horizontal image coverage, which makes the score function monotonically increasing.The marginal gain is defined as the score increase after adding an edge to a set.Source: 1) Monotonicity · p. 8, 2) Submodularity · p. 8

2) Submodularity:

  • An edge can overlap more edges in a larger set, reducing its marginal gain and satisfying the submodularity condition.Source: 2) Submodularity · p. 8

3) Matroid constraint type:

  • Conflicting edge pairs define partition constraints, and the union of these matroid constraints forms the original overlap constraint.The construction permits at most one selected edge from each conflicting pair.Source: 3) Matroid constraint type · p. 8

4) Optimality:

  • The greedy algorithm has a stated worst-case guarantee, though the appendix says its solution is usually better than that lower bound.Source: 4) Optimality · p. 8
Loading 1703.07334v1…