PaperlayerView exact revision on arXiv

Source-linked AI summary

Structure-SLAM: Low-Drift Monocular SLAM in Indoor Environments

Yanyan Li, Nikolas Brasch, Yida Wang, Nassir Navab, Federico Tombari

arXiv:2008.01963v1cs.RO

TL;DR

Low-textured indoor scenes and irregular feature distributions challenge monocular SLAM, while accumulated errors cause drift. Structure-SLAM predicts dense surface normals and decouples Manhattan World rotation estimation from point-and-line translation; it reports outperforming state of the art on common benchmarks.

  • Problem

    Low-textured indoor surfaces and irregular feature points challenge monocular SLAM, and accumulated errors cause drift that misaligns dense reconstructions.

    Source: §I · p. 1

  • Method

    Structure-SLAM predicts dense normals from RGB images, estimates rotation using the Manhattan World assumption, computes translation from point and line features, and refines poses.

    Source: §I · p. 2, B. Translation Estimation · p. 4

  • Results

    The proposed approach outperforms state-of-the-art monocular SLAM on common benchmarks including ICL-NUIM and TUM RGB-D.

    Source: §I · p. 2

  • Takeaways & Limitations

    Learned normals from a single RGB image can support rotation estimation under the Manhattan World assumption, with point-and-line fallback when no Manhattan frame is found.

    Source: §VII · p. 8

    Rotation estimation relies on finding a Manhattan frame; without one, point-and-line fallback supports tracking over short sequences before re-localization within the Manhattan world.

    Source: §VII · p. 8

Abstract

from arXiv · show

In this paper a low-drift monocular SLAM method is proposed targeting indoor scenarios, where monocular SLAM often fails due to the lack of textured surfaces. Our approach decouples rotation and translation estimation of the tracking process to reduce the long-term drift in indoor environments. In order to take full advantage of the available geometric information in the scene, surface normals are predicted by a convolutional neural network from each input RGB image in real-time. First, a drift-free rotation is estimated based on lines and surface normals using spherical mean-shift clustering, leveraging the weak Manhattan World assumption. Then translation is computed from point and line features. Finally, the estimated poses are refined with a map-to-frame optimization strategy. The proposed method outperforms the state of the art on common SLAM benchmarks such as ICL-NUIM and TUM RGB-D.

I. INTRODUCTION

Low-textured indoor scenes challenge monocular SLAM, and accumulated pose errors create drift that misaligns dense reconstructions. The paper proposes decoupled rotation and translation estimation using predicted surface normals and point and line features, and reports outperforming state of the art on common benchmarks.

I. INTRODUCTION

  • Low-textured walls, floors, and ceilings, together with irregularly distributed feature points, challenge both feature-based and direct methods.Accumulated frame-to-frame errors cause drift that misaligns reconstructed surfaces.Source: §I · p. 1
  • The Manhattan World assumption can provide drift-free rotation estimation by aligning frames to global indoor structure, since rotation is a main source of accumulated drift.Source: §I · pp. 1–2
  • Prior monocular Manhattan World methods rely on parallel and orthogonal lines because monocular RGB images provide limited 3D information beyond vanishing points.Source: §I · p. 2
  • Structure-SLAM predicts dense normal maps from each RGB image to use scene structure in low-textured environments.Source: §I · p. 2
  • The framework decouples rotation and translation, estimating rotation from normals and translation from point and line features.Source: §I · p. 2
  • The proposed approach outperforms state-of-the-art monocular SLAM on ICL-NUIM and TUM RGB-D benchmarks.Source: §I · p. 2

II. RELATED WORK

Prior monocular SLAM work includes keyframe, optical-flow, feature-based, and learned-structure approaches. Structure-SLAM adds predicted normal maps to point and line features for low-textured scenes.

A. Monocular SLAM

  • PTAM separates camera tracking and mapping into parallel threads, while later systems combine feature tracking, mapping, relocalization, and loop closure.Source: A. Monocular SLAM · p. 2
  • CNN-SLAM estimates depth for dense reconstruction, while Structure-SLAM predicts normals from every RGB frame; CodeSLAM jointly optimizes latent scene structure and pose.The paper states that normal prediction avoids differentiating depth maps and appears to generalize better between datasets.Source: A. Monocular SLAM · p. 2

B. RGB-D SLAM

The related work discusses RGB-D methods and Manhattan World estimation, then introduces the structural features analyzed for Structure-SLAM. Points and lines supplement rotation estimation to support translation under limited parallax or pure rotation.

B. RGB-D SLAM

  • RGB-D systems combine points, lines, and planes or align plane segments, with some methods using global plane landmarks or smoothing and mapping.Source: B. RGB-D SLAM · pp. 2–3

C. Manhattan World

  • Manhattan World methods estimate orientation from line groups, planes, or vanishing points, while translation approaches include point-based estimation and density alignment.Structure-SLAM adds initialization from points, lines, and predicted normals, followed by pose refinement.Source: C. Manhattan World · p. 3

III. SCENE STRUCTURE ANALYSIS

  • The scene-structure analysis section covers extraction and triangulation of points and lines, followed by an architecture for surface-normal prediction.Source: §III · p. 3

A. Points and Lines Analysis

  • ORB points are supplemented with LSD/LBD line features because low-textured scenes may not provide enough points for robust pose estimation.Triangulated point and line landmarks support tracking; separating rotation from translation allows pose estimation under pure rotation or small parallax and reduces the landmarks needed for translation.Source: A. Points and Lines Analysis · p. 3

B. Surface Normal Prediction

The system predicts planar regions and pixel-wise surface normals from RGB images, then uses the normals to initialize rotation relative to a Manhattan coordinate frame. It estimates translation from point correspondences between frames.

B. Surface Normal Prediction

  • Surface normals and planar masks are predicted with a two-branch CNN using a ResNet101-FPN encoder.The branches perform planar-area segmentation and normal estimation.Source: B. Surface Normal Prediction · p. 3
  • The planar mask is used to filter the normal-estimation loss, while balanced cross entropy addresses the imbalance between planar and non-planar regions.The mask weights the normal loss to planar regions.Source: B. Surface Normal Prediction · p. 3

IV. INITIALIZATION

  • Normals are clustered on the unit Gaussian sphere to estimate the rotation from the Manhattan frame to the first camera frame.The method projects normals onto tangent planes around Manhattan axes and uses mean shift to find dominant directions.Source: §IV · pp. 3–4
  • The tangent plane and cluster centers are iteratively updated until the rotation estimate converges, yielding the camera-to-Manhattan rotation.The Gaussian-kernel width c is set to 2 in the experiments.Source: §IV · p. 4
  • Point correspondences between two frames are used to construct a translation relationship, which is solved with SVD.The translation setup uses the relative rotation between the frames and camera intrinsics.Source: §IV · p. 4

V. TRACKING

Rotation is estimated between each frame and the underlying Manhattan World rather than between consecutive frames. Point and line correspondences provide the translation estimate.

V. TRACKING

  • Estimating rotation against the Manhattan World makes residual rotation errors independent of sequence length and prevents their propagation between frames.Point and line correspondences estimate the three translation degrees of freedom using frame-to-frame and frame-to-map methods.Source: §V · p. 4
  • Rotation is estimated relative to the underlying Manhattan World, not between two frames.Figure 3 depicts rotation estimation between multiple frames via the Manhattan World.Source: §V · p. 4, Figure 3 · p. 4
  • Point and line correspondences estimate translation through a combination of frame-to-frame and frame-to-map methods.The translation estimate has three degrees of freedom.Source: §V · p. 4

A. Manhattan Rotation Estimation

The method estimates camera rotation relative to the Manhattan system by clustering predicted surface normals and planar-region masks. Clustering helps reject normal-map outliers before subsequent refinement.

A. Manhattan Rotation Estimation

B. Translation Estimation

After estimating rotation, the method estimates three-degree-of-freedom translation from points and line segments. It fixes rotation during optimization and combines point reprojection and line-distance errors.

B. Translation Estimation

  • The method estimates 3-DoF translation from points and line segments after obtaining the rotation matrix.This requires fewer features than full 6-DoF estimation.Source: B. Translation Estimation · p. 4
  • For point features, 3D points from the last frame are reprojected into the current frame, and translation is optimized with rotation fixed.The optimization uses the translation portion of the Jacobian for the reprojection error.Source: B. Translation Estimation · p. 4
  • For line features, the error is the point-to-line distance from projected 3D endpoints to the matched 2D line.The 2D line is formed from its start and end points.Source: B. Translation Estimation · p. 4
  • The point and line errors are combined in a least-squares cost and solved with Levenberg-Marquardt.Source: B. Translation Estimation · p. 5

C. Fallback and Pose Refinement

The method checks whether a Manhattan-World pose is reliable, falls back to feature-based tracking when needed, and refines the new frame against a local map without reusing the Manhattan-World assumption.

C. Fallback and Pose Refinement

  • Reprojection errors from features in the last n keyframes are thresholded, and a minimum inlier count is required to accept a Manhattan-World pose.This check addresses cases where the Manhattan Frame is not visible or the scene is non-Manhattan.Source: C. Fallback and Pose Refinement · p. 5
  • When the Manhattan-World estimate lacks enough inliers, tracking falls back first to the last frame and then, if needed, to the last keyframe.The fallback uses point and line reprojection searches, with descriptor matching for points in the keyframe stage.Source: C. Fallback and Pose Refinement · p. 5
  • The new frame is finally optimized against a local map from the last n keyframes, without the Manhattan-World assumption.The authors report that initial rotation estimation is sufficient for drift reduction, while errors in predicted normals can make later pose estimates inconsistent.Source: C. Fallback and Pose Refinement · p. 5

VI. EXPERIMENTS

Experiments evaluate normal prediction and SLAM on indoor datasets, with qualitative normal results reported for common and uncommon scenes. The setup also compares normal-map sources and evaluates trajectory accuracy and tracking robustness.

VI. EXPERIMENTS

  • The normal network is trained on ScanNet, with an ImageNet-pretrained backbone and PlaneReconstruction pretraining; the experiments report real-time normal-map prediction.The GeoNet baseline was trained by its authors on NYU-DepthV2, and the models were not fine-tuned on other datasets.Source: §VI · p. 5
  • Evaluation uses ATE and RPE, reports median trajectory accuracy across five runs, and compares monocular SLAM systems and alternative normal maps.Experiments ran on an Intel Core i7-8700 CPU and NVIDIA 2080 Ti GPU; trajectories were similarity-aligned to ground truth because monocular scale is unknown.Source: §VI · p. 5
  • The evaluation spans synthetic ICL-NUIM living-room and office scenes, real TUM RGB-D scenes, and a 12,000-frame monocular corridor sequence.The supplied passages identify the datasets and scene scope but do not provide further sequence-level results for these experiments.Source: §VI · p. 5

A. Normal Prediction

  • On uncommon scenes, the proposed model still produces reasonable planar-region detection and normal estimates, while GeoNet’s prediction quality decreases severely.In common indoor examples, both perform well, although GeoNet shows smaller inaccuracies; the proposed method masks non-planar lampshade and box regions.Source: A. Normal Prediction · p. 5, Figure 5 · p. 6
  • The normal-prediction network is evaluated against ground truth on the ScanNet test set, while the text notes that depth-map plane detection struggles on highly slanted surfaces with degraded depth.The table caption identifies the ScanNet test-set evaluation but provides no result values in the supplied passage.Source: A. Normal Prediction · p. 5, Table I · p. 6

B. Pose Estimation

Pose-estimation experiments compare translation error and tracking across indoor sequences, including alternatives for the surface normals. The method is reported as robust across five sequences, with more stable rotation estimates and earlier initialization on HRBB4 than ORB-SLAM.

B. Pose Estimation

  • The comparison uses structured ICL-NUIM and TUM RGB-D sequences, including textured and non-textured variants, and reports translation RMSE.The s-t-near and s-t-far sequences contain textured planes; s-not-near and s-not-far have similar structure without texture.Source: B. Pose Estimation · p. 6
  • AHC-based normals from ground-truth depth give the best results among the tested normal maps, while the proposed network is more robust than GeoNet, especially on s-not-far.The authors note that excessive normal outliers can prevent a good rotation estimate; LPVO uses depth and achieves comparable performance, while Structure-SLAM does not require a depth sensor.Source: B. Pose Estimation · p. 6
  • Structure-SLAM performs robustly across all five sequences, and its use of lines for translation achieves better results than point-based ORB-SLAM, which loses tracking in some sequences.Table II compares translation RMSE and marks tracking failures, but the supplied caption contains no cell values.Source: B. Pose Estimation · p. 6, Table II · p. 7
  • On textured sequences, ORB-SLAM and the proposed method have similar performance, while ORB-SLAM’s relative translation error is significantly larger on of-kt3.The authors also report that Structure-SLAM is more stable than ORB-SLAM in rotation estimation.Source: B. Pose Estimation · p. 7, Figure 6 · p. 7, Figure 7 · p. 7
  • On HRBB4, Structure-SLAM initializes around frame 110, compared with ORB-SLAM after frame 628, and behaves more stably in the corridor’s upper-right corner.The trajectory figure compares the estimated camera paths from the two methods.Source: B. Pose Estimation · p. 7, Figure 8 · p. 7

VII. CONCLUSIONS

The system combines Manhattan World rotation estimation with point- and line-based windowed translation to achieve state-of-the-art performance. Learned normals support rotation estimation, while point and line features provide a fallback when no Manhattan frame is found.

  • Using the Manhattan World assumption for rotation and point and line features for windowed translation, the system achieves state-of-the-art performance.Source: §VII · p. 8
  • Surface normals learned from a single RGB image can estimate rotation between frames under the Manhattan World assumption.Source: §VII · p. 8
  • When no Manhattan frame is found, points and lines provide a full-pose fallback that supports tracking short sequences before later relocalization in the Manhattan world.Source: §VII · p. 8
  • Future work could use global bundle adjustment to correct frames without global frames and leverage learned structure for translation estimation.Source: §VII · p. 8
Loading 2008.01963v1…