PaperlayerView exact revision on arXiv

Source-linked AI summary

ORB-SLAM: a Versatile and Accurate Monocular SLAM System

Raul Mur-Artal, J. M. M. Montiel, Juan D. Tardos

arXiv:1502.00956v2cs.ROcs.CV

TL;DR

Monocular SLAM must estimate camera motion and reconstruct environments accurately in real time. ORB-SLAM uses one ORB-feature-based system across tracking, mapping, relocalization, and loop closing, and demonstrates accurate operation across varied indoor and outdoor scenarios, with more than twice PTAM’s relocalization recall in one benchmark.

  • Problem

    Visual SLAM needs accurate camera trajectories and sparse reconstructions while keeping bundle-adjustment computation feasible for real-time operation.

  • Method

    ORB-SLAM uses the same rotation- and illumination-invariant ORB features across tracking, mapping, relocalization, and loop closing, with covisibility-based local processing.

  • Results

    ORB-SLAM operates across indoor and outdoor scenes and varied motions, with accuracy typically below 1 cm indoors and a few meters outdoors after scale alignment.

  • Takeaways & Limitations

    The system provides a versatile monocular SLAM solution that supports real-time tracking, mapping, relocalization, and loop closing across diverse scenarios.

Abstract

from arXiv · show

This paper presents ORB-SLAM, a feature-based monocular SLAM system that operates in real time, in small and large, indoor and outdoor environments. The system is robust to severe motion clutter, allows wide baseline loop closing and relocalization, and includes full automatic initialization. Building on excellent algorithms of recent years, we designed from scratch a novel system that uses the same features for all SLAM tasks: tracking, mapping, relocalization, and loop closing. A survival of the fittest strategy that selects the points and keyframes of the reconstruction leads to excellent robustness and generates a compact and trackable map that only grows if the scene content changes, allowing lifelong operation. We present an exhaustive evaluation in 27 sequences from the most popular datasets. ORB-SLAM achieves unprecedented performance with respect to other state-of-the-art monocular SLAM approaches. For the benefit of the community, we make the source code public.

I. INTRODUCTION

ORB-SLAM is a novel monocular SLAM system designed for real-time operation in large environments, using ORB features across tracking, mapping, relocalization, and loop closing. Its contributions include robust initialization, local covisibility-based processing, Essential Graph loop closing, and selective map maintenance, supported by extensive evaluation and public code.

  • Core contributions: ORB-SLAM uses ORB features for tracking, mapping, relocalization, and loop closing, enabling real-time performance without GPUs with viewpoint and illumination invariance.The shared feature representation is intended to make the system more efficient, simple, and reliable.
  • Core contributions: ORB-SLAM achieves real-time operation in large environments by restricting tracking and mapping to a local covisible area independent of global map size.A covisibility graph supports this local processing strategy.
  • Core contributions: The Essential Graph enables real-time loop closing through pose-graph optimization using a spanning tree, loop-closure links, and strong covisibility edges.The graph combines persistent system structure with loop-specific and strong local connections.
  • Core contributions: ORB-SLAM provides real-time camera relocalization with significant viewpoint and illumination invariance, supporting recovery from tracking failure and map reuse.Relocalization is based on the same feature strategy used throughout the system.
  • Core contributions: A model-selection-based initialization procedure robustly creates initial maps for both planar and non-planar scenes.The paper adds this initialization method to the current system and evaluates it experimentally.
  • Evaluation and release: Selective culling of map points and keyframes improves tracking robustness and lifelong operation by discarding redundant keyframes after generous spawning.The paper reports extensive evaluation across indoor and outdoor public-dataset sequences and makes the source code public.

II. RELATED WORK … C. Monocular SLAM

The related work reviews scalable appearance-based place recognition, monocular map-initialization strategies, and keyframe-based SLAM systems. It positions ORB-SLAM against prior approaches by emphasizing automatic initialization, map reuse, loop closing, and robust keyframe selection.

  • A. Place Recognition: Appearance-based place recognition scales better in large environments than map-to-map or image-to-map methods, with bags of binary words offering high efficiency.FAB-MAP is cited as a probabilistic bags-of-words approach, while DBoW2 applies bags of binary words from BRIEF descriptors.
  • B. Map Initialization: Monocular SLAM must initialize a map despite unavailable depth from a single image, using known structures, uncertain inverse-depth estimates, or two-view reconstruction.Two-view methods use either homographies under local planarity or essential matrices for planar and general scenes.
  • B. Map Initialization: Two-view homography and essential-matrix reconstructions are poorly constrained under low parallax and can suffer twofold ambiguity in certain planar configurations.The essential-matrix approach uses Nister’s five-point algorithm, which must handle multiple solutions.
  • B. Map Initialization: ORB-SLAM introduces automatic initialization through model selection between a homography for planar scenes and a fundamental matrix for non-planar scenes.Its heuristic accounts for the risk of selecting a fundamental matrix in close-to-degenerate cases, including planar, nearly planar, and low-parallax scenes.
  • C. Monocular SLAM: Filtering SLAM processes every frame but wastes computation on consecutive frames and accumulates linearization errors, whereas keyframe methods enable costlier, more accurate mapping.PTAM pioneered parallel camera tracking and mapping and succeeded for real-time augmented reality in small environments.
  • C. Monocular SLAM: LSD-SLAM builds real-time large-scale semi-dense maps through direct image-intensity optimization, while SVO achieves high frame-rates but lacks loop detection and targets downward-looking cameras.LSD-SLAM’s semi-dense output is described as having more potential robotics applications than sparse feature-based SLAM output.
  • C. Monocular SLAM: Keyframe selection must balance computational cost and tracking robustness: retaining many points with non-redundant keyframes is cost-effective, whereas overly cautious insertion can fail during hard exploration.ORB-SLAM’s survival-of-the-fittest strategy is presented as addressing this tradeoff.

III. SYSTEM OVERVIEW · A. Feature Choice · B. Three Threads: Tracking, Local Mapping and Loop Closing

ORB-SLAM uses fast, viewpoint-invariant ORB features for tracking, mapping, relocalization, and loop detection. Its parallel tracking, local mapping, and loop-closing threads support camera localization, reconstruction, and globally consistent mapping.

  • A. Feature Choice: ORB-SLAM uses the same features for mapping, tracking, place recognition, frame-rate relocalization, and loop detection.This avoids interpolating recognition-feature depth from nearby SLAM features and requires feature extraction to take much less than 33ms per image.
  • A. Feature Choice: ORB features are oriented multi-scale FAST corners with 256-bit descriptors, enabling fast computation and matching with good viewpoint invariance.Their wide-baseline matching improves bundle-adjustment accuracy.
  • B. Three Threads: Tracking, Local Mapping and Loop Closing: Three parallel threads handle tracking, local mapping, and loop closing.The overview organizes the system around these concurrent processes.
  • B. Three Threads: Tracking, Local Mapping and Loop Closing: Tracking localizes the camera in every frame, decides when to insert keyframes, and uses motion-only bundle adjustment after initial feature matching.If tracking is lost because of occlusions or abrupt movements, place recognition performs global relocalization before retrieving a local visible map.
  • B. Three Threads: Tracking, Local Mapping and Loop Closing: Local mapping processes new keyframes, performs local bundle adjustment, triangulates points from connected keyframes, and culls low-quality points.Unmatched ORB features are searched through the covisibility graph to create new correspondences and points.
  • B. Three Threads: Tracking, Local Mapping and Loop Closing: Loop closing detects loops, estimates a similarity transformation, aligns both sides, fuses duplicated points, and optimizes a pose graph for global consistency.The optimization runs over the Essential Graph, a sparser subgraph of the covisibility graph.
  • B. Three Threads: Tracking, Local Mapping and Loop Closing: All optimizations use the Levenberg-Marquardt algorithm implemented in g2o.The paper specifies the associated error terms, cost functions, and variables in its Appendix.

C. Map Points, KeyFrames and their Selection · D. Covisibility Graph and Essential Graph

ORB-SLAM stores map points and keyframes with geometric, appearance, scale, pose, calibration, and feature information, using generous creation followed by strict culling. It represents keyframe covisibility as a weighted graph and constructs a sparser Essential Graph for loop-closing pose optimization.

  • C. Map Points, KeyFrames and their Selection: Map points store their 3D world position and mean viewing direction.The viewing direction averages the unit rays from observing keyframe optical centers to the point.
  • C. Map Points, KeyFrames and their Selection: Each map point uses a representative ORB descriptor selected by minimum Hamming distance among associated observations.
  • C. Map Points, KeyFrames and their Selection: Map points record dmax and dmin observation distances determined by ORB feature scale-invariance limits.
  • C. Map Points, KeyFrames and their Selection: Keyframes store a world-to-camera rigid-body pose, camera intrinsics, and all undistorted ORB features, whether or not associated with map points.
  • C. Map Points, KeyFrames and their Selection: Generous map-point and keyframe creation is followed by exigent culling of redundant keyframes and wrongly matched or untrackable map points.This supports flexible map expansion during exploration and improves tracking robustness under rotations and fast movements.
  • D. Covisibility Graph and Essential Graph: The covisibility graph is an undirected weighted graph whose keyframe edges require at least 15 shared map-point observations, with weights equal to shared-point counts.
  • D. Covisibility Graph and Essential Graph: For loop correction, pose-graph optimization distributes closing error along a sparser Essential Graph while retaining all keyframe nodes and a strong edge network.The Essential Graph is introduced because the covisibility graph can be very dense.
  • D. Covisibility Graph and Essential Graph: The Essential Graph incrementally includes a spanning tree rooted at the initial keyframe, linking each new keyframe to the one sharing most point observations and updating links after culling.The spanning tree provides a connected covisibility subgraph with a minimal number of edges.

E. Bags of Words Place Recognition · IV. AUTOMATIC MAP INITIALIZATION

ORB-SLAM uses a DBoW2-based bags-of-words module for loop detection, relocalization, and efficient ORB matching. Its automatic initialization evaluates homography and fundamental-matrix models in parallel, selects the appropriate reconstruction based on scene geometry, and refines it with bundle adjustment.

  • E. Bags of Words Place Recognition: The system embeds a DBoW2-based bags-of-words module for loop detection and relocalization using an offline vocabulary built from ORB descriptors.A general-enough vocabulary can be reused across environments with good performance.
  • E. Bags of Words Place Recognition: Keyframes are grouped through the covisibility graph so place recognition can combine views of the same place recorded at different times.This replaces DBoW2’s temporal-neighborhood grouping, which excludes keyframes connected spatially but inserted at different times.
  • E. Bags of Words Place Recognition: Vocabulary-tree nodes restrict ORB matching to descriptors sharing the same node, speeding correspondence searches for triangulation, loop detection, and relocalization.The system selects the second level of a six-level vocabulary tree.
  • IV. AUTOMATIC MAP INITIALIZATION: Automatic initialization estimates relative pose between two frames while remaining applicable to planar and general scenes without manually selecting a high-parallax configuration.It computes homography and fundamental-matrix models in parallel.
  • IV. AUTOMATIC MAP INITIALIZATION: Outlier rejection uses χ2 thresholds of TH = 5.99 for the homography and TF = 3.84 for the fundamental matrix, assuming 1-pixel measurement-error standard deviation.The scoring normalization makes both models score equally for the same inlier-region error, and the highest-scoring model is retained.
  • IV. AUTOMATIC MAP INITIALIZATION: The homography is selected for planar, nearly planar, or low-parallax scenes, while the fundamental matrix is selected for non-planar scenes with sufficient parallax.This avoids incorrectly recovering motion from an underconstrained fundamental-matrix solution or a homography explaining only a subset of matches.
  • IV. AUTOMATIC MAP INITIALIZATION: For homography recovery, the method triangulates all 8 motion hypotheses directly before selecting a reconstruction, avoiding unreliable cheirality tests under low parallax.After model recovery, a full bundle adjustment refines the initial reconstruction.

V. TRACKING … D. Track Local Map

ORB-SLAM tracks each frame by extracting distributed multi-scale FAST corners, estimating pose from the previous frame or global relocalization, and matching against a bounded local map. Its automatic initialization can recover from a fundamental-matrix solution once sufficient parallax is detected.

  • V. TRACKING: Tracking comprises repeated per-frame steps, including camera-pose optimization through motion-only bundle adjustment.The paper states that these tracking-thread steps are performed with every camera frame.
  • V. TRACKING: ORB-SLAM automatically initializes from the fundamental matrix after detecting enough parallax, whereas PTAM and LSD-SLAM can initialize a corrupted planar solution.PTAM can also initialize well depending on manually selected keyframes.
  • A. ORB Extraction: ORB-SLAM extracts FAST corners at 8 scale levels using a 1.2 scale factor and grid-based threshold adaptation for homogeneous distribution.It targets 1000 corners at resolutions from 512 × 384 to 752 × 480 pixels and 2000 corners for the 1241 × 376 KITTI resolution.
  • B. Initial Pose Estimation from Previous Frame: With successful previous-frame tracking, a constant-velocity model predicts the pose and guides searches for map points observed in the last frame.When matches are insufficient, the search widens around their previous positions before pose optimization.
  • C. Initial Pose Estimation via Global Relocalization: When tracking is lost, ORB-SLAM queries a bag-of-words recognition database for keyframe candidates and uses ORB correspondences, RANSAC, and PnP for relocalization.A pose with enough inliers is optimized, followed by a guided search for additional matches.
  • D. Track Local Map: The tracker projects only a local map to bound complexity in large maps after obtaining a camera-pose estimate and initial feature matches.The local map includes keyframes sharing map points with the current frame and their covisibility-graph neighbors, plus a reference keyframe.
  • D. Track Local Map: Local-map point projections are filtered by image bounds, a 60° viewing-angle constraint, and the map point’s scale-invariance distance region.The tracker then computes frame scale from d/dmin and compares the map point’s representative descriptor with the frame features.

E. New Keyframe Decision … D. Local Bundle Adjustment

ORB-SLAM inserts keyframes readily under explicit tracking, timing, and visual-change conditions, then locally maps each new keyframe by updating connectivity, culling unreliable points, triangulating validated points, and optimizing a covisibility-based neighborhood.

  • E. New Keyframe Decision: Keyframes are inserted as fast as possible, improving robustness to challenging camera movements while allowing local mapping to cull redundant keyframes.Insertion requires all listed conditions to hold.
  • E. New Keyframe Decision: A new keyframe requires more than 20 frames since relocalization, an available or delayed local mapper, at least 50 tracked points, and fewer than 90% of Kref’s points.The visual-change condition replaces PTAM’s distance criterion.
  • E. New Keyframe Decision: These conditions support relocalization and tracking; inserting while local mapping is busy signals local bundle adjustment to stop and process the keyframe.Condition 1 ensures good relocalization, and condition 3 ensures good tracking.
  • A. KeyFrame Insertion: For each new keyframe, local mapping updates the covisibility graph, spanning tree, and bag-of-words representation for data association.The new graph node connects through shared map points, and the spanning tree links to the keyframe with most points in common.
  • B. Recent Map Points Culling: New map points must be found in more than 25% of predicted-visible frames and observed from at least three keyframes after creation.After passing, a point is removed only if it is later observed from fewer than three keyframes.
  • C. New Map Point Creation: ORB matches from connected keyframes are triangulated only after satisfying the epipolar constraint, positive depth, parallax, reprojection error, and scale consistency checks.Unmatched ORBs in the new keyframe are matched against unmatched points in other connected keyframes.
  • D. Local Bundle Adjustment: Local bundle adjustment optimizes the current keyframe, its covisibility-connected keyframes, and all map points seen by them.Other keyframes observing those points are included but remain fixed.
  • D. Local Bundle Adjustment: Outlier observations are discarded during and after local bundle adjustment.The optimization removes marked outliers at its middle and end.

E. Local Keyframe Culling … VIII. EXPERIMENTS

ORB-SLAM maintains a compact, lifelong-capable map through local keyframe culling and closes loops using covisibility-based candidate detection, similarity validation, map-point fusion, and Essential Graph optimization. Experiments evaluate performance across NewCollege, TUM RGB-D, and KITTI sequences, with real-time processing at the acquisition frame rate.

  • E. Local Keyframe Culling: Keyframes are culled when 90% of their map points have been seen in at least three other keyframes, limiting reconstruction growth and bundle-adjustment complexity.This supports lifelong operation unless scene visual content changes.
  • VII. LOOP CLOSING: The loop-closing thread processes the latest locally mapped keyframe and attempts to detect and close loops.
  • A. Loop Candidates Detection: Loop candidates are filtered by comparing bag-of-words similarity with covisibility neighbors using θmin = 30, then querying the recognition database.Keyframes scoring below the lowest neighbor score are discarded, while covisibility information improves robustness.
  • B. Compute the Similarity Transformation: Loop closure estimates a seven-degree-of-freedom similarity transformation covering three translations, three rotations, and scale drift.The transformation both measures accumulated loop error and geometrically validates the loop.
  • B. Compute the Similarity Transformation: ORB correspondences yield 3D-to-3D matches, and RANSAC with Horn’s method accepts a loop candidate only when the optimized similarity has enough inliers.The system then performs guided correspondence search and re-optimization before final acceptance.
  • C. Loop Fusion: Loop fusion merges duplicated map points, adds covisibility edges, and propagates the similarity-based pose correction from the current keyframe through its neighbors.Map points observed around the loop are projected into neighboring keyframes to search for matches.
  • D. Essential Graph Optimization: Essential Graph pose-graph optimization distributes loop-closing error across the graph using similarity transformations that correct scale drift.Each map point is transformed according to the correction of an observing keyframe.
  • VIII. EXPERIMENTS: Experiments cover 1 NewCollege sequence, 16 TUM RGB-D indoor sequences, and 10 KITTI outdoor sequences, assessing general performance, accuracy, relocalization, lifelong operation, and efficiency.The system processes images in real time at their acquisition frame rate on an Intel Core i7-4700MQ with 8 GB RAM.

A. System Performance in the NewCollege Dataset · B. Localization Accuracy in the TUM RGB-D Benchmark

On NewCollege, ORB-SLAM processes a challenging 2.2km monocular sequence in real time, detects loops efficiently, and maintains a sparse map. On the TUM RGB-D benchmark, it provides robust and accurate localization, outperforming LSD-SLAM and RGBD-SLAM in reported comparisons.

  • A. System Performance in the NewCollege Dataset: ORB-SLAM detects loop closures and extends the local map along both sides of the detected loop.The full sequence reconstruction is shown after processing, although a loop traversed in opposite directions does not perfectly align when no visual loop closure is found.
  • A. System Performance in the NewCollege Dataset: 25-30Hz: tracking operates at frame-rates around 25-30Hz, while local bundle adjustment is the most demanding local-mapping task.Bundle-adjustment time varies between exploration and well-mapped areas because it can be interrupted when tracking inserts a new keyframe.
  • A. System Performance in the NewCollege Dataset: 6 loop closures: loop-detection time increases sublinearly with the number of keyframes.The system queries only database images sharing words with the current image, and its Essential Graph has edges around 5 times the number of keyframes.
  • B. Localization Accuracy in the TUM RGB-D Benchmark: The TUM RGB-D benchmark evaluates camera-localization accuracy using sequences with accurate external-motion-capture ground truth, after unsuitable sequences are discarded.The comparison includes ORB-SLAM, LSD-SLAM, PTAM, and provided RGBD-SLAM trajectories; monocular trajectories are aligned with a similarity transformation and evaluated using ATE.
  • B. Localization Accuracy in the TUM RGB-D Benchmark: 8 and 3 sequences: PTAM and LSD-SLAM respectively lose track, while ORB-SLAM remains more robust across the remaining sequences.LSD-SLAM also has very high error in a sequence where PTAM sometimes initializes with either the true or corrupted solution.
  • B. Localization Accuracy in the TUM RGB-D Benchmark: ORB-SLAM achieves higher accuracy than PTAM on large-loop trajectories and is clearly more accurate than LSD-SLAM and RGBD-SLAM overall in the reported comparisons.ORB-SLAM and PTAM are similar on open trajectories; the paper attributes the difference from LSD-SLAM and RGBD-SLAM partly to retaining sensor measurements through bundle adjustment.
  • B. Localization Accuracy in the TUM RGB-D Benchmark: 24.28cm: a previously reported PTAM RMSE on fr2 xyz is not reproducible from the paper’s available details.The paper also notes that RGBD-SLAM has a scale bias in fr2 sequences, with 7-DoF alignment substantially reducing error.

C. Relocalization in the TUM RGB-D Benchmark · D. Lifelong Experiment in the TUM RGB-D Benchmark

ORB-SLAM is evaluated for relocalization under viewpoint and dynamic changes, then for lifelong mapping in static and changing TUM RGB-D scenes. The experiments show robust localization and a map whose size is bounded in static scenes and grows with scene content rather than time.

  • C. Relocalization in the TUM RGB-D Benchmark: ORB-SLAM is evaluated by building a map from the first 30 seconds of fr2 xyz, then globally relocalizing every successive frame and comparing recovered-pose accuracy with PTAM.The experiment uses the TUM RGB-D benchmark and includes ground-truth poses for the relocalized frames.
  • C. Relocalization in the TUM RGB-D Benchmark: ORB-SLAM successfully handles relocalization from very different viewpoints and remains robust under moderate dynamic changes.The experiments include challenging cases involving severe scale change and dynamic objects.
  • D. Lifelong Experiment in the TUM RGB-D Benchmark: ORB-SLAM, PTAM, and LSD-SLAM results are reported as medians over 5 executions per sequence, with trajectories aligned using 7DoF against ground truth.RGBD-SLAM trajectories are available only for fr1 and fr2, and tracking loss is marked when a significant sequence portion is unprocessed.
  • D. Lifelong Experiment in the TUM RGB-D Benchmark: In static scenes, keyframe culling keeps the map size bounded despite changing viewpoints.A 93-second desk sequence demonstrates this behavior while the camera continuously changes its viewpoint.
  • D. Lifelong Experiment in the TUM RGB-D Benchmark: The lifelong dynamic-scene experiment concatenates six fr3 sequences showing the same desk under different trajectories, moving people, and changed chairs or other objects.The study tracks total keyframe count and each keyframe’s creation and destruction over time.
  • D. Lifelong Experiment in the TUM RGB-D Benchmark: During the first two dynamic sequences, the map grows as additional scene views are incorporated, probably because of dynamic changes.The keyframe history also shows that most keyframes are culled soon after creation, while only a small subset survive to the experiment’s end.
  • D. Lifelong Experiment in the TUM RGB-D Benchmark: The lifelong experiments show that the map grows with scene content rather than elapsed time and can retain dynamic scene changes.Accumulated changes may support scene understanding through experience gathered in the environment.

E. Large Scale and Large Loop Closing in the KITTI Dataset

On the challenging KITTI odometry benchmark, ORB-SLAM processes sequences at their recorded frame rate and produces accurate trajectories, typically around 1% of map dimensions. Loop closures are essential for correcting drift, while full bundle adjustment offers only modest additional improvement.

  • KITTI benchmark: KITTI challenges monocular vision with fast rotations, heavy foliage, high car speed, and recordings at 10 fps.ORB-SLAM processes the sequences at their real recorded frame rate.
  • Loop closing: Sequence 09 contains a loop detectable only in a few final frames, and the system does not detect it in every execution.Reported results are from executions in which the loop was detected.
  • Accuracy and loop closure: Trajectory error is typically around 1% of map dimensions, reaching 0.3% in sequence 03 and 5% in sequence 08.Sequence 08 has no loops, so drift—especially scale drift—cannot be corrected.
  • Accuracy and loop closure: Loop closures are necessary for accurate reconstructions because open trajectories cannot correct accumulated drift.Sequence 08 demonstrates this limitation because it contains no loops.
  • Bundle adjustment: Twenty iterations of full BA slightly improve trajectories with loops but have negligible effect on open trajectories, indicating that the system output is already very accurate.Full BA is available when the most accurate results are needed.

IX. CONCLUSIONS AND DISCUSSION … C. Future Work

ORB-SLAM is a versatile monocular SLAM system that operates across varied scenes and motions with high accuracy, while combining feature-based robustness with real-time performance. The authors identify hybrid direct-feature methods and denser mapping as future directions.

  • A. Conclusions: ORB-SLAM processes indoor and outdoor sequences under car, robot, and hand-held motion, typically achieving below 1 cm accuracy indoors and a few meters outdoors after scale alignment.The reported accuracy applies to small indoor and large outdoor scenarios, respectively.
  • A. Conclusions: The system expands PTAM’s versatility to environments that PTAM cannot handle by introducing new algorithms while incorporating recent loop detection, loop closing, covisibility, and optimization techniques.The passage identifies this expansion as the work’s main contribution.
  • A. Conclusions: ORB features provide enough recognition power for place recognition under severe viewpoint changes while enabling fast, real-time tracking and mapping without multithreading or GPU acceleration.Their speed applies to both feature extraction and matching.
  • B. Sparse/Feature-based vs. Dense/Direct Methods: Direct methods produce dense or semi-dense reconstructions and can be more robust to blur, low texture, and high-frequency texture, while avoiding feature-extraction artifacts.These methods localize cameras by optimizing directly over image pixel intensities.
  • B. Sparse/Feature-based vs. Dense/Direct Methods: Direct methods are limited by photometric consistency, narrower matching baselines, and sensitivity to modeling errors such as rolling shutter, auto-gain, and auto-exposure.Narrower baselines can reduce reconstruction accuracy because wide-baseline observations reduce depth uncertainty.
  • B. Sparse/Feature-based vs. Dense/Direct Methods: Feature-based methods support wide-baseline matching through viewpoint and illumination invariance, and experiments in real-time SLAM provide evidence of their superior accuracy over direct methods.Bundle adjustment jointly optimizes camera poses and points over sensor measurements; the authors advocate combining both approaches.
  • C. Future Work: ORB-SLAM could improve accuracy by incorporating points at infinity, which provide informative camera-rotation constraints despite insufficient parallax for map inclusion.The current system does not include these points in its map.
  • C. Future Work: The sparse map could seed a denser reconstruction because selected keyframes provide a compact environmental summary, highly accurate poses, and rich covisibility information.The sparse map can serve as an initial guess and skeleton for building a dense, accurate scene map.

APPENDIX NON-LINEAR OPTIMIZATIONS

The appendix defines nonlinear optimizations for bundle adjustment, pose-graph refinement, and relative Sim(3) estimation. These methods minimize robust, covariance-weighted reprojection or graph errors while fixing selected variables to remove gauge freedom or stabilize estimation.

  • Bundle Adjustment: Bundle adjustment jointly optimizes map-point 3D locations and keyframe poses by minimizing reprojection error against matched keypoints.The formulation uses camera projection functions and SE(3) keyframe poses.
  • Bundle Adjustment: The BA cost uses a Huber robust function and keypoint-scale covariance, with full, local, and motion-only variants optimizing different point and keyframe subsets.Full BA fixes the first keyframe as the world origin; local BA fixes a subset of keyframes, while pose optimization concerns motion only.
  • Pose Graph Optimization over Sim(3) Constraints: Pose-graph optimization minimizes errors from binary Sim(3) edges in R7, using relative transformations and fixing the loop-closure keyframe to remove 7 gauge degrees of freedom.The edge error is mapped to tangent space with logSim3, and the information matrix is set to the identity.
  • Pose Graph Optimization over Sim(3) Constraints: The Sim(3) pose-graph method is a rough approximation of full BA but achieves significantly faster and better convergence experimentally.The comparison is reported for the experiments in Section VIII-E.
  • Relative Sim(3) Optimization: Relative Sim(3) optimization estimates the transformation between two keyframes by minimizing reprojection error in both images while keeping the associated 3D points fixed.The formulation uses matches between keypoints and map points and incorporates keypoint-scale covariance matrices from both images.
Loading 1502.00956v2…