PaperlayerView exact revision on arXiv

Source-linked AI summary

Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age

Cesar Cadena, Luca Carlone, Henry Carrillo, Yasir Latif, Davide Scaramuzza, Jose Neira, Ian Reid, John J. Leonard

arXiv:1606.05830v4cs.RO

TL;DR

SLAM has advanced substantially, but its adequacy depends on the robot, environment, and performance requirements. This paper surveys SLAM methods and concludes that robust perception and navigation for long-lived autonomous robots remain open research challenges.

  • Problem

    The field lacks clear criteria for choosing optimal map representations and still faces unresolved challenges in robust, long-term autonomous perception.

  • Method

    The paper synthesizes SLAM formulations, representations, decision-making frameworks, sensors, and open research directions as a survey and position paper.

  • Results

    SLAM is not solved universally: its status depends on the specific robot, environment, and performance combination, with major challenges remaining for many applications.

  • Takeaways & Limitations

    More SLAM research is needed to achieve truly robust perception and navigation for long-lived autonomous robots.

  • Takeaways & Limitations

    Key unsolved areas include robust performance, high-level understanding, resource awareness, and task-driven inference, especially for long-term autonomy and resource-constrained systems.

Abstract

from arXiv · show

Simultaneous Localization and Mapping (SLAM)consists in the concurrent construction of a model of the environment (the map), and the estimation of the state of the robot moving within it. The SLAM community has made astonishing progress over the last 30 years, enabling large-scale real-world applications, and witnessing a steady transition of this technology to industry. We survey the current state of SLAM. We start by presenting what is now the de-facto standard formulation for SLAM. We then review related work, covering a broad set of topics including robustness and scalability in long-term mapping, metric and semantic representations for mapping, theoretical performance guarantees, active SLAM and exploration, and other new frontiers. This paper simultaneously serves as a position paper and tutorial to those who are users of SLAM. By looking at the published research with a critical eye, we delineate open challenges and new research issues, that still deserve careful scientific investigation. The paper also contains the authors' take on two questions that often animate discussions during robotics conferences: Do robots need SLAM? and Is SLAM solved?

MULTIMEDIA MATERIAL

The paper provides supplementary multimedia material online, including an extended bibliography and pointers to SLAM datasets.

  • MULTIMEDIA MATERIAL: Supplementary material, including BibTeX references and pointers to online SLAM datasets, is available at slam-future.github.io.The website hosts additional resources accompanying the paper.

I. INTRODUCTION

SLAM jointly estimates a robot’s state and constructs an environmental map, using loop closures to maintain global consistency and support applications requiring coherent representations. The paper surveys current SLAM, argues that the field is entering a robust-perception age, and identifies challenges involving difficult conditions and demanding performance requirements.

  • I. INTRODUCTION: SLAM simultaneously estimates the robot’s state and constructs a model of the environment from onboard sensor observations.The state may include pose, velocity, sensor biases, and calibration parameters, while the map represents environmental aspects of interest.
  • I. INTRODUCTION: Maps support downstream tasks such as path planning and visualization while limiting localization drift by enabling error correction against environmental landmarks.Without a map, dead-reckoning quickly drifts over time.
  • I. INTRODUCTION: The paper reviews current SLAM with emphasis on metric and semantic mapping, while addressing whether autonomous robots need SLAM and whether SLAM is solved.It presents a community perspective on open problems and future research directions.
  • I. INTRODUCTION: SLAM is distinguished by globally consistent mapping that combines ego-motion measurements with loop closures; without loop closures, it reduces to odometry.Loop closures also reveal the environment’s true topology rather than an apparently infinite corridor.
  • I. INTRODUCTION: SLAM remains challenging for fast robot dynamics, highly dynamic environments, and strict requirements such as high-rate estimation for fast closed-loop control.The paper also frames evaluation through accuracy, robustness, success rate, latency, operation time, and mapped-area size.
  • I. INTRODUCTION: The paper argues that SLAM is entering a robust-perception age defined by robust performance, high-level understanding, resource awareness, and task-driven perception.These requirements include low failure rates over extended operation, self-tuning, adaptive computational load, selective perceptual processing, and adaptive map representations.

II. ANATOMY OF A MODERN SLAM SYSTEM

A modern SLAM system separates front-end sensor-data abstraction from back-end inference. Its standard back-end formulates SLAM as MAP estimation over factor graphs, typically yielding a nonlinear least-squares problem solved by sparse optimization methods.

  • A SLAM architecture has a front-end that abstracts sensor data and a back-end that performs inference on the resulting models.
  • Back-end: MAP estimation: The de-facto SLAM formulation estimates robot poses and landmarks by maximizing the posterior probability p(X|Z), often using factor graphs.The posterior combines measurement likelihoods with a prior over the unknown variables; without prior knowledge, MAP reduces to maximum likelihood.
  • Back-end: MAP estimation: With Gaussian measurement noise, MAP estimation becomes a nonlinear least-squares problem, while alternative noise models produce different cost functions such as the ℓ1-norm.Robust formulations commonly replace the squared ℓ2-norm to increase resilience to outliers.
  • Back-end optimization: Successive linearization methods such as Gauss-Newton and Levenberg-Marquardt iteratively approximate the cost and solve the resulting normal equations.
  • Back-end optimization: Factor-graph sparsity enables fast linear solvers and incremental updates as new observations arrive, while MAP estimation has been shown more accurate and efficient than original nonlinear-filtering approaches.

III. LONG-TERM AUTONOMY I: ROBUSTNESS

Long-term SLAM must remain robust despite perceptual aliasing, changing or harsh environments, and sensor or actuator degradation. Key open problems include fail-safe recovery, metric relocalization, time-varying maps, and automatic parameter tuning.

  • Algorithmic robustness: Perceptual aliasing makes data association difficult because different sensory inputs can produce the same sensor signature.Incorrect associations can propagate through loop closure and severely corrupt the MAP estimate.
  • Algorithmic robustness: Unmodeled short-term and seasonal dynamics undermine the common static-world assumption and complicate long-term data association.Dynamic SLAM must detect, discard, or track changes and update the map for permanent or semi-permanent changes.
  • Open problems: An ideal SLAM system should be fail-safe and failure-aware because nonconvex optimization is sensitive to initialization and can be destabilized by a single outlier.Outlier-induced degradation can produce an incorrect linearization point from which recovery is difficult, especially incrementally.
  • Open problems: Robust long-term SLAM must handle sensor degradation, non-rigid and time-varying environments, and arbitrary scenarios without extensive manual parameter tuning.Hardware degradation can make measurement noise and bias inconsistent with the back-end noise model, while current systems rely on rigid, static-world assumptions and scenario-specific thresholds.
  • Open problems: Metric relocalization remains unreliable across day-night or seasonal changes because feature descriptors lack sufficient invariance, despite appearance-based methods closing such loops topologically.Metric relocalization estimates relative pose with respect to a previously built map.

IV. LONG-TERM AUTONOMY II: SCALABILITY

Long-term SLAM must scale beyond building-sized environments as continuous exploration causes factor graphs to grow without bound. The section reviews sparsification, parallel and multi-robot methods, while identifying unresolved challenges in map maintenance, distributed robustness, and resource-constrained platforms.

  • Motivation: Continuous exploration in large-scale applications can make the SLAM factor graph grow without bound, requiring long-term scalability beyond typical indoor building-scale demonstrations.Examples include ocean monitoring, non-stop urban cleaning, and large-scale precision agriculture.
  • Approaches: Scalability approaches reduce factor-graph complexity through sparsification or by distributing computation and memory across processors or robots.Sparsification trades information loss for efficiency, while out-of-core methods divide graphs into subgraphs for alternating local optimization and global refinement.
  • Distributed multi-robot SLAM: Distributed SLAM can use centralized fusion of robot submaps or decentralized local communication, but Gaussian elimination incurs dense-marginal communication costs and depends on good linearization points.Sparsification has been proposed to reduce communication costs associated with dense marginals.
  • Open Problems: Long-term mapping still lacks grounded answers about map storage, information updating, forgetting, and recalling offloaded map components.Raw point-cloud, volumetric, and feature-descriptor representations can become wasteful or cumbersome even when storage is available in the cloud.
  • Open Problems: Robust distributed mapping and resource-constrained SLAM remain relatively unexplored, especially for rejecting outliers with partial information and operating on small platforms.Target platforms include mobile phones, micro aerial vehicles, and robotic insects, for which many existing algorithms are too expensive.

V. REPRESENTATION I: METRIC MAP MODELS

The section surveys metric map representations for encoding environment geometry, from sparse landmarks and dense models to emerging object- and solid-based forms. It highlights unresolved questions about choosing and adapting representations to environment complexity, task demands, and future SLAM capabilities.

  • Metric maps symbolically encode environmental geometry, and selecting or extending suitable representations could affect long-term navigation, physical interaction, and human-robot interaction.
  • 3D metric modeling spans sparse landmarks, raw dense point clouds and polygons, boundary or spatial-partitioning models, and higher-level object-based representations.Examples include TSDFs, meshes, voxels, octrees, and solid shapes.
  • Feature-based methods are mature and support accurate, robust SLAM with relocation and loop closing, but depend on environmental features and detection and matching thresholds.
  • High-level object-based representations: Higher-level representations promise compact maps and richer object geometry for map compression, data association, place recognition, semantic understanding, and human-robot interaction.
  • Optimal and Automatic, Adaptive Representations: Open problems include criteria for selecting representations across environments and adaptive systems that change representation complexity according to task and context.The section contrasts parametrized primitives for simple indoor scenes with mesh models for complex outdoor environments.

VI. REPRESENTATION II: SEMANTIC MAP MODELS

Semantic mapping augments geometric maps with concepts whose detail and organization depend on robotic tasks, supporting greater autonomy, robustness, task planning, and human-robot interaction. The field includes three main approaches but remains without a cohesive formulation, with open challenges in semantic-metric fusion, actionable representations, adaptation, and semantic reasoning.

  • Motivation: Semantic mapping associates semantic concepts with geometric entities to overcome purely geometric maps’ limitations and support autonomy, robustness, task planning, and human-robot interaction.Applications include avoiding muddy roads and moving from path-planning to task-planning.
  • Semantic vs. topological SLAM: Semantic mapping differs from topological mapping: topological maps encode places and reachability without metric information, whereas semantic maps represent concepts such as kitchens or corridors.Topological mapping relies on recognizing previously seen places, while semantic mapping focuses on what those places or entities are.
  • Semantic SLAM: Structure and detail of concepts: Semantic representations must choose concept detail and organization according to the task, because entities can have multiple properties and relationships.Coarse room, corridor, and door categories may suffice for navigation, whereas picking up a tea cup requires finer categories such as tables and cups; flat or hierarchical organizations must handle overlapping properties.
  • Brief Survey: Three main approaches assign semantic concepts to data: semantics can be extracted from metric SLAM maps, used to improve SLAM through known classes, or inferred jointly with SLAM.The surveyed examples include offline and online semantic mapping, semantic priors for geometric estimation, and joint formulations that improve recognition and robustness but may impose substantial runtime costs.
  • Open Problems: Semantic SLAM remains in its infancy and lacks a cohesive formulation, especially for consistently fusing uncertain semantic evidence with metric information collected at different times.Potential directions include incorporating semantic categorization confidence or uncertainty into factor-graph formulations for metric representations.
  • Open Problems: Open problems extend beyond categorization to representing affordances, actionability, agent interactions, newly discovered concepts, environmental adaptation, and efficient semantic-based localization and mapping.Robots should infer occluded structure, such as planar ground beneath a detected car, and update the global pose and map coherently when the car moves.

VII. NEW THEORETICAL TOOLS FOR SLAM

This section reviews theoretical tools for analyzing SLAM, emphasizing factor-graph optimization, nonconvexity and local-convergence failures, global optimization and verification techniques, and open questions about generality, duality, and outlier resilience.

  • Foundations: Factor graph optimization offers a practical and analytically tractable framework for studying SLAM beyond early EKF-based analyses.The section focuses on factor graph optimization because of its accuracy, efficiency, and amenability to analysis.
  • Limitations: Most SLAM algorithms use iterative nonlinear optimization, but nonconvexity guarantees only local convergence and can produce navigation-inappropriate estimates.State-of-the-art iterative solvers can fail to reach a global minimum even with relatively small noise levels.
  • Algorithmic tools: Theoretical analysis has motivated convex relaxations, dual SDP methods, and improved initialization strategies to reduce convergence to local minima.These methods aim to design better algorithms by exploiting problem structure or providing suitable starting points for iterative optimization.
  • Verification: Lagrangian-duality methods enable verification techniques that assess whether a SLAM estimate is optimal, supporting failure detection and recovery in safety-critical applications.Current approaches provide verification capabilities, while broader guarantees remain an active research area.
  • Open Problems: Open problems include extending guarantees beyond pose graphs and structured noise, establishing a priori conditions for strong duality, and achieving global resilience to outliers.Existing results do not yet cover arbitrary factor graphs, arbitrary noise models, or spurious measurements with comparable guarantees.

VIII. ACTIVE SLAM

Active SLAM deliberately controls robot motion to reduce map and localization uncertainty, extending passive SLAM through decision-making and exploration. Its practical deployment remains limited by computational intractability and open challenges in prediction, stopping criteria, and performance guarantees.

  • Definition: Active SLAM controls robot motion to minimize uncertainty in the map representation and localization, rather than passively processing collected sensor data.The approach leverages deliberate motion to improve mapping and localization results.
  • Decision frameworks: Decision-making frameworks for active SLAM include optimal experimental design and information-theoretic approaches that select actions using predicted map uncertainty or information gain.TOED compares stochastic action designs through covariance-based criteria such as A-opt, D-opt, and E-opt.
  • Action selection: Evaluating every possible action is computationally intractable because evaluation complexity grows exponentially with the robot-and-map search space, so systems select small subsets of vantage locations.Practical methods include frontier-based exploration.
  • Open Problems: Active SLAM needs fast and accurate forecasts of how future actions affect map uncertainty and localization, including the effects of important loop closures.Predictions must meet latency constraints while remaining precise enough to support decisions.
  • Open Problems: Stopping active SLAM is an open challenge because it is computationally expensive and must be balanced against exogenous tasks, while approximation methods need clear performance bounds.Task-oriented TOED criteria appear promising for termination, and submodularity offers an example of pursuing mathematical guarantees for near-optimal policies.

IX. NEW FRONTIERS: SENSORS AND LEARNING

New sensors and computational tools have driven SLAM development, with this section examining unconventional sensors and deep learning as frontiers that may improve, affect, or restate the SLAM problem.

  • IX. NEW FRONTIERS: SENSORS AND LEARNING: New sensors and computational tools have often been key drivers of SLAM development.The section reviews unconventional and new sensors, along with their challenges and opportunities for SLAM.
  • IX. NEW FRONTIERS: SENSORS AND LEARNING: Deep learning is analyzed as a frontier that may improve, affect, or even restate the SLAM problem.The discussion considers the role of learning as an important computational tool for SLAM.

A. New and Unconventional Sensors for SLAM

SLAM is expanding beyond lidar and conventional cameras to alternative modalities that offer distinctive capabilities, including robustness in darkness, linearized motion estimation, and operation under high-speed or high-dynamic-range conditions. Their broader adoption remains constrained by sensor-specific limitations, noise, computational demands, and the lack of a complete framework for selecting sensor–algorithm combinations.

  • Alternative SLAM sensors include depth, light-field, event-based, magnetic, olfaction, and thermal cameras or sensors, beyond dominant lidar and conventional vision.
  • Range cameras: Range cameras work in dark and untextured scenes because they carry their own light source, enabling remarkable SLAM results.
  • Event-based cameras: Event-based cameras offer 1ms latency, up to 1MHz update rates, up to 140dB dynamic range, 20mW power consumption, and low bandwidth and storage requirements.These properties support SLAM in high-speed-motion and high-dynamic-range scenes.
  • Light-field cameras: Light-field cameras can formulate motion estimation as a linear optimization and provide more accurate motion estimates when designed properly.
  • Event-based cameras: Event-camera SLAM remains challenged by noise and sensor nonidealities, while individual events lack sufficient information for state estimation and cameras generate about 100,000 events per second.
  • Sensor selection: No complete method yet determines the best sensor–algorithm pair, whose performance depends on sensor and algorithm parameters, environment, and available sensing power.A preliminary study suggests that robust architectures may switch multiple sensors on and off or measure the same phenomenon through different physical principles.

B. Deep Learning

Deep learning is already enabling learned alternatives and perceptual tools for SLAM, but its role remains unsettled relative to traditional methods. Key challenges include embedded deployment, online adaptation, limited labeled data, and integrating learned architectures with recursive state estimation.

  • Learned SLAM capabilities: Deep networks can regress inter-frame pose, localize a camera’s 6DoF, and estimate scene depth directly from images.These approaches replace or supplement standard visual-odometry geometry and conventional mapping inputs.
  • Perceptual tool: Deep learning improves SLAM perception through object recognition and semantic pixel labeling, which reaches around 80% accuracy or more.These capabilities make previously difficult perceptual problems more accessible as practical tools for roboticists and SLAM researchers.
  • Practical deployment: Embedded deployment remains difficult because deep-learning successes often require lengthy training on supercomputers and inference on specialized GPU hardware.SLAM systems must either await improved embedded computing or use smaller, cheaper networks that produce sufficiently good results.
  • Online and life-long learning: Long-term SLAM requires online learning in open-world settings, whereas deep networks are typically trained for closed worlds with fixed object classes.The system must adapt as it encounters new objects and scenes, potentially through one-shot or zero-shot learning.
  • Online and life-long learning: Deep learning can reduce dependence on labeled depth data, as stereo pairs have been used to train single-view depth estimation without directly observing or calculating depth.This addresses the difficulty of guaranteeing suitable datasets for supervised training.
  • Future architectures: An end-to-end deep SLAM architecture could eventually combine learned representations with recursive state estimation without explicit feature modeling or data association.Deep Kalman Filters illustrate a possible connection between deep architectures and large-scale graphical-model inference.

X. CONCLUSION

The conclusion argues that SLAM remains necessary for many applications but is not universally required, and that whether SLAM is solved depends on the robot, environment, and performance combination. It identifies robust perception, high-level understanding, resource awareness, and task-driven inference as major open challenges.

  • Is SLAM necessary?: SLAM is often necessary, especially for reliable metric positioning when infrastructure-based solutions such as GPS are unavailable or inaccurate.SLAM and visual-inertial odometry are increasingly deployed from self-driving cars to mobile devices.
  • Is SLAM necessary?: Online SLAM is unnecessary when accurate high-definition maps support precision localization, but dynamic environments require online updates for major infrastructure changes.Distributed updating and maintenance of visual maps are therefore important in changing environments.
  • Choosing SLAM formulations: Different SLAM representations suit different tasks: topological maps support reachability, locally consistent metric maps support local interactions, and globally consistent maps support global planning.The passage contrasts these representations by their suitability for motion planning, obstacle avoidance, accuracy, and global path planning.
  • Is SLAM solved?: SLAM is not solved independently of context; robust long-term autonomy still requires fail-safe systems, time-varying map maintenance, and policies for remembering, updating, or forgetting information.The conclusion organizes unsolved questions around robust performance, high-level understanding, resource awareness, and task-driven inference.
  • Open representation challenges: Current SLAM systems still lack tightly coupled high-level geometric and semantic understanding linked to task-driven optimal representations.The paper states that a tractable framework connecting tasks to optimal representations is lacking.
Loading 1606.05830v4…