PaperlayerView exact revision on arXiv

Source-linked AI summary

maplab: An Open Framework for Research in Visual-inertial Mapping and Localization

Thomas Schneider, Marcin Dymczyk, Marius Fehr, Kevin Egger, Simon Lynen, Igor Gilitschenski, Roland Siegwart

arXiv:1711.10250v1cs.RO

TL;DR

Visual-inertial SLAM systems often lack a complete multi-session workflow that combines mapping, map maintenance, and online localization. maplab addresses this gap with an extensible framework containing offline map-processing tools and the ROVIOLI frontend. Its evaluations include compacted localization maps that retain substantial localization recall.

  • Problem

    Existing visual-inertial SLAM solutions commonly focus on single-session use or omit online localization and complete map-processing workflows, limiting flexible map reuse and research development.

  • Method

    maplab combines multi-session visual-inertial mapping tools, an extensible console, loop-closure and localization backend, and the ROVIOLI online mapping/localization frontend.

  • Results

    Combining keyframing and landmark summarization reduces map size 13 times while retaining 51% recall, compared with 60% for the full map.

  • Takeaways & Limitations

    maplab serves as both a ready-to-use visual-inertial mapping and localization system and an extensible research platform for multi-session mapping.

Abstract

from arXiv · show

Robust and accurate visual-inertial estimation is crucial to many of today's challenges in robotics. Being able to localize against a prior map and obtain accurate and driftfree pose estimates can push the applicability of such systems even further. Most of the currently available solutions, however, either focus on a single session use-case, lack localization capabilities or an end-to-end pipeline. We believe that only a complete system, combining state-of-the-art algorithms, scalable multi-session mapping tools, and a flexible user interface, can become an efficient research platform. We therefore present maplab, an open, research-oriented visual-inertial mapping framework for processing and manipulating multi-session maps, written in C++. On the one hand, maplab can be seen as a ready-to-use visual-inertial mapping and localization system. On the other hand, maplab provides the research community with a collection of multisession mapping tools that include map merging, visual-inertial batch optimization, and loop closure. Furthermore, it includes an online frontend that can create visual-inertial maps and also track a global drift-free pose within a localization map. In this paper, we present the system architecture, five use-cases, and evaluations of the system on public datasets. The source code of maplab is freely available for the benefit of the robotics research community.

I. INTRODUCTION

maplab addresses gaps in existing visual-inertial SLAM systems by combining multi-session mapping, map processing, localization, and an extensible research interface. It introduces ROVIOLI and evaluates selected framework components.

  • Existing visual-inertial SLAM frameworks commonly target single sessions or batch optimization without an online frontend, limiting complete map reuse workflows.
  • maplab is an open C++ visual-inertial mapping framework supporting map creation, localization, maintenance, and processing through an extensible console and plugin system.Its tools include multi-session merging, sparsification, loop closing, dense reconstruction, and visualization.
  • maplab is presented as a unified, open framework integrating varied use-cases and serving both as an off-the-shelf system and a research testbed.The paper reports field deployment across micro aerial vehicles, planes, cars, underwater vehicles, and walking robots.
  • The framework supports feature-based maps with multi-session operation and includes algorithms for map merging, sparsification, place recognition, and visualization.
  • ROVIOLI tightly couples robust visual-inertial estimation with localization, providing the framework’s online mapping and localization frontend.

II. RELATED WORK

Prior systems provide pieces of mapping, localization, or large-scale optimization, but maplab combines offline multi-session processing with an online visual-inertial frontend.

  • PTAM supports local-map tracking but lacks large-scale localization and offline processing tools.
  • OKVIS and dense odometry frameworks estimate visual-inertial or visual motion locally, while maplab targets broader mapping and localization workflows.
  • ORB-SLAM and ORB-SLAM2 can reuse maps in later sessions, but maplab adds an offline console toolkit for flexible processing and pipeline extension.
  • Maplab differentiates itself through multi-session map merging, visual-inertial least-squares refinement, and the ROVIOLI online frontend.The frontend uses image-intensity patches rather than point features, supporting robustness under motion blur.
  • Large-scale image-reconstruction systems offer bundle adjustment and image matching but do not process inertial data or provide online robotic pose estimates.

III. THE MAPLAB FRAMEWORK

maplab is organized around an online ROVIOLI frontend and an offline maplab-console, with modular data structures and extensible software components for visual-inertial research.

  • ROVIOLI takes raw visual-inertial sensor data, outputs global pose estimates, and can build visual-inertial maps.
  • The maplab-console applies offline batch algorithms to maps and serves as a research testbed for new visual-inertial algorithms.
  • The framework uses a modular, extensible design organized into C++11 packages with ROS catkin and established dependencies including Eigen and Ceres.
  • A. Notation: A transformation matrix T_AB maps a vector from frame F_B to frame F_A and is partitioned into rotation and translation components.

B. Workflow for multi-session mapping and localization

maplab’s workflow builds maps in separate sessions, refines and combines them offline, then uses the resulting localization map for online global-pose tracking.

  • Each mapping session first builds an open-loop map with ROVIOLI in VIO mode and stores it for offline refinement.
  • The offline console supports loop closure, visual-inertial optimization, map merging, visualization, and simultaneous loading of multiple maps.
  • The refined multi-session map can be exported and loaded by ROVIOLI in LOC mode to track a global drift-free pose online.
  • The console’s plugin architecture permits new algorithms or functionality without modifying core packages.
  • All missions are anchored in a global gravity-aligned frame, allowing mission transformations to be adjusted without updating individual poses or landmarks.

D. Map structure

maplab organizes visual-inertial data in VI-maps and exposes complementary optimization, localization, mapping, and reconstruction algorithms through a common framework.

  • VI-map representation: A VI-map stores sensor measurements, sparse environmental reconstruction, and multiple missions represented as vertex-edge graphs.Vertices contain estimated states and visual data; IMU edges connect neighboring states, while tracked observations form 3D landmarks.
  • VI-map representation: Mission poses are expressed in mission frames, while mission-to-global transformations anchor multiple sessions without updating individual vertex or landmark poses.
  • Data management: maplab serializes map structures with Google Protobuf and stores large resources separately through a resource management system.Resources can be linked to vertices, mission sets, or timestamps.
  • Core algorithms: VIWLS refines maps using visual-inertial weighted least squares and can optionally incorporate wheel odometry, GPS, or other pose priors.
  • Core algorithms: The framework provides binary-descriptor loop closure and localization, posegraph relaxation, camera-vision utilities, map sparsification, and dense reconstruction.Map sparsification selects landmarks and keyframes for large-scale or lifelong mapping.

F. ROVIOLI: online VIO and localization frontend

ROVIOLI extends ROVIO into maplab’s online frontend, supporting both visual-inertial map creation and online localization against existing maps.

  • Frontend modes: ROVIOLI builds maps from raw visual-inertial data and localizes online against existing maps through VIO and LOC operating modes.It is built around the ROVIO visual-inertial odometry framework and adds localization and mapping capabilities.
  • System architecture: Figure 4 summarizes the modules and data flows that connect ROVIOLI’s tracking and localization functions.
  • Feature processing: The frontend’s feature-tracking module detects and tracks BRISK or FREAK keypoints, matching descriptors between frames.Integrated gyroscope measurements predict the matching window, while LOC mode establishes 2D-3D matches against a localization map.
  • System architecture: ROVIOLI’s modules exchange data through an internal publisher-subscriber layer that supports extensions to the online pipeline.

IV. USE-CASES

maplab demonstrates online mapping and localization workflows in which ROVIOLI creates an initial map, processes it offline, and later estimates global pose against it.

  • Use-case overview: maplab’s five example use cases cover online mapping and localization, multi-session mapping, map maintenance, large-scale mapping, and dense reconstruction.The examples are intended to illustrate system capabilities, expected performance, and scalability.
  • Online mapping and localization: A first session uses ROVIOLI to create a VI-map, which is automatically loop-closed, optimized, and optionally keyframed or summarized into a localization map.Input can be provided offline in a Rosbag or online through ROS topics.
  • Evaluation: Figure 5 overlays ground-truth and ROVIOLI positions and plots position errors for ROVIO, ROVIOLI, and VIWLS.
  • Online mapping and localization: In a second session, ROVIOLI uses the localization map to obtain drift-free global pose estimates within the mapped area.
  • Evaluation: The EuRoC evaluation compares ROVIOLI and ROVIO against a full-batch visual-inertial optimization using separate mapping and evaluation sessions.The comparison reports ground-truth position errors for the three approaches.

B. Multi-session mapping

maplab combines multiple mapping sessions, refines their shared geometry, and compresses large localization maps while preserving measurable localization recall.

  • Multi-session mapping: maplab co-registers maps from multiple sessions and jointly refines them to form a single consistent map.This supports environments that cannot be covered in one session or require varied visual-appearance conditions.
  • Multi-session mapping: Four trajectories cover over 1,000 meters and about 463,000 landmarks, motivating selective processing for large maps.
  • Multi-session mapping: Keyframing uses vertex distance, orientation, and landmark covisibility, while loop closure identifies inter-mission transformations and nonlinear optimization refines the geometry.The resulting localization map is 8.2 MB and covers the entire building.
  • Reproducible workflow: The multi-session workflow is exposed through console commands for loading, keyframing, optimization, mission anchoring, loop closure, and visualization.
  • Map maintenance: Landmark summarization uses integer-based optimization to remove less commonly observed landmarks while maintaining mapping quality.
  • Map maintenance: A 90% landmark reduction and combined keyframing can reduce map size 13 times while retaining 51% recall versus 60% for the full map.The evaluation tests localization against a database built from four sessions and accepts position errors below 50 cm.

D. Large-scale mapping

maplab supports large-scale, multi-session visual-inertial mapping and evaluates map reduction methods for localization. A Zurich deployment demonstrates the scale of the resulting maps, while keyframing and summarization trade map size against recall.

  • Map reduction: Keyframing reduces the map from 6,258 to 760 vertices, while consistently lowering recall by a few percent.Keyframing removes vertices and their vertex-landmark associations.
  • Map reduction: Summarization affects localization quality mainly when more than 85% of landmarks are pruned; random landmark removal provides a comparison curve.
  • Large-scale multi-session mapping: 45 Google Tango sessions produced a Zurich map with 16.48 km of trajectories, 435k landmarks, 7.3M observations, and a size of 480 MB.The recordings covered 231 minutes across sunny and cloudy days.
  • Large-scale multi-session mapping: The optimized Zurich map is consistent with buildings and streets across most of the area, with minor inconsistencies where coverage is low.

E. Dense reconstruction

maplab provides dense reconstruction tools that derive depth from stereo imagery or RGB-D data and attach the results to the VI-map. These resources can then support globally consistent volumetric surface reconstruction.

  • Dense depth computation: Dense reconstruction uses optimized sparse-map poses and camera images to compute depth information for applications requiring denser 3D environment representations.
  • Dense depth computation: For multi-camera systems, the tool selects stereo pairs suitable for planar rectification and computes trajectory-wide depth maps with a semi-global block matcher.Depth maps or point clouds are attached to the VI-map through its resource system.
  • Surface reconstruction: Once depth information is available, maplab uses globally consistent VI-map camera poses with voxblox for TSDF-based depth fusion and surface reconstruction.The resulting surface mesh is stored to the file system.

3) Export to CMVS/PMVS2:

maplab exports sparse VI-maps and images for CMVS/PMVS2 when more accurate dense reconstruction is required. Its console and plugin architecture also support reusable mapping workflows and integration of new algorithms, including online mapping and localization.

  • Export to CMVS/PMVS2: For more accurate dense reconstructions, maplab exports the sparse VI-map and images to the open-source CMVS/PMVS2 multi-view-stereo pipeline.RGB images produce the best results, although grayscale export is supported.
  • Research workflow: The framework makes algorithms and console commands available for mapping and localization, while helper functions and queries support rapid prototyping.
  • Research workflow: maplab’s extensible console plugin architecture enables integration of new algorithms, with extension examples provided in the project wiki.
  • Integrated framework: The framework integrates visual-inertial optimization, loop closure and localization, multi-session merging, pose-graph relaxation, visualization, and an online ROVIOLI frontend.ROVIOLI can build maps from raw visual-inertial data and track a global drift-free pose when given a localization map.
Loading 1711.10250v1…