PaperlayerView exact revision on arXiv

Source-linked AI summary

MVP-SLAM: Multi-Camera Visual-Inertial Floorplan-Prior SLAM

Asier Bikandi-Noya, Miguel Fernandez-Cortizas, Muhammad Shaheer, Holger Voos, Jose Luis Sanchez-Lopez

arXiv:2609.39596v1cs.RO

TL;DR

Construction-site visual-inertial SLAM accumulates drift in challenging conditions, while prior floor-plan correction is often offline and online plan-integrating methods rely on depth or range sensors. MVP-SLAM uses two opposite-facing fisheye cameras and an IMU to map walls, match them to floor-plan counterparts, and integrate each match as an online persistent correction. On the Hilti–Trimble SLAM Challenge 2026, MVP-SLAM ranked second in Localization (0.29 m mean RMSE) and fifth in SLAM (0.24 m), and was the top team in both tasks among systems that operate online, integrate the floor plan, and localize within it.

  • Problem

    Construction-site visual-inertial SLAM accumulates drift in challenging conditions, while prior floor-plan correction is often offline and online plan-integrating methods rely on depth or range sensors.

    Source: §I · p. 1

  • Method

    MVP-SLAM uses two opposite-facing fisheye cameras and an IMU to map walls, match them to floor-plan counterparts, and integrate each match as an online persistent correction.

    Source: §I · p. 2, §V · p. 8

  • Results

    On the Hilti–Trimble SLAM Challenge 2026, MVP-SLAM ranked second in Localization (0.29 m mean RMSE) and fifth in SLAM (0.24 m), and was the top team in both tasks among systems that operate online, integrate the floor plan, and localize within it.

    Source: §V · p. 8

  • Takeaways & Limitations

    The challenge results show that an as-planned floor plan can serve as a reference for online drift correction and localization within the building.

    Source: §V · p. 8

Abstract

from arXiv · show

Indoor building construction sites are demanding environments for visual SLAM, where variable lighting and repetitive, low-textured structures make the system drift over long trajectories, though structural elements such as walls remain distinguishable despite these conditions. These buildings are constructed according to their as-planned floor plans, available from the design phase, and although the actual as-built site can differ from this design, floor plans still provide a metric reference, both to localize the system in the building and to correct drift. Existing methods often use the floor plan to correct an already-built trajectory offline, and those that instead correct it online typically rely on depth sensors. We instead present MVP-SLAM, an online visual-inertial SLAM on two opposite-facing fisheye cameras that corrects drift from cameras alone by matching walls detected in its map to the floor plan, through a drift-aware policy. A multi-stage integration then turns each matched pair incrementally into a persistent correction, so the trajectory stays corrected and localized within the floor plan as it is built. MVP-SLAM was validated on the multi-floor construction sites of the Hilti-Trimble SLAM Challenge 2026, ranking 2nd of 22 teams in the Localization task (0.29 m mean RMSE) and 5th of 62 teams in the SLAM task (0.24 m), the top-ranked one in both tasks among those that operate online, integrate the floor plan, and localize within it.

I. INTRODUCTION

Construction sites challenge visual-inertial SLAM with conditions that cause trajectory drift, while walls remain distinguishable and available floor plans offer a reference. MVP-SLAM addresses the gap between offline plan correction and online methods that rely on depth sensors by integrating wall matches online from cameras alone.

I. INTRODUCTION

  • Construction sites combine variable lighting, moving workers, fast motions, and repetitive, low-textured structures that cause visual-inertial trajectories to accumulate drift, although walls and columns remain distinguishable.Accurate localization supports workflows such as site-progress tracking and autonomous robotic inspection.Source: §I · p. 1
  • Prior floor-plan localization methods correct completed trajectories offline, whereas online construction-site methods integrate plan matches into SLAM but rely on active depth or range sensors.Reliable wall correspondences become harder as accumulated drift grows.Source: §I · p. 1
  • The Hilti–Trimble challenge uses two opposite-facing fisheye cameras and an IMU without depth, and its Localization and SLAM tasks score trajectory accuracy without requiring online operation.The two camera views have too little overlap to act as a stereo pair.Source: §I · pp. 1–2
  • MVP-SLAM integrates a floor plan into online visual-inertial SLAM, matching mapped walls to the plan and incrementally correcting drift as the trajectory is built.The system uses a multi-stage strategy to make each match a bounded, persistent correction.Source: §I · p. 2

II. RELATED WORK

Prior work either localizes within a floor plan without using it to improve the trajectory, aligns completed trajectories to plans offline, or integrates plans online using depth or range sensing. MVP-SLAM builds on graph-based visual-inertial SLAM and adds an online floor-plan pipeline.

II. RELATED WORK

  • Existing challenge systems leave trajectories in arbitrary frames without using available floor plans to localize or correct drift; top Localization teams align completed trajectories to plans offline.Other plan-localization methods return a global position without improving the trajectory or map, while Map-It Ralph relies on the provided anchor pose rather than plan geometry.Source: §II · p. 2, Localization with architectural priors · p. 2
  • Online architectural-prior SLAM methods correct drift using range or depth sensors, while a depth-free alternative requires a separately acquired, evolving photorealistic as-built mesh; camera-only online plan integration remains unexplored.The cited depth-free approach registers sparse map points to a digital twin.Source: Architectural priors inside SLAM · p. 2

A. System Overview

  • MVP-SLAM builds on ORB-SLAM3, a graph-based back-end that uses visual-inertial trajectory estimates and an as-planned floor plan to localize within the building and correct drift.Its pipeline detects mapped walls, matches them to plan counterparts, and jointly re-estimates trajectory and map.Source: A. System Overview · p. 2

B. Multi-camera visual-inertial SLAM

MVP-SLAM adapts monocular-inertial SLAM to two opposite-facing fisheye cameras that track against a shared map, with the IMU constraining one trajectory. It adds fast initialization, cross-hemisphere tracking, and inertial merging of maps after tracking loss.

B. Multi-camera visual-inertial SLAM

  • The two opposite-facing fisheye streams are tracked monocularly against one shared map, with features from both hemispheres and the high-frequency IMU constraining a single trajectory.The extension covers initialization, tracking, and map merging after tracking loss.Source: B. Multi-camera visual-inertial SLAM · p. 2
  • MVP-SLAM initializes by first trying cross-camera metric seeding from features in the narrow shared peripheral band, using surviving-match median depth to set scale.The resulting coarse map is refined by the first inertial bundle adjustment, which recovers depths and scale despite the 4 cm baseline.Source: B. Multi-camera visual-inertial SLAM · p. 3
  • When too few cross-camera matches survive, gyro-aided initialization uses preintegrated rotation and a 2-point RANSAC to recover translation direction.This avoids choosing between a homography and fundamental matrix in scenes with dominant planar surfaces and little parallax.Source: B. Multi-camera visual-inertial SLAM · p. 3
  • Each camera contributes observations to the shared visual-inertial state, and map points are searched in the camera that created them until a hemisphere change requires re-observation in the opposite camera.Temporal triangulation, point fusion, and culling support tracking across turns and corridor reversals.Source: B. Multi-camera visual-inertial SLAM · p. 3
  • After tracking loss, MVP-SLAM merges the new map with the interrupted map using IMU dead reckoning to estimate a yaw-and-position transform, keeping both segments in the first map's floor-plan frame.This lets walls matched in either segment constrain the whole trajectory, and shared structures can refine the join.Source: B. Multi-camera visual-inertial SLAM · p. 3

C. Semantic wall detection and floor-plan association

The association pipeline detects stable semantic wall planes, canonicalizes floor-plan walls, and conservatively matches them using a drift-aware distance policy. It rejects ambiguous matches before sending accepted wall pairs to the integration stage.

C. Semantic wall detection and floor-plan association

  • A pretrained panoptic segmenter labels image pixels, and per-frame projected labels are stabilized before map points are filtered and fitted into contiguous, repeatedly confirmed wall planes.Gravity, depth-band, and label-stability filters suppress spurious wall labels; confirmed planes become eligible for plan matching.Source: C. Semantic wall detection and… · p. 4
  • Floor-plan processing converts CAD wall layers into 2D faces, registers them to the plan raster, and canonicalizes the faces into a set comparable to the larger detected walls.Canonicalization merges collinear spans, removes duplicates or ambiguous faces, assigns outward normals, and completes missing opposite faces.Source: C. Semantic wall detection and… · p. 4
  • Because each accepted wall becomes a persistent constraint, association estimates drift from distance traveled since an exterior match with a parallel wall and widens the perpendicular-distance gate as drift grows.The gate is capped more tightly for interior partitions than exterior walls, whose spacing better tolerates drift.Source: C. Semantic wall detection and… · p. 4
  • When drift is large, the distance used in scoring saturates at dsat instead of continuing to dominate candidate selection.The blend factor σ rises from 0 to 1 as travel since the last correction increases from ℓmin to ℓmax.Source: C. Semantic wall detection and… · p. 4
  • The match cost combines wall-normal angle difference, drift-adjusted distance, and detected-wall extent contained in the plan face, with the lowest-cost candidate considered first.A rejection cascade discards candidates above a class-dependent score ceiling, without sufficient margin over the runner-up, or with a same-class competitor at comparable perpendicular distance.Source: C. Semantic wall detection and… · p. 4

D. Multi-stage floor-plan integration

MVP-SLAM integrates matched walls incrementally so floor-plan corrections reduce drift and persist as the trajectory is built.

D. Multi-stage floor-plan integration

1) Step 1: Local alignment mini-graph:

Step 1 aligns a newly matched map wall with its floor-plan counterpart in a bounded local graph, correcting the keyframes that observed it while preserving local trajectory structure.

1) Step 1: Local alignment mini-graph:

  • The local 4-DoF mini-graph optimizes yaw and translation, with roll and pitch fixed by gravity alignment.Source: 1) Step 1: Local alignment mini-graph · p. 5
  • Only observing keyframes, intervening keyframes, and their covisibility neighbours are free to correct, with long-span wall observations capped by a maximum window size.The mini-graph outputs corrected keyframe targets with associated wall normals for subsequent directional priors.Source: 1) Step 1: Local alignment mini-graph · p. 5
  • A high-information alignment edge pulls the detected wall onto the fixed plan face, while projection and covisibility edges constrain observing poses and preserve local trajectory shape.The alignment penalizes azimuth and signed-distance mismatch; plane-projection consistency uses the stored local observation measurement.Source: 1) Step 1: Local alignment mini-graph · p. 5

2) Step 2: Correction propagation to the system:

Step 2 propagates locally corrected poses to surrounding keyframes using an adaptive 4-DoF graph, then writes the optimized state back into SLAM.

2) Step 2: Correction propagation to the system:

  • An adaptive 4-DoF pose graph propagates Step 1 targets across a window bridging the corrected span to an earlier correction when possible.The graph combines covisibility and temporal-chain relative edges with unary priors from fresh targets and previous corrections.Source: 2) Step 2: Correction propagation to… · p. 5
  • The translation prior is stiff across observed wall normals but permits sliding along the wall and vertically, while the rotation prior constrains heading.Multiple non-parallel wall normals pin keyframes along each corresponding normal.Source: 2) Step 2: Correction propagation to… · p. 5
  • After propagation, poses and map points are updated and IMU preintegration terms re-evaluated to prevent later inertial optimization from pulling the trajectory toward its pre-correction state.Source: 2) Step 2: Correction propagation to… · p. 5

3) Step 3: Persistence in subsequent optimization:

Step 3 carries plan corrections into later optimizations, while the validation methodology evaluates trajectory accuracy, robustness, and wall matching on construction-site sequences.

3) Step 3: Persistence in subsequent optimization:

  • Each corrected target is stored with its wall-normal set and reapplied as a unary anisotropic prior in later optimization.Local inertial BA uses softer plan-prior information so visual and inertial terms can refine the map along walls while the plan prevents drift across wall normals; loop-closure and map-merge constraints apply only to keyframes associated with exterior walls.Source: 3) Step 3: Persistence in subsequent… · p. 6

A. Validation Methodology

  • The evaluation uses the Hilti–Trimble SLAM Challenge 2026 dataset, scoring Localization in the floor-plan frame and SLAM after rigid alignment using per-sequence APE RMSE.Localization has 24 scored sequences; a trajectory must cover at least 99% of reference poses to be scored.Source: A. Validation Methodology · p. 6
  • The plan-prior ablation disables wall associations and floor-plan integration while retaining the same visual-inertial SLAM setup.The comparison is against top-five competitors of each task, with per-sequence results from the official challenge release.Source: A. Validation Methodology · p. 6
  • A six-sequence ablation, with two sequences per environment type, measures initialization success, time to metric scale, trajectory coverage, and challenge coverage compliance.Source: A. Validation Methodology · p. 6
  • Wall detection and plan matching are evaluated on six sequences using precision, recall, and F1 against a hand-labeled reference of visible walls supported by reconstructed map points.Unbuilt planned walls and transient clutter are excluded; a match counts as correct only when the committed association links a detected wall to its manually labeled plan face.Source: A. Validation Methodology · pp. 6–7

1) Trajectory Estimation Performance:

MVP-SLAM ranked second in Localization and fifth in SLAM, with plan integration reducing aligned error. Map preservation improved underground robustness, while wall matching was highly reliable despite conservative detection.

1) Trajectory Estimation Performance:

  • 0.29 m mean APE ranked second of 22 in Localization, while 0.24 m mean APE ranked fifth of 62 in SLAM; plan integration reduced errors 4.9× and 3.0×, respectively.MVP-SLAM was the top-ranked team in both tasks among teams integrating the floor plan online; in SLAM, it was also the only one to localize within it.Source: 1) Trajectory Estimation Performance · p. 7

3) Wall Detection and Matching Performance:

  • Wall detection reached 96.9% precision but 38.4% recall, while wall matching had 98.9% pooled precision, recall, and F1; sparse or noisy map points chiefly limited detection.The detection policy is conservative because each accepted wall becomes a persistent plan prior, making a wrong wall more harmful than a missed one.Source: 3) Wall Detection and Matching… · pp. 7–8, Table IV · p. 8

2) SLAM Initialization and Robustness Ablation:

  • Map preservation merged trajectory segments after tracking loss, restoring full coverage and reducing Floor UG1 (06-18) Localization error from 28m to 2.07m.Without map preservation, abandoned keyframes dropped out, causing task failures and misaligning the floor-plan reference.Source: Table III · p. 7, 2) SLAM Initialization and Robustness… · p. 7

3) Wall Detection and Matching Performance:

  • Floor EG achieved 22 cm Localization APE despite 28.6% wall-detection recall when accepted walls were distributed.This indicates that sufficiently distributed accepted walls can support localization even when detection recall is low.Source: 3) Wall Detection and Matching… · p. 8

V. CONCLUSIONS AND FUTURE WORK

MVP-SLAM corrects construction-site drift online by matching mapped walls to an as-planned floor plan, with strong Hilti–Trimble challenge rankings. Future work targets denser structural maps and improved SLAM consistency.

V. CONCLUSIONS AND FUTURE WORK

  • 2nd in Localization (0.29 m mean RMSE) and 5th in SLAM (0.24 m), MVP-SLAM ranked highest among teams that operate online, integrate the floor plan, and localize within it.Source: §V · p. 8
  • MVP-SLAM tracks with two opposite-facing fisheye cameras and an IMU, then corrects drift online by matching mapped walls to the floor plan and integrating each match into the SLAM back-end.Source: §V · p. 8
  • Future work will explore lightweight map densification to aid wall and column detection, then integrate those structural primitives into drift-correction optimization.Source: §V · p. 8
  • Future work will also target internal SLAM consistency by strengthening loop closure and adopting richer-feature mechanisms used by higher-ranked SLAM teams.Source: §V · p. 8

Related papers

Loading 2609.39596v1…