Source-linked AI summary
Reading the Room: Scale-ambiguous Semantic Registration of Sparse VSLAM Maps to Floor Plans in Partially Built Environments
Bjorn Stragier, Emilia Szymańska, J. Micah Prendergast, Lukas Rosenberger Schmid
TL;DR
Sparse, scale-ambiguous VSLAM maps are difficult to register to symbolic floor plans at partially built, repetitive construction sites, and existing methods address only some of these challenges. Reading the Room uses a shared semantic-geometric energy for global initialization and local refinement. Across 33 sequences at three sites, it improves median ATE by at least 4.5× over the strongest prior baseline.
Problem
Registering sparse, scale-ambiguous VSLAM maps to plans is difficult because the modalities differ and sites may be incomplete and visually repetitive.
Source: §I · p. 1Method
Reading the Room uses a robust, class-balanced semantic-geometric energy for global initialization by matching or an available anchor, followed by local refinement.
Source: §I · p. 2, §IV · p. 3Results
A ≥4.5× improvement in median ATE over the strongest prior baseline was achieved across 33 sequences at three active construction sites.
Source: §I · p. 2, §VI · p. 8Takeaways & Limitations
With an anchor, RR achieves median ATE below 2 m and is robust to anchor inaccuracy.
Source: §VI · p. 8Registration remains unreliable on the hardest site, where low coverage and symmetry drive all methods into incorrect minima.
Source: §VI · p. 8
Abstract
from arXiv · showhide
Many robotic construction applications, such as progress tracking, require registering a sensor trajectory to a prior 2D floor plan. Visual capture is attractive for its low cost and availability, but aligning a visual simultaneous localization and mapping (VSLAM) map to a floor plan is difficult; VSLAM recovers geometry only up to an unknown similarity transform, the sparse point cloud is cross-modal to the symbolic floor plan, sites are only partially built during capture, and construction sites are visually repetitive, degrading map quality while removing distinctive features. Existing methods address at most two of these challenges. We thus introduce Reading the Room(RR), a global-local approach to register scale-ambiguous, sparse, semantic VSLAM maps to partially built floor plans. Our key idea is to define a single robust, differentiable, class-balanced semantic-geometric energy function that governs all stages of the pipeline. We develop two global initialization schemes, leveraging a single inaccurate user-provided correspondence if provided, or estimating the initialization globally using a cross-modal matching network. Both schemes are designed to find initial candidates with respect to the energy function, which is then directly optimized for local refinement. We evaluate RR on 33 sequences across three active construction sites against classical and learning-based baselines, achieving a 4.5x improvement in mean absolute trajectory error over the strongest baseline in the general unconstrained setting, which increases to 7.6x when an anchor is provided.
I. INTRODUCTION
Registering visual trajectories to floor plans can support construction progress tracking, but sparse, scale-ambiguous VSLAM maps are difficult to align with symbolic plans in partially built, repetitive sites. Reading the Room combines a class-balanced semantic-geometric objective with global initialization and local refinement, and outperforms the strongest baselines across three construction sites.
I. INTRODUCTION
- Visual trajectory-to-plan registration supports construction progress tracking, while camera capture offers low cost, availability, and visual fidelity.Progress tracking against design intent is described as a direct lever to reduce project delays and cost overruns.Source: §I · p. 1
- Unknown scale, cross-modal representations, incomplete construction, and visual repetition jointly make sparse VSLAM-to-plan alignment difficult.Monocular VSLAM geometry is recovered only up to an unknown similarity transform; repetitive sites also degrade map quality and remove distinctive features.Source: §I · p. 1
- Reading the Room introduces global-local registration built around a robust, differentiable, class-balanced semantic-geometric energy and two initialization options.Initialization uses cross-modal dense feature matching or an available correspondence, followed by local refinement of the energy.Source: §I · p. 2
- A ≥4.5× improvement in median ATE over the most competitive baselines was achieved across 33 sequences at three active construction sites.The evaluation considers both unconstrained registration and a case with an operator-provided initial guess.Source: §I · p. 2
II. RELATED WORK
Prior floor-plan localization and SLAM-prior methods rely on assumptions that do not fit sparse, unscaled maps from partially built sites. Their assumptions include known scale, an already-built environment, or metric-range sensing.
A. Cross-view Localization
- Cross-view localization methods assume known metric scale and a fully built environment, unlike the setting considered here.These methods localize a query image directly against a floor plan without SLAM or multi-view constraints.Source: A. Cross-view Localization · p. 2
B. SLAM with Floor Plan Priors
- These prior-based approaches do not match registration from sparse monocular VSLAM maps at active construction sites.The setting lacks both direct metric-scale sensing and a guarantee that the built environment matches the plan.Source: A. Cross-view Localization · p. 2, B. SLAM with Floor Plan Priors · p. 2
- Floor-plan-prior SLAM methods use metric LiDAR and assume a known, already-built environment.FP-Loc tracks from a known start pose, while Zimmerman et al. use semantic object cues for global localization,.Source: B. SLAM with Floor Plan Priors · p. 2
C. Map Registration
Map-registration and feature-matching methods offer related alignment strategies, but their assumptions leave the sparse, unscaled, cross-modal construction setting unaddressed. The paper formulates alignment as recovering a similarity transform, optionally using one inaccurate correspondence.
C. Map Registration
- Dense point-cloud and occupancy-grid registration methods require metric scale or dense reconstructions unavailable from sparse monocular VSLAM.Manual tie points and fiducial markers are alternatives, while learned graph matching and region decomposition also rely on dense maps,,,.Source: C. Map Registration · p. 2
D. Cross-modal Feature Matching
- Dense local feature matchers perform well in repetitive low-texture settings, but existing cross-modal adaptations have not addressed sparse, unscaled VSLAM maps against symbolic plans.The overview describes dense cross-modal matching as the unconstrained initialization option in Reading the Room.Source: D. Cross-modal Feature Matching · pp. 2–3, Figure 2 · p. 3
III. PROBLEM STATEMENT
- The problem is to recover a similarity transform mapping a sparse semantic 3D map from its arbitrary frame into the floor-plan frame.The map contains landmark points, trajectory, and semantic classes; the plan is a 2D image with structural semantic masks.Source: §III · p. 3
- A single known but inaccurate trajectory-to-plan correspondence may optionally constrain registration, though the no-correspondence case is more general.The correspondence may be marked once by a site manager at capture start.Source: §III · p. 3
IV. APPROACH
Reading the Room reduces the representation gap before using a shared energy function for global initialization and local refinement. Preprocessing projects the map into a bird’s-eye view and estimates landmark semantics when they are absent.
IV. APPROACH
- The approach uses one energy function to guide global initialization and local refinement, with anchored or cross-modal matching initialization.The final local step refines the map-to-plan alignment.Source: §IV · p. 3
A. Preprocessing
- PCA estimates verticality, floor and ceiling points are filtered by height percentiles, and remaining points are projected into a shared bird’s-eye view.This reduces the alignment search from Sim(3) to Sim(2), comprising scale, vertical-axis rotation, and 2D translation.Source: A. Preprocessing · p. 3
- When map semantics are unavailable, landmarks receive classes from projected keyframes using SAM3 with floor-plan class names as prompts.Landmarks without a projected mask remain unlabelled and contribute only to the class-agnostic term when confidence is at least 0.5.Source: A. Preprocessing · p. 3
B. Energy Function
RR uses one semantic registration energy to score and refine candidate transforms, with global initialization adapted to whether an anchor is available. Its objective handles sparse, noisy landmarks and class imbalance in partially built, self-similar sites.
B. Energy Function
- The shared semantic registration energy governs RR’s components and can be replaced to adapt the architecture to other settings.The paper defines a semantic registration cost for its construction-site problem.Source: B. Energy Function · p. 3
- Candidate transforms are scored by same-class distances from sparse landmarks to floor-plan structures, using a precomputed distance transform and bilinear lookup.Only the plan is used as the reference, avoiding penalties from structures that have not yet been built.Source: B. Energy Function · p. 4
- Truncation and a Huber penalty limit the influence of clutter and large residuals while retaining useful gradients near good alignments.The paper uses τ = 25px and Huber threshold δ = 10px across experiments.Source: B. Energy Function · p. 4
- Averaging confidence-weighted residuals within each class and then across classes gives rare, distinctive classes weight comparable to common classes.This addresses self-similarity and the rarity of classes other than walls.Source: B. Energy Function · p. 4
C. Global Initialization
- With an anchor, global search fixes translation initially, searches scale and rotation, and relies on later refinement to recover small errors despite an inaccurate anchor.The robust cost can absorb local anchor deviations; the full transform is refined afterward.Source: C. Global Initialization · p. 4
- Without an anchor, RR fine-tunes cross-modal matching on augmented synthetic pairs, then scores sampled correspondence-based transforms on the full landmark cloud.The synthetic pairs model partial coverage, sparsity, occlusion, clutter, positional jitter, rotation, scale, and translation; inference retains the lowest-energy candidate.Source: C. Global Initialization · p. 4
D. Local Refinement
Local refinement jointly optimizes the similarity transform parameters against the semantic energy, using a bounded-memory quasi-Newton method and a closed-form gradient.
D. Local Refinement
- Local refinement jointly optimizes scale, rotation, and translation by directly minimizing the semantic energy with a bounded-memory quasi-Newton method.Optimization runs for up to 5,000 iterations with unconstrained parameter bounds.Source: D. Local Refinement · p. 4
- A closed-form gradient of the semantic energy is used to make the local optimization efficient.Source: D. Local Refinement · p. 4
- The dataset overview defines trajectory length, area spanned by the trajectory, and floor-plan coverage as dataset descriptors.Coverage is the average ratio of floor-plan area covered by the trajectory.Source: Table I · p. 5, D. Local Refinement · p. 5
V. EXPERIMENTS
The evaluation uses 33 VSLAM trajectories from three active construction sites, with manually floor-plan-aligned ground truth. It reports trajectory error and thresholded recall against classical, learned, and map-alignment baselines.
A. Experimental Setup
- The evaluation spans 33 trajectories across three sites, 14 floors, and two countries, using OpenVSLAM maps and expert-aligned ground-truth trajectories.Synchronized LiDAR and 360° imagery were captured; LiDAR was used only to estimate ground truth.Source: A. Experimental Setup · p. 5
- The sites increase in difficulty from small, distinctive Site A to large, low-coverage, structurally symmetric Site C.Site B has larger floors but relatively high trajectory coverage; Site C has the largest average floor area and 28.7% coverage.Source: A. Experimental Setup · p. 5
- Performance is measured by mean and median absolute trajectory error and recall at 1 m, 3 m, and 5 m error thresholds.The recall measure is the success rate below each threshold.Source: A. Experimental Setup · p. 5
- Baselines include SIFT with RANSAC, pretrained RoMa with MatchAnything, region decomposition, and anchored ICP and Semantic ICP.The ICP variants are initialized with the same anchor and iteratively refit scale, rotation, and translation.Source: A. Experimental Setup · p. 5
B. Trajectory Localization Performance
RR improves registration performance over prior approaches, but results vary with site difficulty and an anchor can be important for resolving symmetry. Residual VSLAM drift also leaves an approximately 1 m average error lower bound in the reported evaluation.
B. Trajectory Localization Performance
- Approximately 1 m average oracle residual error is a lower bound because the VSLAM trajectories retain drift after loop closure and global optimization.The authors leave non-rigid registration and floor-plan-conditioned trajectory deformation for future work.Source: B. Trajectory Localization Performance · p. 5
- Our approach reduces mean ATE by 72% compared with the most competitive baseline and improves median ATE by 359%, as outliers fail catastrophically and drive up mean ATE.RoMa [19]+MatchAnything [20], the strongest prior baseline, has recall 0 at Site B and Site C.Source: B. Trajectory Localization Performance · p. 5
- RR does not correctly register Site C without an anchor, where scene symmetry can flip sparse observations; an anchor can resolve the symmetry.The anchored method achieves median ATE of 4.37 m and 57% recall@5m on Site C, though scale is not quite perfect.Source: Figure 3 · p. 6, B. Trajectory Localization Performance · p. 6
- The results visual compares representative sequences from Sites A, B, and C in increasing difficulty, including a collapsed RoMa+MatchAnything alignment in Site B.Source: Figure 3 · p. 6
C. Ablation Studies
The ablations examine the energy components, global candidate scoring, local refinement, anchor accuracy, and correspondence scoring. They indicate that each stage contributes, while global initialization is especially important to finding a good alignment.
Energy Function
- The semantic term yields the largest improvement, and adding class-imbalance-aware aggregation further raises performance across all sites.The ablation also finds that robust geometric costs contribute; the authors emphasize accounting for self-similarity and scarce semantic landmarks.Source: C. Ablation Studies · pp. 6–7
Importance of the Global Initialization Scoring
- A 32-orientation-by-32-scale grid still leaves many solutions in suboptimal local minima, whereas pre-scoring candidates makes the search efficient.The comparison uses the same anchor and energy function, refining all 1,024 grid initializations.Source: C. Ablation Studies · p. 7
Importance of Local Refinement
- Local refinement consistently improves median ATE by approximately 24% on average, although the global stage dominates performance.The authors attribute global-stage importance to finding the correct local-minimum attraction basin.Source: C. Ablation Studies · p. 7
Robustness to Anchor Inaccuracy
- Median ATE drops only 4.4% with the noisy user-marked anchor compared with the ground-truth anchor.The authors report that the more accurate anchor tends to perform better, while the two-stage approach remains robust to inaccurate placement.Source: C. Ablation Studies · p. 7
Candidate Scoring vs. Correspondence Matching
- RANSAC's largest inlier set can select a suboptimal transform in self-similar or symmetric scenes; globally scoring confident network-predicted correspondences yields a better estimate.The resulting estimate can be passed to local refinement.Source: C. Ablation Studies · p. 7
D. Runtime
Reading the Room registers maps within a few seconds, with runtime shaped by the choice of global initialization. Correspondence-based initialization is faster overall, while anchored initialization has a less predictable runtime and higher accuracy.
D. Runtime
- Registration completes within a few seconds, and correspondence-based global initialization is approximately 6× faster overall than anchored search.Network inference accounts for about 75% of its runtime; anchored search scales with observed landmarks and is less predictable, while offering higher accuracy.Source: D. Runtime · p. 7
- Anchored local refinement is faster because it requires fewer iterations, but global initialization dominates total runtime.Correspondence-based runtime stays comparatively stable across sites, whereas anchored search grows on larger, more densely covered Site B.Source: D. Runtime · p. 7
VI. CONCLUSION
Reading the Room uses one robust, class-balanced semantic-geometric energy to initialize and refine scale-ambiguous VSLAM-to-floor-plan registration. It outperforms prior baselines across construction-site sequences, but low coverage, symmetry, and VSLAM drift leave substantial errors in difficult cases.
VI. CONCLUSION
- A robust, class-balanced semantic-geometric energy scores candidate transforms without correspondences and tolerates unbuilt structure and outlier landmarks.It weights rare but distinctive classes equally with common ones.Source: §VI · pp. 7–8
- 4.5× or greater improvement in median ATE over the strongest prior baseline, with median ATE below 2m when an anchor is provided.Evaluated across 33 sequences over three active construction sites.Source: §VI · p. 8
- The same energy drives global initialization through anchored search or cross-modal matching, then local refinement.Source: §VI · p. 8
- On the hardest site, low coverage and symmetry leave all methods vulnerable to incorrect minima.Source: §VI · p. 8
- Oracle analysis attributes approximately 1m residual error to VSLAM drift; future work may explore floor-plan-conditioned deformation and resolving symmetric ambiguity without anchors.The conclusion also identifies correspondence matching in self-similar environments as a future direction.Source: §VI · p. 8