PaperlayerView exact revision on arXiv

Source-linked AI summary

Spotter: Efficient Urban Visual Localization via Geo-Referenced Facade Landmarks in GPS-Degraded Environments

Antoni Valls, Jordi Sanchez-Riera

arXiv:2608.23290v1cs.CV

TL;DR

Urban wearable localization is challenged by GPS degradation, odometry drift, and the computational cost of global map-based methods. Spotter uses geo-referenced facade landmarks from GSV for GPS-aided or GPS-free pose estimation, achieving competitive accuracy with higher runtime efficiency on Barcelona smart-glasses sequences.

  • Problem

    Wearable urban localization lacks a reliable, efficient global reference when GPS degrades, while odometry drifts and map-based methods impose computational demands.

  • Method

    Spotter builds a compact GSV-based database of segmented facade features with metric coordinates and matches query images through retrieval, 2D–3D pose estimation, and optional GPS spatial filtering.

  • Results

    Spotter achieves competitive or superior localization accuracy with substantially higher runtime efficiency and stable performance under degraded or absent GPS.

  • Takeaways & Limitations

    Geo-referenced building facades support fine-grained global localization on resource-constrained wearable platforms without requiring continuous trajectory integration.

Abstract

from arXiv · show

Accurate visual localization on robotic and wearable platforms remains challenging in dense urban environments. Existing methodologies typically rely on GPS for absolute positioning, yet GPS signals frequently degrade in urban canyons due to multipath propagation. Consequently, standard solutions like visual odometry suffer from unmitigated drift over time, while map-matching techniques struggle to acquire the reliable GPS priors they need, on top of being too computationally heavy for real-time edge execution. To address these limitations, we propose Spotter, a robuts and real-time visual localization framework that uses building facades as a reliable source of global geo-reference, while retaining the capability to integrate GPS signals when available. In an offline stage, Spotter processes Google Street View panoramas by semantically segmenting facades and pairing multi-view stereo depth with cartographic data to build a compact metric database. At runtime, query images are matched via a cascaded retrieval and geometric verification pipeline to recover fine-grained global camera localization. We benchmark Spotter on a newly collected dataset of pedestrian sequences acquired with wearable smart glasses across several districts of Barcelona. Experimental results show that Spotter outperforms odometry-based baselines and achieves localization accuracy comparable to state-of-the-art map-based methods while operating at significantly higher frame rates.

I. INTRODUCTION

Urban visual localization must balance global accuracy with wearable-platform constraints: odometry drifts, while map-based and retrieval methods can require costly computation or reliable GPS priors. Spotter addresses this trade-off by using geo-referenced building facades for efficient GPS-aided or GPS-free localization.

  • Wearable platforms face strict sensing, computation, and energy limits in challenging urban scenes.Dynamic scenes, occlusions, and environmental scale further complicate localization.
  • VO and SLAM operate efficiently but accumulate unbounded drift when reliable global references or GPS correction are unavailable.Pedestrian trajectories often lack loop closures, while urban GPS blockage and multipath effects undermine correction.
  • Map-based localization provides global references but can require substantial computation and become sensitive to inaccurate GPS priors.Large candidate searches increase inference time, and GPS errors can expand or misdirect the search region.
  • Building facades provide stable, spatially distributed visual landmarks naturally associated with geographic coordinates.Their persistence makes them suitable for global localization in GPS-degraded or GPS-denied environments.
  • Spotter combines discriminative facade features with metric world coordinates in a compact framework that avoids dense 3D reconstruction and continuous trajectory integration.Its runtime pipeline supports optional GPS priors while retaining GPS-free operation.
  • On wearable smart-glasses data, Spotter achieves competitive or superior accuracy, higher runtime efficiency, and stable performance with degraded or absent GPS.The benchmark contains 20 pedestrian trajectories across diverse Barcelona urban environments.

II. RELATED WORK

Related localization methods trade lightweight sequential estimation against global consistency and computational cost. Spotter instead constructs a lightweight 3D geo-referenced landmark database and estimates global pose through image-to-landmark correspondences.

  • A. VO/SLAM Localization: VO and SLAM estimate motion sequentially with low memory, but remain vulnerable to accumulated drift over long trajectories.Degraded GPS makes long-range trajectory correction especially difficult without a reliable global reference.
  • B. Map-Based Localization: Digital cartographic maps offer compact structural representations compared with 3D point clouds.Map-based systems match street-level imagery against cartographic features for global pose recovery.
  • C. Image Retrieval: Retrieval methods provide absolute geographic references from geo-tagged imagery but require substantial image storage and search computation.Global descriptors estimate pose by identifying visually similar database images.
  • D. Spotter: Spotter estimates a 3-DoF global pose from a lightweight database of 3D geo-referenced visual landmarks.Its online engine establishes 2D–3D correspondences, solves a full 6-DoF pose with PnP, and derives the global pose analytically.
  • D. Spotter: The database pipeline samples viewpoints, segments building facades, and projects local image features into metric world coordinates verified against cartographic boundaries.This focuses representation and matching on persistent structural elements.

1) Strategic viewpoint sampling:

Spotter samples road-graph viewpoints, extracts facade-focused visual features, and geo-references them into a compact database for runtime localization. Cartographic filtering removes geometrically spurious landmarks while preserving an edge-suitable memory footprint.

  • 1) Strategic viewpoint sampling:: Road-graph viewpoints are sampled every δs = 30 m within a defined geographic search region.The search region is centered at c0 = (lat0, lon0) with radius r and built from the OSM road graph.
  • 1) Strategic viewpoint sampling:: Each sampled location yields four 90° views on straight segments and eight 45° views at intersections.GSV coordinates are updated to the panorama metadata anchor when it differs from the target sample point.
  • 2) Building segmentation and feature extraction:: LangSAM segments building facades so SuperPoint extracts local landmarks from persistent structural regions rather than dynamic clutter.Each landmark is represented by a keypoint-descriptor pair, with keypoints capped at 10,000 per image.
  • 2) Building segmentation and feature extraction:: NetVLAD global descriptors, local features, and camera metadata are indexed for coarse-to-fine visual retrieval.The global descriptor has dimension 4096, while local descriptors have dimension 256.
  • 3) Geo-reference 3D features:: OSM-guided spatial filtering removes candidate landmarks whose planar projections fall beyond a threshold from actual facade boundaries.This reduces storage overhead and eliminates geometrically spurious estimates.
  • 3) Geo-reference 3D features:: A 1 km2 region produces approximately 7,000 viewpoints and roughly 700 MB of database storage.The resulting memory footprint is described as suitable for edge deployment.

B. Online Localization Engine

The online engine uses positional priors and cascaded filtering to narrow reference viewpoints before visual matching. Spotter can obtain its prior from GPS or previous filtering, with adaptive search and heading constraints differing by mode.

  • SpotterGPS uses the current GPS reading, whereas Spotter uses the previous Kalman-filter state to seed candidate search.
  • The runtime pipeline comprises cascaded candidate retrieval, feature matching and pose estimation, Kalman filtering, and operating-mode gating.
  • The retrieval stage extracts 1,024 SuperPoint keypoint-descriptor pairs and a NetVLAD descriptor before candidate filtering.
  • A spatial filter retains viewpoints within radius rt of prior πt, with SpotterGPS using 15.0 · λt + σGPS and Spotter using 38.0 · λt.
  • The adaptive scale factor shrinks to max(0.5, λt × 0.85) after success and expands to min(2.0, λt × 1.30) after failure.The radius reaches its floor after approximately five consecutive successes and its ceiling after three consecutive failures.
  • SpotterGPS applies a GPS-derived heading filter and Euclidean radius, while Spotter bypasses heading filtering and uses a road-graph-constrained filter.

2) Feature matching and pose estimation:

Feature matching verifies retrieved facade candidates geometrically and fuses accepted pose estimates with a mode-specific Kalman prediction. GPS can provide directed motion, while Spotter can propagate external odometry after alignment.

  • Candidates are processed by visual similarity, matched with LightGlue, verified using MAGSAC homographies, and processed until enough strong matches are found.
  • The 2D Kalman filter tracks camera easting and northing with covariance across frames and uses PnP quality to set measurement noise.
  • In SpotterGPS, prediction uses a random walk or a directed GPS-heading variant with σq = vmax ∆t and vmax = 2.5 m/s.
  • In Spotter, prediction uses external odometry displacements aligned to UTM by online Procrustes analysis, with σq = 0.5 m for trusted odometry.
  • The Procrustes alignment is fitted after at least five PnP–odometry pairs spanning 5 m, leaving early predictions vulnerable to localization failures.
  • Each accepted PnP estimate is fused as a direct position observation, while high reprojection error or few inliers increase σr and reduce measurement weight.

4) Operating modes and jump gate:

Spotter operates without GPS by chaining the previous filtered position and constraining retrieval with a street graph. Both modes reject implausible pose jumps, protecting the temporal filter from gross outliers.

  • Without GPS, Spotter uses the previous filtered position as its frame-to-frame prior.
  • Street-graph retrieval follows road extensions on straight segments and explores reachable nodes with Dijkstra at intersections.
  • Chaining priors allows an incorrect prediction to compound across subsequent searches, making outlier rejection critical.
  • Both modes reject pose estimates whose deviation from the predicted prior exceeds a speed-, elapsed-time-, and PnP-margin-based threshold.The fixed single-view PnP margin is mPnP = 22 m.
  • All hyperparameters were selected empirically on a heldout Barcelona validation subset and fixed for reported experiments.

A. Dataset Construction

The Barcelona benchmark targets pedestrian visual localization among continuous building facades under varied GPS quality. It provides 20 manually ground-truthed sequences and evaluates accuracy, threshold recall, and processing speed.

  • The dataset was designed for pedestrian viewpoints and continuous building facades across Barcelona streets with varied GPS accuracy.
  • It contains 20 sequences across Gràcia, Guinardó, and Poblenou, totaling 15,616 frames, roughly 52 minutes, and nearly 4 km of trajectory.
  • GPS quality varies substantially: Guinardó and Poblenou have mean horizontal errors of 3.8 m and 3.9 m, while Gràcia reaches 13.0 m.
  • Gràcia errors range from 6–7 m in Sequences 01–02 to 14–20 m in Sequences 16–20 under heavy multipath conditions.
  • Complete ground-truth trajectories were manually annotated for all 20 sequences using GPS tracks and reference map data.
  • Evaluation reports Absolute Position Error, recall within 3, 5, and 10 m, and processing rate in frames per second.

C. Baselines

Spotter is evaluated against VO/SLAM, map-based, retrieval-based, and sensor-based baselines on 20 sequences. It combines competitive accuracy with high runtime efficiency and better resilience to GPS degradation than methods dependent on precise priors.

  • Baseline groups: The benchmark compares Spotter with VO/SLAM, map-based, and retrieval-based localization methods, including GPS-augmented variants.Evaluations use default parameter configurations and include Spotter, TartanVO, DPVO, MASt3R-SLAM, OrienterNet, OSMLoc, and AnyLoc.
  • GPS-aided results: 6.89 m mean APE and 80.9% R@10m make OSMLocGPS the strongest GPS-aided method under the dataset’s reliable GPS conditions.SpotterGPS records 8.13 m mean APE and 71.6% R@10m in the same comparison.
  • VO/SLAM comparison: Spotter outperforms standalone VO/SLAM methods in localization accuracy and processing efficiency, while drift accumulates in methods lacking global priors or persistent visual landmarks.The VO/SLAM trajectories are Sim(2)-aligned before APE computation for comparison.
  • Map-based limitations: Without a reliable GPS prior, map-based methods search larger regions, increasing pose ambiguity and computational overhead.These effects can cause severe mislocalizations and reduce inference speed.
  • Retrieval comparison: 121.65 m APE for AnyLoc falls to 11.71 m with a GPS prior, but SpotterGPS still achieves finer metric localization through explicit 2D–3D correspondences and PnP.AnyLocGPS remains limited by 30 m database sampling and records 8.4% R@3m.

E. Robustness to GPS Noise

Spotter is tested under controlled Gaussian GPS noise to isolate prior quality from scene and sequence effects. Its visual localization remains comparatively stable as noise increases, unlike methods that use GPS to select the matching map tile.

  • Zero-noise condition: 6.5 m versus 7.6 m accuracy at σ = 0 m shows OSMLocGPS competitive with SpotterGPS before noise is added.The single-sequence qualitative comparison is performed on Gràcia.
  • Qualitative behavior: Spotter remains broadly coherent under increasing GPS noise, whereas OSMLocGPS develops erratic jumps and off-road detours.The qualitative comparison is shown on a single Gràcia sequence.
  • Experimental setup: The ablation injects zero-mean Gaussian GPS noise with σ ∈{0, 5, 10, 20, 30} m into all sequences.This makes GPS noise a controlled variable for comparing GPS-dependent methods.
  • Noise sensitivity: At σ = 5 m, SpotterGPS becomes best overall with 8.21 m APE and 70.3% R@10m, while OSMLocGPS reaches 8.62 m APE and 69.8% R@10m.OSMLocGPS leads at zero added noise but degrades as the prior becomes noisier.
  • Noise sensitivity: At σ = 10 m, SpotterGPS holds at 8.97 m APE and 65.9% R@10m, while OSMLocGPS reaches 12.94 m APE and OrienterNetGPS 21.52 m.Figure 6 shows SpotterGPS remaining nearly flat up to 10 m while the other methods rise steeply.
  • Architectural explanation: Spotter uses GPS only to define a candidate search radius, leaving final pose estimation to visual matching and PnP when the prior is inaccurate.Tile-based methods instead use GPS coordinates to select the OSM tile, so GPS error directly corrupts the matching input.

F. GPS Denied Areas

Spotter provides GPS-free global localization from geo-referenced facade imagery on wearable platforms. In the reported evaluation, it outperforms the sensor-based pipeline and remains effective without satellite signals, while facade geometry also motivates future extensions.

  • GPS-free results: 12.17 m mean APE and 64.4% R@10m make GPS-free Spotter better than ZEDGPS at 12.36 m and ZED at 15.90 m.DPVO has lower APE at 9.78 m, but its result requires global Sim(2) alignment and is not achievable online.
  • Trajectory behavior: Spotter stays near ground truth when ZED loses track, and it can recover from small portions of drift during GPS-denied trajectories.Figure 7 compares both methods on representative sequences without GPS.
  • System design: Spotter uses a compact geo-referenced GSV database, with GPS acting only as a coarse spatial filter and metric pose determined by 2D–3D correspondences with PnP/RANSAC.This decoupling avoids computationally heavy 3D scene reconstruction.
  • Overall conclusion: On a wearable Barcelona dataset, SpotterGPS matches state-of-the-art map-based accuracy under reliable GPS while using substantially less computational overhead.The GPS-free variant also continues to localize effectively without satellite assistance.
  • Limitations and future work: Planar facades can make 2D–3D correspondences geometrically ill-conditioned, motivating added non-planar landmarks and evaluation across cities, seasons, and illumination changes.The authors also identify dynamic-occluder filtering as a future robustness enhancement.
Loading 2608.23290v1…