Source-linked AI summary
AutoCompass: Accurate Visual Localization on Public Maps by Learning from Weak Labels
Javier Tirado-Garín, Alan Savio Paul, Shuai Chen, Axel Barroso-Laguna, Tommaso Cavallari, Daniyar Turmukhambetov, Victor Adrian Prisacariu, Eric Brachmann
TL;DR
Neural map matchers are trained with geo-referenced pose labels that often contain inaccuracies. AutoCompass learns from weak supervision using GPS positions and relative poses, and consistently outperforms strongly supervised methods across driving and egocentric benchmarks.
Problem
Large-scale geo-referenced image datasets contain noisy absolute position and heading labels, limiting reliable supervision for neural map matchers.
Method
AutoCompass uses tolerance regions around GPS positions, omits heading labels, and incorporates relative-pose supervision to train map matchers from weak labels.
Results
AutoCompass consistently outperforms strongly supervised methods across driving and egocentric benchmarks, with relative-pose supervision achieving a new state of the art.
Takeaways & Limitations
Neural map matchers can be trained accurately using raw GPS positions or relative poses instead of accurate geo-referenced 3-DoF labels.
Takeaways & Limitations
Relative translation supervision requires ground-truth translations expressed in the map’s geo-referenced coordinate system.
Abstract
from arXiv · showhide
Neural map matchers estimate an image's 3-DoF pose relative to a 2D map. These models are trained on large-scale datasets of geo-referenced images, whose position and heading labels often contain noise that affects the trained models. To address this, we present AutoCompass, a supervision approach for training neural map matchers from inaccurate absolute pose labels. First, we show that heading labels are unnecessary: trained from raw GPS labels, models learn to predict accurate headings, automatically. Second, defining a tolerance region around raw GPS improves positional accuracy. Third, if available, our supervision uses relative poses between training images, obtained via SLAM or SfM, which provide a more accurate training signal. Across driving and egocentric benchmarks, AutoCompass consistently outperforms counterparts trained with the usual strong reliance on absolute pose labels.
1 Introduction
AutoCompass addresses noisy absolute pose supervision in neural map matching by eliminating heading labels, tolerating GPS position errors, and exploiting relative poses. Across multiple datasets, it outperforms strong-supervision baselines and achieves state-of-the-art visual localization on public 2D maps.
- Motivation: The method targets noise in large-scale geo-referenced training data, where automated absolute position and heading annotations are susceptible to inaccuracies.Prior approaches commonly combine local SfM reconstructions with raw GPS measurements.
- Method: AutoCompass removes the need for geo-referenced heading labels and assumes the true image position lies within a tolerance region of up to 20 m around GPS coordinates.The approach relies on architectural inductive bias for accurate heading prediction and increases robustness to ground-truth inaccuracies.
- Method: AutoCompass provides absolute and practical relative pose losses, including variants for non-metric or approximately geo-referenced poses, to improve existing single-image supervised approaches.Relative poses can be estimated reliably using SfM or SLAM.
- Results: Raw GPS-trained AutoCompass models outperform strong-supervision baselines across multiple datasets, while relative pose supervision achieves state-of-the-art visual localization on public 2D maps.The method remains applicable to existing architectures without modification.
2 Related work
Cross-view visual localization estimates 3-DoF poses from large-scale georeferenced 2D maps, avoiding dense ground-level mapping. Recent weakly supervised methods reduce dependence on accurate pose labels, while AutoCompass uses noisy GPS or relative poses with perspective images and outperforms strongly supervised approaches.
- Map-based localization: Neural-map matchers use lightweight, publicly available maps and neural matching, reducing the need to build and store dense ground-level 3D representations.OrienterNet uses approximately 200KB for a 128x128m area and cross-correlates learned BEV image and map features.
- Cross-view visual localization: Cross-view methods localize images by estimating 3-DoF poses against 2D maps from aerial, satellite, or planimetric imagery, enabling wide-area localization without dense image-based maps.These maps are typically georeferenced and available at large scale.
- Weak supervision: Weakly supervised approaches address inaccurate pose labels through emergent pose estimation, panorama-based teacher-student learning, weak position supervision, orientation priors, or label-quality adaptation.These methods relax supervision under different assumptions, including panoramas, strong orientation priors, or heterogeneous ground-truth quality.
- AutoCompass: AutoCompass requires only perspective images plus noisy GPS measurements or relative poses for supervision and outperforms strongly supervised approaches.This contrasts with methods that assume accurate 3-DoF pose labels and addresses the effects of noisy geo-referencing.
3 Method
AutoCompass estimates a discrete 3-DoF pose distribution from a calibrated image and rasterized local map, while replacing exact absolute-pose supervision with heading marginalization, GPS-region supervision, and relative-pose losses. These mechanisms exploit geometric matching to tolerate inaccurate labels and use relative information from multi-view poses.
- Pose estimation: AutoCompass estimates a categorical distribution over discrete 3-DoF poses by cross-correlating learned map features with rotated BEV image features.The output is an S×S×N tensor normalized with softmax over candidate positions and headings.
- Automatic heading angle: Heading supervision is removed by marginalizing the predicted probability volume over all headings at the labeled position.Feature matching supplies a geometric inductive bias that enables heading prediction from single perspective images, with accuracy on-par with or better than heading-supervised methods.
- Relative poses: Relative rotation and translation supervision use relative information, making them invariant to shared offsets in absolute-pose labels.Relative translation still requires the ground-truth relative translation to be expressed in the same geo-referenced coordinate system as the map tiles.
- Learning from GPS: GPS supervision marginalizes probability mass within a ±r meter position neighborhood while also marginalizing over orientation, reducing sensitivity to biased or adverse-condition GPS errors.The method encourages probability anywhere within the GPS neighborhood, and similar performance is obtained for r ∈[5, 20] m.
- Relative poses: Relative poses from SLAM or SfM provide training supervision without explicit geo-referencing, provided a coarse 2D location is available to select a covering map tile.Relative rotation supervision is formed by circular cross-correlation of the two images’ marginalized heading distributions and optimized at the ground-truth relative rotation.
4 Experiments
Across driving and egocentric benchmarks, AutoCompass trained with raw GPS already outperforms strongly supervised baselines, while relative-pose supervision achieves the strongest results. Its gains extend to DINOv2 backbones, unseen urban scenes, and sequential fusion.
- Egocentric sequences: On LaMAria and ODN, AutoCompass generalizes to unseen urban scenes, outperforms alternatives trained on more data, and improves LaMAria strict-threshold performance over GPS.Using DINOv2 consistently improves over ResNet-101.
- Overall results: Across KITTI and egocentric benchmarks, AutoCompass outperforms strongly supervised and alternative methods, with relative-pose supervision performing best and DINOv2 further improving results.The egocentric evaluation reports the relative-pose, DINOv2 variant as best across both datasets.
- Sequential fusion: Under 10 fused frames, AutoCompass surpasses GPS recall@5m on LaMAria, whereas retrained OrienterNet requires over 20 frames.The result demonstrates that more accurate single-view estimates benefit sequential fusion.
5 Conclusion
AutoCompass enables 3-DoF visual localization on 2D maps using weak supervision from 2D GPS labels or relative poses, remaining robust to geo-referencing errors and outperforming strongly supervised methods.
- 5 Conclusion: AutoCompass consistently outperforms strongly supervised methods while handling GPS noise and absolute pose offsets.The approach uses weak supervision rather than accurate geo-referenced 3-DoF poses.
- 5 Conclusion: AutoCompass performs 3-DoF visual localization on 2D maps using only 2D GPS labels or relative poses.These supervision signals are easier to obtain at scale than accurate geo-referenced 3-DoF poses.
Supplementary Material
The supplementary material provides additional training and evaluation details, along with quantitative and qualitative results.
- Supplementary Material: Sections A–D provide additional training, evaluation, quantitative, and qualitative results.Sections A and B cover training and evaluation details, while Sections C and D present additional quantitative and qualitative results.
A Additional training details
The training setup uses perturbed raw-GPS-centered map tiles, SfM-linked image pairs for relative-pose supervision, and backbone-specific image preprocessing. It follows OrienterNet’s architecture while fine-tuning pretrained backbones with separate decoder training.
- Tile sampling: Training tiles are centered on raw GPS coordinates and randomly perturbed within ±48 m, while refined SfM coordinates are available in MGL.MGL provides raw GPS and SfM-refined coordinates; raw GPS centers the 128 × 128 m OSM tiles.
- Relative poses: Relative-pose pairs come from images jointly optimized in the same SfM cluster, with independent augmentation corrections applied before computing relative distributions.Clusters are identified through the Mapillary merge_cc attribute, and predicted probability volumes are untransformed before relative-pose computation.
- Images: Training images receive color jitter and are resized to 512×512 for ResNet or 518 × 518 for DINOv2 backbones.The MGL images are mostly square 90° FoV perspective crops with an average focal length of approximately 256 pixels.
- Architecture and hyperparameters: The setup uses 64 discrete rotations, a 64.5 × 32 m BEV at 0.5 m/pixel, and separate learning rates for the pretrained backbone and DPT decoder.The DINOv2 backbone is fine-tuned at 10^-7, while the decoder is trained from scratch at 10^-5.
- Relative poses: Relative-pose supervision derives distance, chunked distance, shift, and rotation probability distributions from paired absolute translation predictions.Figure 6 illustrates the distributions used for relative-pose supervision and notes that the raw distance distribution is non-smooth.
B Additional evaluation details
The evaluation uses standardized image preprocessing and dataset-specific protocols to address aspect-ratio mismatch, biased or locally inaccurate ground truth, and coordinate-system differences. Oxford Day-and-Night evaluation robustly aligns each model’s predictions with ground truth before computing errors.
- General: Across evaluation datasets, images are resized to a 256-pixel focal length, with LaMAria and Oxford Day-and-Night additionally cropped to 512×512 or 518×518 pixels for DINOv2.OrienterNet, OSMLoc, and AutoCompass use 256 discrete rotations during evaluation.
- KITTI: KITTI images are center-padded to 518×518 pixels for DINOv2-based models after focal-length resizing, while ResNet models only undergo focal-length resizing.The padding addresses KITTI’s approximately 3.3:1 aspect ratio versus the 1:1 training distribution; ResNet models benefit less from padding.
- LaMAria: LaMAria evaluation uses eleven training sequences with pseudo-dense ground truth, excluding clips recorded while surveying ground control points.Sequences require available pseudo-dense ground truth and more than one observed control point for geo-referencing.
- LaMAria: LaMAria headings are corrected by approximately 0.8° to convert grid north in LV95 ground truth to geodetic north in the evaluation coordinate system.The offset uses meridian convergence at the center of the topocentric system.
- Oxford Day-and-Night: Oxford Day-and-Night evaluation robustly aligns ground truth to each model’s estimates per sequence before computing errors, addressing meter-level geo-referencing errors and single-view outliers.The alignment estimates a 3-DoF position transformation and heading offset using truncated losses, with 2 m and 3° outlier thresholds.
C Additional quantitative results
Additional evaluations show that AutoCompass’s weak chunk-marginalization supervision consistently benefits from informative relative-pose labels, while remaining robust across protocols and parameter choices. GPS neighborhood marginalization improves performance over direct GPS supervision, whereas relative-distance binning is comparatively insensitive.
- Supervision impact: Across KITTI, LaMAria, and ODN, performance improves with more informative supervision: weak chunk marginalization outperforms strong supervision, and relative-pose labels perform best.The conclusions remain consistent across average and sequential results, although ODN shows similar performance across checkpoints.
- KITTI evaluation with a ±10° orientation prior: Under a ±10° orientation prior, AutoCompass still improves over strongly supervised baselines, with relative-pose supervision performing best among OSM-based methods.The protocol restricts orientation search to ±10° and position search to a 40×40 m area centered on the OSM tile.
- Varying the chunk size r for GPS-only supervision: GPS marginalization with chunk sizes r ∈[5, 20] m consistently outperforms strong supervision, while r = 0 and r = 30 m perform worse.Direct GPS supervision lacks local-neighborhood marginalization, whereas larger chunks weaken the training signal.
- Varying the norm binning size r_∆t: Relative-distance supervision achieves similar performance across different norm binning sizes r_∆t, indicating that this parameter is not sensitive.The comparison uses the additional experiments summarized in Table 7.
- GPS + relative pose supervision: Combining GPS with relative-pose supervision improves over GPS alone, but remains less accurate than using relative-pose supervision by itself.Increasing the supervised GPS neighborhood size improves the combined setup, unlike GPS-only supervision.
D Additional qualitative results
Qualitative comparisons across LaMAria, Oxford Day-and-Night, and KITTI show that AutoCompass learns sharper, more selective map features than strongly supervised baselines. This behavior remains consistent across the proposed weak-supervision types and becomes sharper with relative poses.
- Comparison across methods: Across LaMAria, Oxford Day-and-Night, and KITTI, qualitative visualizations compare AutoCompass with OrienterNet and OSMLoc using relative-pose supervision.The visualizations use AutoCompass trained with ΔΘ + ΔXY supervision from Table 3.
- Comparison across methods: AutoCompass learns sharper map features that distinguish narrow roads and alleys, focus on distinctive keypoints, and ignore uninformative regions such as large bodies of water.Feature norms track cross-correlation magnitudes, highlighting building corners and road intersections visible in the input images.
- Comparison across types of weak supervision: The sharper-feature behavior is consistent across the different weak-supervision types and becomes even sharper when relative poses are used.The qualitative comparison suggests that relative poses and GPS-neighborhood supervision are less noisy than absolute pose labels.