Source-linked AI summary
AtLoc: Attention Guided Camera Localization
Bing Wang, Changhao Chen, Chris Xiaoxuan Lu, Peijun Zhao, Niki Trigoni, Andrew Markham
TL;DR
Single-image camera localization remains vulnerable to scene dynamics and changing appearance, while multi-frame or geometry-constrained methods address robustness with additional information or design constraints. AtLoc uses self-attention to emphasize geometrically robust features from one image and reports state-of-the-art results across indoor and outdoor scenarios, including performance beyond sequential techniques.
Problem
Single-image camera-localization methods lack robustness in scenes affected by dynamic objects, changing illumination, and other environmental variability.
Method
AtLoc is a self-attention guided neural network that extracts features from a single image, reweights them with attention, and regresses the camera pose.
Results
AtLoc achieves state-of-the-art camera pose-regression results across indoor and outdoor scenarios, outperforming previous and sequential techniques.
Takeaways & Limitations
Attention can encourage the network to learn geometrically robust features while mitigating impacts from dynamic objects and changing illumination in single-image localization.
Takeaways & Limitations
The method restricts orientation quaternions to one hemisphere because q and −q represent the same 3D rotation.
Abstract
from arXiv · showhide
Deep learning has achieved impressive results in camera localization, but current single-image techniques typically suffer from a lack of robustness, leading to large outliers. To some extent, this has been tackled by sequential (multi-images) or geometry constraint approaches, which can learn to reject dynamic objects and illumination conditions to achieve better performance. In this work, we show that attention can be used to force the network to focus on more geometrically robust objects and features, achieving state-of-the-art performance in common benchmark, even if using only a single image as input. Extensive experimental evidence is provided through public indoor and outdoor datasets. Through visualization of the saliency maps, we demonstrate how the network learns to reject dynamic objects, yielding superior global camera pose regression performance. The source code is avaliable at https://github.com/BingCS/AtLoc.
Introduction
AtLoc addresses the limited robustness of single-image camera localization by using attention to emphasize geometrically stable features and suppress dynamic content. The resulting single-image framework achieves state-of-the-art pose-regression performance across indoor and outdoor scenarios.
- Camera localization seeks to recover a camera’s 3D position and orientation from one or more images, supporting applications including virtual reality, delivery drones, and autonomous driving.
- Single-image methods can lose robustness in variable outdoor scenes containing moving vehicles, pedestrians, lighting changes, blur, and other appearance shifts.
- AtLoc uses attention to focus on temporally consistent, informative regions such as buildings while ignoring dynamic regions such as vehicles and pedestrians from a single image.
- The framework combines a Visual Encoder, Attention Module, and Pose Regressor to extract, reweight, and map image features into camera pose.
- AtLoc is presented as a self-attention guided neural network for accurate and robust single-image camera pose estimation.
- Visualization shows that attention encourages stable features, while experiments report state-of-the-art pose-regression performance across indoor and outdoor scenarios, including performance beyond multiple-frame methods.
Related Work
Prior camera-localization research progressed from manually constructed or learned visual features toward recurrent, probabilistic, residual, geometric, and synthetic-data approaches. AtLoc instead introduces attention to self-regulate feature selection without imposing temporal information or geometric constraints.
- Deep neural camera-localization methods learn features from data and estimate camera pose from a single image, avoiding hand-built landmark maps or databases.
- Prior work improved localization with spatial or temporal recurrent networks, Bayesian uncertainty estimation, and residual feature-extraction architectures.
- Other approaches add geometric constraints from paired images, synthetically augment training data, or use pose-graph optimization with neural graph models.
- AtLoc applies an attention mechanism to DNN-based camera localization so the network can self-regulate its focus instead of relying on temporal information or geometry constraints.
- Self-attention mechanisms have been used to capture long-term dependencies and have been integrated into translation, image generation, and non-local operations.
Attention Guided Camera Localization
AtLoc estimates camera pose from one image by combining a visual encoder, non-local self-attention, and a pose regressor. Its attention emphasizes informative, geometrically meaningful features, while an optional temporal extension adds constraints between image pairs.
- Attention Guided Camera Localization: AtLoc uses a visual encoder, attention module, and pose regressor to learn camera poses from a single image.The visual encoder compresses the image into features, attention produces attention-guided features, and the regressor maps them to position and orientation.
- Visual Encoder: The visual encoder uses an ImageNet-initialized ResNet34 to extract features needed for camera pose regression.Its final classification layer is replaced with a C-dimensional fully connected layer, and Softmax layers are removed.
- Attention Module: The attention module addresses overfitting to featureless appearance and environmental dynamics that can degrade generalization, especially outdoors.Outdoor changes include moving vehicles and weather variation; the model uses learned attention rather than temporal information or geometric constraints as its primary mechanism.
- Attention Module: Non-local self-attention captures long-range dependencies and global correlations, helping the model emphasize geometrically meaningful features without hand-engineered geometry.Feature similarities are computed across spatial positions, transformed features are aggregated, and a residual embedding produces the attention-guided representation.
- Learning Camera Pose: The pose regressor maps attention-guided features to a 3D location and quaternion orientation, optimized with a weighted L1 loss.The weights β and γ balance position and rotation losses, while unit quaternions are restricted to one hemisphere so equivalent rotations receive a unique representation.
- Temporal Constraints: AtLoc+ extends AtLoc with temporal constraints between image pairs, combining absolute pose learning with relative-pose supervision.The coefficient α weights the relative-pose loss against the absolute single-image pose loss.
Experiments
The experiments compare camera-localization models and include implementation details, temporal-constraint training, and saliency-map visualizations.
- Implementation: Images are rescaled with the shorter side at 256 pixels, normalized to [-1, 1], and cropped to 256 × 256 pixels.
- Implementation: The network uses a pretrained ImageNet ResNet34 encoder, while remaining components receive random initialization.
- Evaluation: Table 4 reports median and mean position and rotation errors for PoseNet+, MapNet, AtLoc, and AtLoc+ across LOOP and FULL sequences.
- Qualitative Analysis: Figure 3 compares PoseNet and attention-based AtLoc saliency maps for two Chess scenes.
- Temporal Constraints: Temporal-constraint training samples consecutive triplets every 10 frames.
7 Scenes
Experiments evaluate AtLoc on indoor 7 Scenes and outdoor Oxford RobotCar data, showing improved accuracy and robustness through attention, including under dynamics and illumination changes.
- 7 Scenes: The 7 Scenes dataset contains seven static indoor office scenes captured with RGB-D images across multiple sequences and camera motions.
- 7 Scenes: AtLoc improves position accuracy by 13% and rotation accuracy by 7% over the best single-image baseline.
- 7 Scenes: On Stairs, AtLoc reduces position error from 0.35m to 0.26m and rotation error from 12.4° to 10.5°.
- 7 Scenes: AtLoc focuses on geometrically meaningful areas and remains more temporally consistent than PoseNet’s scattered saliency maps.
- Oxford RobotCar: On Oxford RobotCar LOOP1, AtLoc improves mean position accuracy from 25.29m to 8.61m over PoseNet+.
- Oxford RobotCar: AtLoc reduces outliers by focusing on unique, static, stable areas such as vanishing lines, points, and buildings instead of roads, cars, and bicycles.
- Robustness Analysis: Under dynamic vehicles and changing illumination, AtLoc produces more stable feature distances than PoseNet+, with spikes mainly when large trucks block or reveal substantial view areas.
- Ablation Study: Adding attention improves 7 Scenes location accuracy by 9% and rotation accuracy by 6% over the basic model.
Conclusion and Discussion
AtLoc uses self-attention to improve single-image camera localization under scene dynamics and changing illumination. It achieves state-of-the-art results and surpasses sequential techniques in challenging scenarios.
- Conclusion and Discussion: Self-attention guides single-image camera localization toward geometrically robust features.This helps mitigate impacts from dynamic objects and changing illumination.
- Conclusion and Discussion: The framework achieves state-of-the-art results in challenging camera-localization scenarios.Its reported performance surpasses sequential-based techniques.
- Conclusion and Discussion: Future work will refine the attention module and test whether it improves multi-frame camera pose regression.