Source-linked AI summary
DenserNet: Weakly Supervised Visual Localization Using Multi-scale Feature Aggregation
Dongfang Liu, Yiming Cui, Liqi Yan, Christos Mousas, Baijian Yang, Yingjie Chen
TL;DR
Visual localization must retrieve geographically relevant images despite ambiguous structures and changing conditions. DenserNet aggregates multi-level features, learns from GPS-tagged image pairs with weak supervision, and shares computation; experiments report competitiveness with state-of-the-art localization methods under the same supervision.
Problem
Visual localization requires discriminative image representations that distinguish nearby from distant locations despite similar structures, illumination changes, and occlusions.
Method
DenserNet aggregates lower-, mid-, and higher-level features and trains detection and description end-to-end with a weakly supervised triplet ranking loss using GPS-tagged image pairs.
Results
DenserNet is competitive with current state-of-the-art methods on large-scale localization tasks using the same level of supervision.
Takeaways & Limitations
DenserNet combines denser multi-semantic representations, weak supervision, and shared computation for visual localization under challenging conditions.
Abstract
from arXiv · showhide
In this work, we introduce a Denser Feature Network (DenserNet) for visual localization. Our work provides three principal contributions. First, we develop a convolutional neural network (CNN) architecture which aggregates feature maps at different semantic levels for image representations. Using denser feature maps, our method can produce more keypoint features and increase image retrieval accuracy. Second, our model is trained end-to-end without pixel-level annotation other than positive and negative GPS-tagged image pairs. We use a weakly supervised triplet ranking loss to learn discriminative features and encourage keypoint feature repeatability for image representation. Finally, our method is computationally efficient as our architecture has shared features and parameters during computation. Our method can perform accurate large-scale localization under challenging conditions while remaining the computational constraint. Extensive experiment results indicate that our method sets a new state-of-the-art on four challenging large-scale localization benchmarks and three image retrieval benchmarks.
Introduction
Visual localization compares a query image with GPS-tagged database images, but similar structures and environmental changes make discriminative representations difficult. DenserNet addresses these challenges by aggregating multi-level features, using weak supervision, and sharing computation for efficient localization.
- Introduction: Visual localization estimates a query image's geographic location by comparing it with GPS-tagged images, supporting applications such as robot navigation and autonomous driving.It can supplement GPS where signals are partially or completely shadowed.
- Introduction: The method targets image retrieval for localization, where nearby images should have similar representations and distant images dissimilar ones.Large-scale databases create ambiguities through similar landmarks, while illumination changes and occlusions alter appearances.
- Introduction: Conventional CNN localization methods mainly use one semantic feature level, motivating DenserNet's multi-level feature aggregation.The paper identifies multi-scale semantic information as a limitation of prior feature representations.
- Introduction: DenserNet aggregates lower-, mid-, and higher-level backbone features to produce denser keypoint features for challenging visual localization.The additional branches are designed to increase inlier matches and improve matching accuracy under environmental variations.
- Introduction: Weakly supervised training uses GPS-tagged image pairs and a modified triplet ranking loss instead of expensive pixel-level ground truths.Joint detection-description optimization encourages repeatable discriminative keypoints and improves description accuracy.
- Introduction: Shared features and parameters keep DenserNet's additional feature-extraction branches computationally efficient.The authors report limited additional computation despite aggregating more feature maps.
Related Work
CNN localization methods struggle with visually homogeneous scenes and changing conditions because conventional approaches rely on features from a single semantic level. DenserNet responds by producing more keypoint features through additional feature-extraction branches.
- Related Work: Conventional localization methods struggle with similar man-made structures and environmental changes that make image features less distinctive.Architectural similarities create feature homogeneousness, while changing conditions can degrade localization performance.
- Related Work: Using only one semantic feature level prevents conventional methods from capturing multi-scale details needed for feature learning and prediction.The paper describes the resulting semantic gap as a critical problem in localization.
DenserNet
DenserNet aggregates multi-level CNN features through shared computation, then uses a one-stage decoder and weakly supervised training to produce efficient image representations for visual localization.
- Feature Extraction Branch: DenserNet aggregates feature maps from different semantic levels through three branches, producing denser features while sharing backbone computation.The branches attach to lower-, mid-, and higher-level backbone features; adding branches improves density but increases memory use.
- One-stage Feature Decoder: The one-stage decoder combines attention filtering, local descriptors, and detection scores to delineate image representations.Attention filters suppress spatial locations associated with pedestrians, vehicles, and vegetation, while descriptors are normalized for feature correspondence.
- One-stage Feature Decoder: The detector selects strong channel responses and computes neighborhood softmax and image-level normalization to obtain pixel detection scores.The decoder’s detector produces n response maps and derives local detection scores around detected pixels.
- Time Complexity Analysis: Shared features and parameters limit the extra computation from DenserNet’s additional branches, supporting efficient inference under computational constraints.The paper attributes the increased cost mainly to the added branches and reports empirical runtime evaluation consistent with the complexity analysis.
- Training Objective: A modified triplet ranking loss jointly trains descriptors and detectors using positive and negative image references, without requiring pixel-level ground truths.The objective reduces descriptor distances for positive pairs, increases them for negative pairs, and encourages repeatable effective detections.
Experiment and Results
Experiments evaluate DenserNet on feature matching, large-scale localization, image retrieval, and runtime. Across these tasks, multi-branch feature aggregation improves matching and localization, while the two backbone variants deliver strong accuracy-efficiency trade-offs.
- Feature matching: DenserNet improves HPatches detector and descriptor metrics as branches are added, increasing repeatability, mAP, and matching scores with only small error increases.Qualitative results also show more keypoint features and more effective matching than the baseline.
- Ablation study: Adding feature-extraction branches steadily improves recall@N over the baseline and variants on all VGG-based localization benchmarks.The ablation compares a higher-level-only baseline with lower-level, mid-level, and full multi-branch variants.
- Large-scale localization benchmark results: DenserNet outperforms state-of-the-art methods on all four large-scale localization benchmarks, with Pitts250k r@1 gains of 3.45% using VGG and 1.87% using MobileNet over NetVLAD.On Sf-0, the VGG and MobileNet models achieve r@1 values of 80.08% and 79.12%, exceeding CRN by 3.18% and 1.5%, respectively.
- Image retrieval: On image retrieval benchmarks, DenserNet exceeds competing methods by 1–4% mAP, including 3.71% and 2.93% gains on Oxford 5k full for VGG and MobileNet.For Oxford 5k crop, the corresponding improvements are 3.88% and 2.74%.
- Runtime evaluation: The MobileNet-based DenserNet models are around 4× faster than VGG-based CRN and NetVLAD, while the VGG-based model has comparable speed to those same-backbone methods.Shared features in the feature extraction branch limit the additional runtime cost.
Conclusion
DenserNet aggregates denser features from multiple semantic levels to strengthen image representations for visual localization. Extensive experiments find it competitive with current state-of-the-art methods on large-scale localization tasks under the same supervision.
- DenserNet aggregates denser features from multiple semantic levels to achieve strong image representations.
- Extensive experiments indicate that DenserNet is competitive with current state-of-the-art methods on large-scale localization tasks.The comparison uses the same level of supervision.