Where am I looking at? Joint Location and Orientation Estimation by Cross-View Matching

• Cross-view geo-localization estimates a ground camera’s latitude, longitude, and azimuth by matching it against geo-tagged aerial imagery.Source: §1 · p. 1, Figure 1 · p. 1
• Unknown relative orientation increases localization ambiguity and search space, while limited FoV exposes only local information that may match multiple aerial images.Source: §1 · p. 2
• Polar transformation approximately aligns aerial imagery with ground-view geometry by mapping corresponding horizontal and vertical scene structures.Source: §1 · p. 2
• The two-stream CNN extracts spatially aware ground and aerial feature volumes, whose correspondences support cross-view matching.Source: §1 · p. 2
• Dynamic Similarity Matching correlates ground and aerial features across azimuth angles, selects the maximum as orientation, and crops the corresponding aerial region for retrieval.The resulting feature distance provides a similarity measure for location retrieval, including limited-FoV images.Source: Figure 1 · p. 1, §1 · p. 2
• The method jointly estimates position and orientation and reports significant improvements over state-of-the-art methods across geolocalization scenarios.Source: §1 · p. 2

• Existing cross-view methods use CNN-based representations to replace hand-crafted feature matching for ground–aerial localization.Source: §2 · pp. 2–3
• The proposed framework combines a polar transform, a two-stream CNN, and Dynamic Similarity Matching to align domains and account for orientation.Source: Figure 3 · p. 3
• Prior approaches include fine-tuned, classification, hybrid, Siamese, triplet, transformation, generative, and feature-transport networks.Source: §2 · p. 3
• Many existing methods focus on location estimation, and some require orientation as an input rather than estimating it.Source: §2 · p. 3
• This paper jointly estimates ground-image location and orientation to facilitate cross-view matching for panoramas and limited-FoV images.Source: §2 · p. 3

3.1. A Polar Transform to Bridge the Domain Gap

• The polar origin corresponds to the aerial image’s geo-tag location, while 0° denotes northward direction and aligned-image upward direction.Source: §3.1 · p. 4
• Polar transformation approximately aligns aerial and ground spatial layouts, allowing CNNs to focus on feature correspondences rather than learning the geometric relationship between domains.The transform maps aerial-image geometry into a representation compatible with ground-view structure.Source: §3.1 · p. 4

3.2. A Spatially-Aware Feature Representation

• Spatially-aware feature volumes preserve scene layout information, but orientation changes make them difficult to compare, especially for limited-FoV ground images.A limited-FoV image may contain only a small sector of the aerial scene.Source: §3.2 · p. 4, §3 · p. 4, §3.3 · p. 4

3.3. Dynamic Similarity Matching (DSM)

• Dynamic Similarity Matching computes cross-view feature correlation across azimuth offsets, using the maximum similarity position to estimate ground-image orientation.The ground feature acts as a sliding window over aerial features.Source: §3.3 · p. 5
• For limited-FoV images, the method crops aerial features at the maximum-similarity position, re-normalizes them, and uses their L2 distance for matching.For panoramas, the maximum correlation is directly converted into an L2 distance from normalized features.Source: §3.3 · p. 5

3.4. Training DSM

• During training, DSM handles matching and non-matching pairs: it learns discriminative azimuth features for matches and minimizes maximum similarity for non-matches.Non-matching pairs are challenging when aligned because their similarity is larger.Source: §3.4 · p. 5

Datasets

• Experiments use CVUSA and CVACT, including CVACT_test for fine-grained city-scale geo-localization.CVUSA and CVACT contain 35,532 training pairs and 8,884 testing pairs; CVACT_test contains 92,802 pairs.Source: §4.1 · p. 5
• The constructed realistic test setting randomly rotates ground images and crops them to predetermined FoVs, removing known orientation.The experiments use CVUSA and CVACT_val image pairs with varying fields of view.Source: §4.1 · p. 5

Evaluation

• Top-K recall measures whether the corresponding aerial image appears among the K retrieved candidates.Retrieval ranks aerial images by L2 distance between global descriptors.Source: §4.2 · p. 6

Orientation-Aligned Panoramas

• The method achieves superior results to state-of-the-art methods on the challenging CVACT_test fine-grained localization benchmark.Success is defined by retrieving a top-K aerial image within 5 meters of the query location.Source: §4.3 · p. 7

Unknown Orientation and Limited FoV

• For unknown orientation and limited FoV, DSM reduces orientation-induced ambiguity and improves recall over existing methods across 180°, 90°, and 70° FoVs.Lower FoV makes ground images less discriminative, especially when orientation is unknown.Source: §4.4 · p. 7, Table 3 · p. 8
• Orientation is predicted correctly for almost all correctly localized ground images with 360° and 180° FoV.The orientation experiments evaluate ground images that were correctly localized at top-1.Source: §4.4 · p. 7, Figure 6 · p. 8, Table 4 · p. 8

Conclusion

• The proposed algorithm jointly recovers location and orientation when neither is known.It performs joint cross-view image matching.Source: §5 · p. 8, Figure 7 · p. 8
• The framework combines polar transformation with a Dynamic Similarity Matching module that regresses relative orientation.These components help extract aerial features when the ground image is disoriented and has limited FoV.Source: §5 · p. 8
• The method obtains higher location recall and significantly improves state-of-the-art performance across practical cross-view matching scenarios.The conclusion specifically highlights disoriented and limited-FoV settings.Source: §5 · p. 8

• The method achieves consistently better localization performance than state-of-the-art algorithms across unknown-orientation scenarios and varying FoVs.Source: §A.1 · p. 10
• Models trained with a FoV similar to the test image generally produce better localization results.The authors advise using a pretrained model with a similar FoV.Source: §A.1 · p. 10

• DSM estimates ground-image orientations across 360°, 180°, 90°, and 70° FoVs.Source: §A.2 · p. 11
• Small FoVs create high orientation ambiguity because the ground image may cover only a limited sector of the aerial scene.Source: §A.2 · p. 11
• Symmetric roads or scene contents can produce two similarity peaks, making a 180° orientation error possible.Source: §A.2 · p. 11
• When similar scene contents occur in multiple directions, orientation may be incorrect while the estimated location remains correct.Source: §A.2 · p. 11

• Similarity curves across azimuth angles use their maxima to indicate the estimated ground-image orientation.Source: Figure 11 · p. 12

• The network outperforms state-of-the-art methods and is more compact, facilitating deployment.Source: C. Trainable Parameters · p. 12, Table 6 · p. 12