Source-linked AI summary
X$^2$Localizer: Cross-grained Alignment for Progressive Cross-view Video Geo-localization
Zichao Zeng, Weijia Fan, Yufan Chen, June Moh Goo, Junwei Zheng, Ruiping Liu, Kunyu Peng, Jiaming Zhang, Rainer Stiefelhagen, Jan Boehm
TL;DR
Existing cross-view video geo-localization assumes complete offline sequences, limiting evaluation under progressive observations. This paper introduces PCVG, X2Localizer, and SWRL for prefix-based, cross-grained, and dynamically refreshed localization, improving progressive retrieval including the single-frame setting.
Problem
Existing cross-view video geo-localization typically assumes complete query sequences, lacking evaluation for progressive, arbitrary-start, and long-range localization.
Method
X2Localizer jointly aligns video prefixes with aerial images and frames with aerial tiles using budget-dependent weighting, while SWRL refreshes candidate regions during online inference.
Results
SWRL improves R@1/5/10 over fixed initial candidates in the challenging single-frame setting, while X2Localizer supports progressive localization across temporal budgets.
Takeaways & Limitations
PCVG provides a more deployment-oriented framework for evaluating and performing localization from partial observations and long video streams.
Abstract
from arXiv · showhide
Cross-view Video Geo-localization (CVG) aims to localize ground-view videos by retrieving their corresponding geo-tagged aerial images. However, CVG approaches rely on fixed-length inputs and post-hoc refinement, hindering online-oriented localization under partial or dynamic observations. In this work, we formulate Progressive Cross-view Video Geo-localization (PCVG) as a deployment-oriented extension and evaluation protocol of CVG, enabling localization under varying temporal budgets, prefix-based inference, random-start evaluation, and long-range localization with interruptions. To explore PCVG, we introduce X$^2$Localizer, a cross-grained alignment framework that jointly supervises global prefix-to-aerial retrieval and token-aggregated frame--aerial-tile matching with a budget-dependent asymmetric objective. Furthermore, we introduce a Sliding-Window Re-Localization (SWRL) strategy that dynamically refreshes candidate regions for failure recovery and long-range deployment without full-sequence reprocessing. Extensive experiments show that X$^2$Localizer preserves conventional full-video performance, with marginal gains of +0.1 Recall@1 and +0.3 Recall@10, while substantially improving early localization. In the challenging single-frame setting, X$^2$Localizer improves coarse retrieval by +4.7 Recall@1 and +11.5 Recall@10 over the previous state-of-the-art method. With SWRL, our approach further enables robust progressive localization under random-start and long-distance scenarios, narrowing the gap between benchmark evaluation and real-world deployment.
Introduction
The introduction reframes cross-view video geo-localization as Progressive Cross-view Video Geolocalization (PCVG), targeting deployment under partial, variable, interrupted, and long-distance observations. It presents X2Localizer and Sliding-Window Re-Localization (SWRL) as cross-grained alignment and recovery mechanisms for this setting.
- Motivation: Existing methods aggregate complete sequences or use camera geometry, while newer frame-level approaches generally assume offline access to the full sequence.These assumptions limit localization under partial observations and online deployment.
- Progressive Cross-view Video Geolocalization: PCVG evaluates localization under varying temporal budgets from single frames to full-length videos.The reconstructed GAMa protocol supports multi-duration prefix evaluation, random-start testing, and long-distance or interrupted scenarios.
- X2Localizer: X2Localizer addresses limited short-segment context through cross-grained, cross-view alignment.Its objectives combine global prefix-to-aerial alignment with token-aggregated frame–aerial-tile alignment and use asymmetric weighting according to available duration.
- Sliding-Window Re-Localization: Sliding-Window Re-Localization preserves stable alignment signals during long-range or interrupted video streams.The strategy supports localization despite interruptions or missing frames without requiring uninterrupted full-sequence processing.
2 Related Work
Related work progresses from single-image cross-view geo-localization toward sequence-based video matching. However, single-image methods lack temporal continuity, while some video approaches depend on camera intrinsics and odometry.
- Cross-view Image Geo-localization: Single-image methods learn view-invariant representations through Siamese or triplet-based metric learning to match ground-view images with geo-tagged aerial or satellite imagery.Representative methods address viewpoint discrepancies using orientation alignment, polar transformation, or dynamic similarity matching.
- Cross-view Image Geo-localization: Recent single-image work improves localization precision through fine-grained correspondence learning, pose-aware modeling, or large language models, but does not model temporal continuity.This limitation makes such methods insufficient for video-based progressive localization.
- Cross-view Video Geo-localization: SeqGeo aggregates short ground-view sequences for cross-view matching, while GAMa introduces a large-scale cross-view video dataset with hierarchical coarse-to-fine matching.CVLNet adds geometric projection and temporal constraints but relies on camera intrinsics and odometry.
3 Methodology
The methodology formulates PCVG as prefix-based cross-view retrieval under varying temporal budgets, addressing limited context and interruptions in long videos. X^2Localizer combines global prefix-to-aerial alignment, token-aggregated frame–tile matching, ranking distillation, and Sliding-Window Re-Localization.
- Progressive formulation: PCVG localizes video prefixes against global aerial images under single-frame, short-clip, half-length, and full-length temporal budgets.The model is designed to remain discriminative across every observable prefix length.
- Cross-grained alignment: For each temporal budget, asymmetric cross-grained alignment jointly supervises global prefix-to-aerial retrieval and fine-grained token-level frame–tile matching.Two-stage soft aggregation produces fine-grained similarities, with shorter prefixes emphasizing local evidence and longer prefixes emphasizing global alignment.
- Progressive formulation: PCVG addresses limited contextual information at short prefixes and robustness to interruption, restart, or cross-region transitions in long videos.These challenges distinguish the deployment-oriented formulation from conventional full-video CVG.
- Encoder adaptation: The framework preserves pretrained frame–tile representations while adapting frozen spatial backbones with lightweight GeoAdapter modules for video-to-global matching.Dual-tower ground-view and aerial-tile encoders are pretrained using frame-to-aerial-tile pairs before full-video global adaptation.
- Cross-grained alignment: Ranking distillation preserves the similarity structure learned during full-video adaptation while the objective applies retrieval supervision in the ground-to-aerial direction.This complements the global and fine-grained alignment losses across temporal lengths.
- Sliding-Window Re-Localization: SWRL periodically refreshes candidate regions from the current sliding-window prefix, enabling frame-level localization across cross-region or long-distance trajectories without full-sequence reprocessing.The initial coarse retrieval incurs a warm-start cost; subsequent localization uses the updated candidate tile gallery.
Experiments and Results
Experiments on GAMa show that X^2Localizer preserves conventional full-video retrieval while improving localization under short prefixes and single-frame inputs. SWRL further supports recovery after interruptions and long-distance online re-localization by refreshing candidate regions dynamically.
- Experimental Setup: X^2Localizer is evaluated on GAMa using train-day videos for training and val-day videos for evaluation under conventional CVG and proposed PCVG protocols.Each sample pairs an approximately 40-second street-view video with a global aerial image and frame-level geo-tagged aerial tiles.
- Coarse Retrieval: Under full-video evaluation, X^2Localizer performs comparably to GAReT, while consistently improving retrieval at 20 seconds, 5 seconds, and single-frame budgets.The largest improvement occurs in the most constrained single-frame setting.
- Frame-Level Localization: Frame-level predictions are correct when GPS error is within 0.05 miles, with fine-grained retrieval constructed from the top-10 global aerial candidates for GAReT and X^2Localizer.The evaluation reports Recall@1, Recall@5, Recall@10, Recall@1%, and inference latency.
- Sliding-Window Re-Localization: SWRL periodically refreshes candidate aerial regions from sliding-window prefixes, mitigating error accumulation and recovering from suboptimal initial candidates, especially with short warm-start budgets.This strategy enables continuous retrieval beyond the initially localized region during long-distance trajectories.
- Ablation Studies: Ablations show that asymmetric budget-dependent weighting is necessary: the symmetric variant performs worse, particularly at τ = 8, while removing either Lself or Lteacher lowers average performance.In the single-frame setting with Kc = 5, SWRL improves R@1/5/10 but slightly decreases R@1%.
5 Conclusion
The paper reformulates cross-view video geo-localization as Progressive Cross-view Video Geo-localization (PCVG), targeting deployment under varying temporal budgets, arbitrary starting positions, and long-range continuous streams. It proposes X2Localizer, which learns robust prefix representations through an asymmetric cross-grained alignment objective.
- PCVG requires localization under varying temporal budgets, arbitrary starting positions, and long-range continuous streams.
- X2Localizer addresses PCVG by learning robust prefix representations.
- Its asymmetric cross-grained alignment objective provides the learning framework for these prefix representations.