Source-linked AI summary
OpenCVL: An Open, Diverse, and Large-Scale Dataset for Fine-Grained Cross-View Localization
Zimin Xia, Mubariz Zaffar, Junsheng Fu, Alexandre Alahi, Julian F. P. Kooij
TL;DR
Fine-grained CVL needs precise ground-image pose estimates, but existing datasets restrict diversity and in-the-wild imagery has noisy labels unsuitable for reliable evaluation. OpenCVL combines open, large-scale, heterogeneous imagery with pose curation and accurate evaluation splits. Experiments show that adding noisy but diverse in-the-wild data improves localization accuracy across test sets, although performance remains far from satisfactory in unconstrained settings.
Problem
Existing CVL datasets rely on high-end sensors that limit diversity, while abundant in-the-wild imagery has noisy geo-tags unsuitable for reliable fine-grained evaluation.
Method
OpenCVL combines accurately supervised high-end data with diverse in-the-wild imagery, corrects in-the-wild poses, and provides curated evaluation splits from permissive sources.
Results
Training with noisy but diverse in-the-wild data improves localization accuracy on all test sets.
Takeaways & Limitations
OpenCVL supports broader and more reproducible fine-grained CVL benchmarking while indicating that diverse noisy imagery can benefit model training.
Takeaways & Limitations
Performance remains far from satisfactory in unconstrained real-world settings, and training benefits depend on accounting for uncertainty in noisy data.
Abstract
from arXiv · showhide
Fine-grained Cross-View Localization (CVL) estimates the precise position and orientation of a ground-level image by aligning it with geo-referenced aerial imagery, offering a scalable alternative to Global Navigation Satellite Systems (GNSS) in challenging urban environments. Existing datasets rely on data collected with high-end sensor suites, which inherently limit image diversity and scalability. While in-the-wild images are abundant, their noisy geo-tags make them unsuitable for reliable evaluation. To bridge this gap, we introduce OpenCVL, a large-scale, diverse, and open dataset containing 617,388 ground-aerial image pairs spanning 41 cities across four European countries. All images are sourced from permissive platforms, ensuring long-term accessibility and supporting open and reproducible research. The training set combines images captured with high-end sensors with diverse in-the-wild imagery. We further develop a data curation framework that filters and corrects pose annotations to construct reliable in-the-wild evaluation data. In addition, OpenCVL includes dedicated cross-area and snowy test sets to assess generalization and robustness. Experiments with a state-of-the-art CVL model on OpenCVL show that incorporating noisy in-the-wild data consistently improves performance on clean test sets, suggesting a promising direction for scaling CVL with diverse real-world imagery.
1 Introduction
Fine-grained CVL seeks precise ground-image position and orientation from geo-referenced aerial imagery, but existing data is either sensor-limited or too noisy for reliable fine-grained evaluation. OpenCVL addresses these constraints with open, diverse data, curated poses, and evaluation splits designed for broader robustness.
- Motivation: Fine-grained CVL estimates a ground image’s precise geographic location and orientation by aligning it with geo-referenced aerial imagery.It offers a scalable alternative or complement to GNSS in urban environments affected by multipath effects.
- Limitations of Existing Data: High-end sensor-based datasets provide accurate poses but constrain geographic, temporal, camera, and mounting diversity.This acquisition paradigm is suboptimal for scaling CVL to diverse real-world use cases.
- Limitations of Existing Data: In-the-wild imagery offers broad camera, location, and viewpoint diversity, but noisy phone-grade GNSS tags are inadequate for fine-grained evaluation.The paper also identifies uncertainty about whether such data benefits fine-grained CVL training when combined with accurate data.
- OpenCVL: OpenCVL combines high-end sensor data with more diverse in-the-wild imagery and sources all images from permissive platforms.Its design spans four European countries and includes highly accurate test splits plus automatic filtering and pose correction for in-the-wild imagery.
- OpenCVL: OpenCVL provides cross-area, seasonal, and in-the-wild challenges for evaluating generalization, robustness, and performance under unconstrained conditions.The dataset is intended to support reliable evaluation while broadening training diversity.
- Main Finding: Training with noisy but diverse in-the-wild data improves localization accuracy on all test sets, supporting its use for scaling fine-grained CVL.The authors present this as a promising direction for using diverse real-world imagery.
2 Related Work
Cross-view localization research distinguishes coarse image retrieval from fine-grained pose estimation. Existing fine-grained datasets improve pose-based evaluation but remain limited in diversity, geographic coverage, or openness.
- Task Scope: Cross-view localization is commonly divided into image retrieval and fine-grained localization.Both tasks match ground-level imagery with geo-referenced aerial or satellite imagery.
- Image Retrieval: Cross-view image retrieval identifies the aerial image covering a query location within a known geographic database.Its objective is coarse localization at city- or country-scale coverage.
- Fine-Grained Localization: Fine-grained CVL estimates the query’s precise location and orientation within an aerial image using descriptor matching, bird’s-eye-view projection, or direct local correspondences.These approaches support geometric alignment at a finer scale than image retrieval.
- Dataset Landscape: Most earlier cross-view datasets lack precise camera-pose annotations, while fine-grained datasets often use limited camera types, viewing directions, geographic regions, or restrictive imagery sources.VIGOR covers four US cities, later datasets build on driving datasets, and SNAP relies on proprietary data.
3 The OpenCVL Dataset
OpenCVL combines accurate vehicle-collected imagery with diverse in-the-wild data to provide open, large-scale fine-grained CVL training and evaluation. Its curated splits support reliable testing across geographic, seasonal, and viewpoint variation.
- Data Sources: The dataset retrieves 100 m × 100 m high-resolution aerial images and randomly places camera locations within a centered 40 m × 40 m region.Aerial imagery comes from national mapping agencies in Sweden, Poland, Norway, and the Netherlands.
- Data Sources: OpenCVL combines ZOD’s accurate vehicle-mounted sensor data with diverse but noisier Mapillary imagery for scalable training.Mapillary contributes varied cameras, viewpoints, locations, and capture conditions, while ZOD provides reliable supervision.
- Data Splits: 617,388 ground-aerial image pairs are provided, including 579,752 training pairs, 14,756 validation pairs, and dedicated cross-area, snowy, and in-the-wild test sets.The training set contains 238,212 ZOD pairs and 341,540 Mapillary pairs; the specialized test sets contain 18,504, 3,015, and 1,361 pairs, respectively.
- Data Splits: ZOD validation and test samples are manually pose-verified by projecting camera poses and LiDAR into aerial imagery, filtering sequences with noticeable errors.Good alignment is checked geometrically against building edges and LiDAR points, while misaligned samples are removed.
- Data Splits: OpenCVL tests robustness through geographically unseen cross-area samples, winter scenes, and an in-the-wild split with diverse viewpoints and sensing conditions.The snowy split evaluates robustness to seasonal appearance variation rather than strict cross-season generalization.
- Comparison to Prior Datasets: OpenCVL spans 41 cities across four European countries and combines vehicle, handheld, bicycle-mounted, and dash-camera imagery under permissive licenses.This broadens viewpoints, weather, seasons, architectural styles, and road layouts while supporting long-term accessibility and reproducibility.
4 Experiments
Experiments evaluate Loc2 across OpenCVL and KITTI under varied training data, pose quality, orientation-prior, and transfer settings. Diverse OpenCVL training improves in-the-wild performance while preserving clean-set performance, although unconstrained and cross-dataset cases remain challenging.
- Experimental setup: Loc2 is evaluated on OpenCVL and KITTI using mean and median localization and orientation errors, with and without a ±10° orientation prior.Localization error is measured in meters and orientation error as absolute yaw-angle difference.
- Experimental setup: A 0.3 loss downweighting factor is applied to Mapillary samples to reduce the influence of noisier supervision during training.The study tests factors of 0.1, 0.3, 0.5, and 0.7 and selects 0.3 for the final setting.
- Training-data diversity: ZOD-trained models outperform KITTI-trained models on OpenCVL, while ZOD-only training performs worse on the in-the-wild split than on the ZOD-derived splits.The in-the-wild data has greater variability in camera types, viewpoints, and capture conditions than ZOD.
- Training-data diversity: Training on ZOD plus Mapillary improves localization and orientation on the in-the-wild test set while maintaining cross-area and snowy-set performance.Applying Mapillary loss downweighting further improves performance on the cross-area and snowy sets.
- Limitations and robustness: The in-the-wild evaluation remains difficult, and smaller gains without downweighting highlight the need to account for uncertainty in noisy training data.The reported performance remains far from satisfactory in unconstrained real-world settings.
- Cross-dataset transfer: OpenCVL-trained models achieve KITTI localization accuracy comparable to KITTI-trained models, but produce more approximately 180° opposite-direction errors.The authors hypothesize that direct KITTI training provides stronger domain-specific cues for distinguishing forward from opposite driving directions.
- Pose quality: Fine-tuning with Mapillary always improves performance, with corrected poses producing the lowest localization and orientation errors.Raw GNSS and OpenSfM-refined poses provide improvements but remain noisier than corrected poses for supervision and evaluation.
- Cross-dataset transfer: OpenCVL pre-training followed by KITTI fine-tuning yields the best overall performance, whereas joint training slightly improves localization but increases orientation error.The difference is attributed to KITTI’s ±10° orientation prior and OpenCVL’s lack of one.
5 Conclusion
OpenCVL is presented as an open, diverse, and large-scale benchmark for fine-grained cross-view localization. Its curated evaluation data and experiments show that noisy but diverse imagery can improve localization while exposing substantial challenges in unconstrained settings.
- Dataset and evaluation: OpenCVL combines high-precision data with in-the-wild imagery and provides cross-area, snowy, and in-the-wild test sets for broader benchmarking.A dedicated framework creates accurate labels for the in-the-wild imagery.
- Openness: All OpenCVL imagery comes from permissive sources, allowing the dataset to be freely shared and extended.The dataset is intended to support open and reproducible research.
- Findings: Experiments with Loc2 show that incorporating noisy but diverse data can significantly improve localization performance.The in-the-wild test set also reveals opportunities for further improvement.
Appendix
The appendix supplies supplementary material supporting the main paper, including dataset coverage, additional baselines, imagery discussion, and pose-correction visualizations.
- Supplementary material: The appendix lists OpenCVL city–country pairs, reports extra baseline methods, discusses orthophoto versus true orthographic imagery, and provides more pose-correction visualizations.These materials extend dataset documentation, benchmarking, imagery analysis, and qualitative analysis.
A. Complete City–Country List of OpenCVL
OpenCVL’s Mapillary training data covers 41 cities across Sweden, Poland, Norway, and the Netherlands, using approximately 1 km^2 tiles with up to 100 images per tile.
- 341,540 Mapillary training images cover 41 cities across Sweden, Poland, Norway, and the Netherlands.
- Each city is partitioned into uniform urban tiles of approximately 1 km^2.
- 100 images are downloaded per tile whenever enough images are available.
- The Mapillary subset also includes 1,361 test images geographically close to the ZOD data.
B. Extra Baseline Methods on OpenCVL
OpenCVL evaluates additional homography-based and global-descriptor baselines alongside Loc2, with Loc2 consistently achieving the strongest localization performance across evaluation splits.
- The comparison includes homography-based HC-Net and global descriptor-based CCVPE, trained and tested with the same data weighting.
- HC-Net warps each ground-level image into bird’s-eye-view perspective using image-specific HFoV and fixed remaining homography parameters.
- CCVPE matches limited-HFoV ground images to aerial-descriptor sectors while assuming a fixed 120° HFoV for all images.
- Loc2 consistently outperforms CCVPE and HC-Net in localization across all evaluation splits.
- The supplementary results are reported in Table 8, whose best values are highlighted in bold.
C. Discussion on Orthophoto vs. True Orthographic Imagery
OpenCVL uses aerial imagery operationally to mean overhead imagery, primarily orthophotos, while distinguishing these from ideal true orthographic projections and including both representations for Poznan.
- Aerial imagery denotes the overhead images used for cross-view localization, most commonly distributed as orthophotos.
- Orthophotos correct perspective effects through DEM projection and map-grid resampling but are not strictly orthographic, so building facades may remain visible.
- True orthographic imagery is an ideal nadir-view projection without perspective distortions or occlusions.
- Poznan includes both standard orthophotos and true orthographic imagery, but experiments use orthophotos consistently across locations.
D. More Qualitative Examples of Our Pose Correction Pipeline
Qualitative projections compare raw, OpenSfM, and corrected Mapillary poses, with corrected poses producing the strongest visual alignment between LiDAR structure and image content.
- The pose-correction visualization compares raw Mapillary, OpenSfM, and pipeline-corrected poses for projecting ZOD LiDAR into Mapillary images.
- Corrected poses make LiDAR structures such as poles, traffic signs, and buildings overlap well with structures visible in the images.
- These projections visually assess pose-estimation quality and the ground-truth poses of OpenCVL’s in-the-wild test set.
- Figure 9 presents the LiDAR projection visualization using the corrected pose.