PaperlayerView exact revision on arXiv

Source-linked AI summary

Pixel-wise Geo-registration of Drone and Satellite Images

Qingyang Liu, David G Shatwell, Parth Parag Kulkarni, Mubarak Shah

arXiv:2608.28891v1cs.CV

TL;DR

Pixel-level drone-to-satellite geo-registration remains underexplored because existing benchmarks and methods mainly provide or evaluate a single camera-level GPS coordinate. SkyReg introduces dense supervised data, standardized benchmarks, and a geometry-aware 3D reconstruction pipeline, achieving substantially lower geodetic error and higher recall than retrieval and homography baselines.

  • Problem

    Existing cross-view geolocalization benchmarks largely provide single GPS labels, while satellite–drone registration requires dense pixel-wise geographic correspondence under viewpoint, occlusion, and modality differences.

  • Method

    SkyReg uses feed-forward 3D reconstruction with camera parameters and dense point maps to warp a single drone image into a georeferenced satellite frame and assign pixel-wise GPS coordinates.

  • Results

    SkyReg consistently outperforms retrieval and homography baselines across dense and image-level evaluations, reducing pixel-wise mean geodetic error versus RoMa by 88.18 m in Urban and 90.76 m in Suburban splits.

  • Takeaways & Limitations

    Dense geodetic supervision and explicit scene-geometry modeling support robust single-image drone–satellite registration across challenging urban and suburban settings.

  • Takeaways & Limitations

    The reference-side formulation assumes known satellite camera parameters and that reference pixels or DEM locations map to 3D positions in an ECEF/WGS-84 frame.

Abstract

from arXiv · show

Pixel-level cross-view geo-registration aims to align a query image (e.g., drone) to a geo-referenced satellite map so that every query pixel can be mapped to real-world GPS coordinates. Despite strong progress in cross-view geo-localization, existing benchmarks largely provide only GPS labels, limiting evaluation to a single coordinate per image and leaving dense geodetic alignment underexplored. We introduce SkyReg, a dataset and standardized benchmark for pixel-level drone-to-satellite geo-registration, providing dense per-pixel geo-location supervision across diverse settings (orthographic and perspective), scene types (urban, landmark-centric, suburban/rural), and camera configurations. Using SkyReg, we evaluate a broad set of baselines spanning retrieval, feature matching, homography-based alignment, and feed-forward 3D reconstruction. Finally, cross-view pairs from SkyReg, we train a geometry-aware reconstruction pipeline that achieves state-of-the-art results,improving performance by a significant margin.

1 Introduction

Existing cross-view geolocalization methods estimate a camera-level location, but dense pixel-to-world alignment is needed for scenes with depth variation and oblique viewpoints. SkyReg addresses this gap with dense supervision, standardized evaluation, and geometry-aware registration across challenging drone–satellite settings.

  • Motivation: Cross-view geolocalization paradigms reduce the task to estimating the query camera’s geo-location from a geo-referenced satellite view.Existing approaches use retrieval or 3-DOF pose regression, but both ultimately produce camera-level localization.
  • Motivation: Pixel-level geo-registration is motivated by applications requiring the geographic location of visible scene content rather than only the camera center.Small camera-localization errors can create larger spatial displacement for distant structures, especially in urban scenes with depth variation and oblique viewpoints.
  • Research gap: Existing geo-registration methods often assume small viewpoint changes, planar geometry, or sparse correspondences, leaving satellite–drone dense geodetic alignment underbenchmarked.The modality gap between orthographic satellite imagery and perspective drone imagery makes standardized dense evaluation particularly important.
  • SkyReg: SkyReg models 3D scene geometry with reconstruction backbones to predict pixel-wise GPS coordinates under occlusions and large viewpoint differences.It composes geometric mappings from query and reference image planes into a shared 3D frame, then warps query pixels into the reference plane for latitude–longitude assignment.
  • Datasets and benchmark: SkyReg-Train provides diverse drone–satellite pairs with dense latitude–longitude labels, depth maps, and camera parameters, while SkyReg-Bench tests unseen Urban and Suburban scenes.The benchmark includes varied scene layouts and reference modalities, supporting evaluation under distribution shifts.
  • Evaluation: SkyReg’s standardized protocols benchmark retrieval, feature-matching, and homography baselines using geodetic error and recall, showing consistent improvements.The benchmark is designed to evaluate fine-grained cross-view geo-registration rather than only image-level localization.

2 Background and Related Work

Prior work largely treats geolocalization as image-level camera-center estimation or uses planar and multi-view registration assumptions. SkyReg builds on feed-forward 3D reconstruction to support single-image, dense pixel-level alignment between perspective drone imagery and geodetically accurate satellite references.

  • Geolocalization: Cross-view geolocalization methods estimate a query image’s GPS coordinates through classification, retrieval, or related image-level paradigms.Retrieval transfers coordinates from geo-tagged references, including top-down satellite images.
  • Geolocalization: Image-level geolocalization can recover camera location or orientation but does not provide geographic coordinates for individual visible objects.A camera-center estimate does not determine the real-world position of structures or vehicles within the image.
  • Geo-registration: Geo-registration seeks a mapping from query-image pixels to an Earth-fixed reference coordinate system and corresponding geodetic coordinates.The formulation aligns query pixels with locations in a geodetically accurate reference image.
  • Prior registration methods: Earlier registration approaches commonly rely on DEM rendering, feature matching, or planar homographies, which can be brittle in nonplanar scenes with strong 3D structure.Later methods improve correspondence or reconstruct surfaces, but many still retain 2D alignment assumptions.
  • 3D reconstruction: Feed-forward reconstruction models predict dense point maps and camera parameters in a shared coordinate frame, but their performance can degrade across satellite–drone appearance and viewpoint gaps.Aerial MegaDepth provides calibrated aerial geometry but does not target satellite-conditioned pixel-level geo-registration.
  • Open gap: The field lacks satellite-conditioned training data and standardized pixel-level evaluation for single-drone-image registration under parallax, occlusions, and modality differences.This gap motivates SkyReg’s geometry-aware model, dataset, and benchmark with dense geodetic supervision.

3 Method

SkyReg predicts dense 3D point maps from query–reference image pairs, estimates camera geometry, and composes these predictions into a warp that assigns WGS-84 coordinates to every query pixel.

  • Feed-Forward 3D Reconstruction: SkyReg first predicts dense point maps, camera intrinsics, and relative pose, then warps query pixels into the reference image plane.The feed-forward reconstruction backbone produces point maps in shared camera frames; the method estimates intrinsics and relative transformation before projection.
  • Inputs and Geometry: Unlike planar homography methods, the formulation models 3D geometry without assuming a flat ground plane, addressing parallax and terrain relief.The reference may provide a DEM, while each reference pixel is assumed mappable to the WGS-84 ellipsoid in ECEF coordinates.
  • Pixel-Wise Geo-Registration: The query-to-reference warp projects each query point map through reference intrinsics and homogeneous normalization.The resulting reference-plane location is x′q = h^-1(Kr Xq,r(xq)).
  • Pixel-Wise Geo-Registration: Each warped query pixel is mapped through the reference geodetic function to a global 3D location or latitude–longitude coordinate.The pipeline applies the ellipsoid mapping E(·) to the warped reference position and can also form a query-over-reference mosaic.
  • Training Details: The MASt3R-based model is trained for 20 epochs on balanced Urban, Landmarks, and traditional 3D reconstruction pairs.Each epoch contains 20k pairs from each source group, preserving geometric priors while adapting to cross-view registration.
  • Training Details: Training uses confidence-weighted point-map regression, with predicted confidence regularized alongside the regression loss.The loss sums confidence-weighted regression terms over valid supervised pixels and includes a confidence regularization term controlled by α.

4 Dataset

SkyReg provides dense geodetic supervision across Urban, Landmarks, and Suburban subsets, with varied reference modalities, camera configurations, and held-out geographic settings.

  • Dataset Overview: SkyReg comprises SkyReg-Train and SkyReg-Bench, providing dense per-pixel GPS coordinates, metric depth, and full 6-DoF camera poses.These annotations enable evaluation beyond image-level geo-localization.
  • Dataset Structure: The dataset supports orthorectified satellite tiles with DEMs and perspective satellite views with metric depth and camera parameters.Drone queries are perspective images with metric depth and camera parameters in the ECEF frame.
  • Dataset Composition: Urban, Landmarks, and Suburban subsets cover U.S. cities, worldwide landmarks, and non-urban or disaster-affected environments.Urban samples Chicago, San Francisco, and Seattle; Landmarks contains 83 worldwide scenes; Suburban follows WRIVA imagery distributions.
  • Train and Test Splits: SkyReg-Train combines Seattle and San Francisco Urban scenes with all Landmarks scenes, balancing scene breadth against viewpoint diversity.Urban uses controlled acquisition settings, while Landmarks randomizes camera intrinsics and extrinsics.
  • Train and Test Splits: SkyReg-Bench tests strict geographic generalization using unseen Chicago Urban scenes and all Suburban scenes.The benchmark differs from training in locations, environments, sensor models, and scene layouts.
  • Data Collection Details: Urban supervision uses USGS LiDAR to derive DEMs and dense drone depths through projection and ray-traced mesh intersections.LiDAR is densified into a triangular mesh, and per-pixel depth comes from ray intersections with that mesh.
  • Data Collection Details: Landmark depth is generated with COLMAP SfM/MVS, yielding 7.7k satellite and 87k drone perspective images with depth and poses.Satellite viewpoints are rendered with randomized camera configurations around sampled ground-plane anchors.

5 Benchmarking and Evaluation

SkyReg is evaluated with dense pixel-wise and image-level geodetic protocols, alongside recall under distance thresholds. It consistently outperforms retrieval and homography baselines, with especially large gains in challenging Urban scenes and improved generalization from diverse training data.

  • Evaluation Protocol: SkyReg predicts dense query GPS coordinates by warping valid query pixels into the reference image and interpolating its geodetic map.The model uses predicted dense 3D point maps and camera parameters to define the cross-view warp.
  • Evaluation Protocol: Median geodetic error is computed from per-pixel Haversine distances between predicted and ground-truth latitude–longitude coordinates.Ground-truth coordinates are derived from known depth maps and camera parameters.
  • Evaluation Protocol: Image-level evaluation summarizes predicted and ground-truth pixel coordinates by their medians, while recall measures successful registration below multiple distance thresholds.This complements dense evaluation with a single GPS estimate per query and a robustness measure for failed registrations.
  • Results and Discussion: 88.18 m and 90.76 m lower pixel-wise mean geodetic error than RoMa are reported on the Urban and Suburban splits, respectively.Median-GPS mean geodetic error is also 68.30 m and 76.33 m lower than RoMa on those splits.
  • Results and Discussion: SkyReg improves geodetic error and recall over retrieval and homography baselines, with the strongest gains on Urban scenes containing tall structures and strong parallax.RoMa can fail under viewpoint and scale changes or distort elevated structures because planar homography cannot model parallax.
  • Training Ablation: Combining Urban and Landmark scenes provides the best overall accuracy, whereas training on a single subset reduces generalization performance.The ablation uses the same median-GPS evaluation protocol on SkyReg-Bench.

6 Conclusion

SkyReg combines feed-forward 3D reconstruction with dense geodetic supervision for pixel-level drone–satellite registration. Across dense and image-level evaluations, it outperforms retrieval and homography baselines on unseen Urban and Suburban scenes.

  • Conclusion: SkyReg estimates cross-view warping with feed-forward 3D reconstruction, enabling dense pixel-wise GPS assignment without relying on 2D point matches.The approach targets perspective drone queries and geodetically accurate satellite references while handling occlusions and large viewpoint changes.
  • Conclusion: SkyReg-Train and SkyReg-Bench provide dense latitude–longitude supervision with depth and camera parameters derived from LiDAR and SfM.SkyReg-Bench evaluates generalization on unseen Urban and Suburban scenes.
  • Conclusion: SkyReg consistently outperforms strong retrieval and homography baselines, substantially improving geodetic error and recall across dense and image-level protocols.The conclusion highlights large gains over RoMa on unseen Urban and Suburban splits.

Pixel-wise Geo-registration of Drone Images

The supplementary material provides additional details about SkyReg’s data gathering, dataset information, and qualitative results.

  • Supplementary Material: The supplementary material is organized into additional data gathering details, dataset information, and qualitative results.Listed subsections include Landmarks and Suburban Scenes within the data-gathering details.
  • Supplementary Material: The listed supplementary sections cover Landmarks and Suburban Scenes as well as broader dataset information.The material also includes a dedicated section for additional qualitative results.

A Additional Data Gathering Details

SkyReg constructs geodetic supervision by combining LiDAR-derived ground truth with orthorectified satellite imagery and synthetic drone queries. Sparse projected correspondences are densified through surface reconstruction and ray tracing.

  • Ground Truth Pixel-wise GPS: Urban ground-truth geodetic data comes from USGS 3DEP LiDAR, transformed from projected coordinates into WGS-84 latitude and longitude.Each tile contains approximately 18 million 3D points.
  • Reference Satellite Images: Orthorectified Bing Maps imagery is retrieved using LiDAR-tile corner coordinates, avoiding the need for ground-truth satellite poses.Imagery is collected at specified zoom levels for Chicago, Seattle, and San Francisco.
  • Query Drone Images: Synthetic drone queries use camera intrinsics and ECEF poses, with projected LiDAR points assigning per-pixel ECEF and geodetic coordinates.LiDAR from corresponding and neighboring tiles is projected into the query camera frame.
  • Ground-Truth Completion: Direct projection produces sparse and potentially occluded labels, so SkyReg constructs a surface mesh and applies ray tracing to densify supervision.The supplied passage identifies sparse coverage and incorrect assignments behind foreground structures as limitations of direct projection.

A.2 Landmarks and Suburban Scenes

Landmark and suburban scenes are reconstructed from fixed-camera image collections using feature matching and triangulation. The resulting 3D geometry is projected into images as metric depth maps, alongside representative urban samples.

  • A.2 Landmarks and Suburban Scenes: Landmark and suburban scenes begin with predefined camera poses and intrinsics that remain fixed during triangulation.Nearby image pairs provide the observations used for reconstruction.
  • A.2 Landmarks and Suburban Scenes: Triangulation estimates 3D scene points by intersecting viewing rays from matched observations across multiple cameras.Because camera geometry is known, this stage reconstructs scene structure rather than estimating camera parameters.
  • A.2 Landmarks and Suburban Scenes: Reconstructed 3D points are projected back into each image to produce per-image scene geometry and metric depth maps.
  • A.2 Landmarks and Suburban Scenes: Representative Urban Chicago and Seattle samples pair drone queries with local reference tiles within large orthorectified satellite images.The figures show reference satellite imagery centrally and associated drone queries on the sides.

B Additional Dataset Information

This appendix provides additional statistics about the dataset introduced earlier in the paper.

  • B Additional Dataset Information: The appendix reports additional statistics describing the distribution of the dataset introduced in Section 4.
  • B Additional Dataset Information: The statistics concern the dataset introduced in Section 4.
  • B Additional Dataset Information: The appendix focuses on dataset distribution information rather than introducing a new dataset.

B.1 SkyReg Urban

The dataset spans urban cities, landmark-centered scenes, and suburban environments, with scene-level image distributions and representative satellite references described across the appendix.

  • SkyReg Urban: SkyReg Urban covers Seattle, San Francisco, and Chicago, totaling 41,027 drone images and 390 satellite images.
  • SkyReg Landmark: SkyReg Landmark contains 83 scenes centered on landmarks around the world.The appendix also reports a histogram of drone-image counts per scene.
  • SkyReg Landmark: Most SkyReg Landmark scenes contain approximately 1.0k–1.1k drone images, indicating dense image coverage for the majority of scenes.The histogram uses the Freedman–Diaconis rule to determine the number of bins.
  • SkyReg Suburban: SkyReg Suburban-Bench contains five scenes across Baltimore, Beverly, and Pittsburgh, with 100–250 drone queries per scene.Each query is paired with a georeferenced satellite image and ground-truth GPS coordinates.
  • SkyReg Suburban: Suburban reference imagery uses one satellite image encompassing each entire scene.

C Additional Qualitative Results

Figures 6 and 7 present qualitative SkyReg-Bench examples comparing query images, ground-truth warps, RoMa, and SkyReg overlays. The examples show accurate geometry-aware alignment under large viewpoint differences and substantial scene depth variation.

  • Additional Qualitative Results: SkyReg produces accurate alignments despite large viewpoint differences and significant scene depth variation.
  • Additional Qualitative Results: Figures 6 and 7 compare query drone images, ground-truth warps, RoMa results, and SkyReg overlays on georeferenced satellite references.Green warped-query overlays make alignment visually inspectable.
Loading 2608.28891v1…