PaperlayerView exact revision on arXiv

Source-linked AI summary

Vision-Based UAV Self-Positioning in Low-Altitude Urban Environments

Ming Dai, Enhui Zheng, Zhenhua Feng, Jiedong Zhuang, Wankou Yang

arXiv:2201.09201v2cs.CVcs.AI

TL;DR

UAVs can have difficulty maintaining satellite-based positioning when signals are disrupted, while prior datasets often focus on localizing captured objects rather than the UAV itself. This paper introduces DenseUAV for low-altitude urban self-positioning and develops a vision-based pipeline with SDM@K; the reported flight-height analysis finds R@1 and SDM@1 increase as flight height increases within the collected range.

  • Problem

    Stable satellite positioning can be difficult under signal obstruction or interference, and prior datasets had not addressed positioning the UAV itself.

    Source: §I · p. 1

  • Method

    The paper introduces DenseUAV with densely sampled real-scene UAV imagery and proposes a vision-based self-positioning pipeline with SDM@K combining retrieval and localization evaluation.

    Source: §I · p. 2, 1) Dense Sampling · p. 3, B. Spatial Continuity Index (SDM@K) · p. 6

  • Results

    R@1 and SDM@1 show an increasing trend as drone flight height increases in the collected data.

    Source: 1) Impact of Flight Altitude · p. 11

  • Takeaways & Limitations

    DenseUAV and its baseline provide a low-altitude urban dataset and process framework for studying vision-based UAV self-positioning.

    Source: §VII · p. 12

    DenseUAV covers only low-altitude urban scenes, leaving other UAV self-positioning scenarios unstudied.

    Source: §VII · p. 12

Abstract

from arXiv · show

Unmanned Aerial Vehicles (UAVs) rely on satellite systems for stable positioning. However, due to limited satellite coverage or communication disruptions, UAVs may lose signals from satellite-based positioning systems. In such situations, vision-based techniques can serve as an alternative, ensuring the self-positioning capability of UAVs. However, most of the existing datasets are developed for the geo-localization tasks of the objects identified by UAVs, rather than the self-positioning task of UAVs. Furthermore, the current UAV datasets use discrete sampling on synthetic data, such as Google Maps, thereby neglecting the crucial aspects of dense sampling and the uncertainties commonly experienced in real-world scenarios. To address these issues, this paper presents a new dataset, DenseUAV, which is the first publicly available dataset designed for the UAV self-positioning task. DenseUAV adopts dense sampling on UAV images obtained in low-altitude urban settings. In total, over 27K UAV-view and satellite-view images of 14 university campuses are collected and annotated, establishing a new benchmark. In terms of model development, we first verify the superiority of Transformers over CNNs in this task. Then, we incorporate metric learning into representation learning to enhance the discriminative capacity of the model and to lessen the modality discrepancy. Besides, to facilitate joint learning from both perspectives, we propose a mutually supervised learning approach. Last, we enhance the Recall@K metric and introduce a new measurement, SDM@K, to evaluate the performance of a trained model from both the retrieval and localization perspectives simultaneously. As a result, the proposed baseline method achieves a remarkable Recall@1 score of 83.05% and an SDM@1 score of 86.24% on DenseUAV. The dataset and code will be made publicly available on https://github.com/Dmmm1997/DenseUAV.

I. INTRODUCTION

The paper frames UAV self-positioning as a vision-based task for GPS-denied environments and introduces DenseUAV, a dataset built for that task. It also presents a Transformer-based pipeline and SDM@K, which evaluates retrieval and localization together.

I. INTRODUCTION

  • The proposed task localizes the UAV itself in GPS-denied environments, using a vertically oriented camera rather than panoramic UAV-view images.The motivation is that disrupted satellite signals can make stable positioning difficult.Source: §I · p. 1
  • The baseline uses a vision Transformer and Siamese network, with metric learning to address modality discrepancy and mutual learning to align the two perspectives during training.The paper also investigates augmentation, backbones, prediction heads, and loss functions.Source: §I · p. 2
  • The pipeline extracts satellite and UAV-view features, matches them by feature similarity, and applies a positioning strategy to obtain an optimal match.Satellite images are sliced from 20-level Google Maps imagery to build the database; post-processing supports continuous positioning.Source: §I · p. 2
  • SDM@K evaluates retrieval and localization together because binary Recall@K treats even small spatial errors as wholly incorrect.The introduction describes this as a better fit for spatially continuous UAV positioning than retrieval-only indicators.Source: §I · p. 2
  • DenseUAV uses densely sampled real-world UAV imagery, with multi-scale satellite images and multiple temporal satellite images included in the dataset design.The supplied passages also describe sampling at three flight heights, but that passage is not part of this section's evidence set.Source: §I · pp. 1–2

II. RELATED WORK

Prior geo-localization work spans ground-to-aerial matching and drone-to-satellite matching. DenseUAV differs from earlier drone-view datasets by targeting the UAV's own position rather than prominent buildings or other captured objects.

1) Ground-to-Aerial Matching:

  • Ground-to-aerial work developed from publicly sourced image pairs to panoramic ground-to-satellite datasets and later allowed query images to come from arbitrary locations within an area of interest.CVUSA paired ground panoramas with satellite images; CVACT added orientation maps, and VIGOR changed the center-to-center matching assumption.Source: 1) Ground-to-Aerial Matching · p. 3

2) Drone-to-Satellite Matching:

  • University-1652 introduced drone-view cross-view geo-localization and UAV subtasks, while SUE-200 added drone imagery from four flight altitudes.University-1652 used drone-view as a transition view for ground-to-satellite matching.Source: 2) Drone-to-Satellite Matching · p. 3
  • DenseUAV shifts the target from identifying discretely distributed prominent buildings to positioning the UAV itself in GPS-denied environments.That task requires awareness of spatial distribution in images as well as prominent visual features.Source: 2) Drone-to-Satellite Matching · p. 3

B. Deep-Learning-Based Geo-Localization

Deep-learning geo-localization methods use supervision to distinguish matching features or matching-based strategies to address differences between views. DenseUAV's data design emphasizes dense sampling and authentic scenes for UAV self-positioning.

1) Supervision-Based Methods:

  • Supervision-based geo-localization methods use classification, triplet, contrastive, and multi-task losses to improve feature discrimination and matching reliability.Triplet-loss variants aim to bring positive pairs closer and separate negative pairs.Source: 1) Supervision-Based Methods · p. 3

2) Matching-Based Methods:

  • Matching-based methods address cross-domain spatial asymmetry through view transformation, image generation, viewpoint alignment, segmentation, or feature chunking.The passage notes distortion and information loss as drawbacks of polar-coordinate transformation.Source: 2) Matching-Based Methods · p. 3

A. Characteristics

  • DenseUAV is designed for low-altitude urban UAV self-positioning and includes both UAV-view and satellite-view perspectives.The dataset section frames it as a task not addressed by previous datasets.Source: A. Characteristics · p. 3, §III · p. 3

1) Dense Sampling:

  • Dense sampling means adjacent frames overlap, requiring models to capture fine-grained features and spatial-relative information.The resulting neighboring-frame interference makes testing more challenging.Source: 1) Dense Sampling · pp. 3–4

2) Real Scene:

  • Unlike datasets built from synthetic map imagery, DenseUAV captures UAV-view images in authentic real-world scenes.The passage identifies time gaps between UAV and satellite images as a source of same-location discrepancy.Source: 2) Real Scene · p. 4

3) Others:

DenseUAV combines real UAV sampling across altitudes and conditions with multi-scale, multi-year satellite imagery. Its fourteen-university dataset composition and evaluation metrics are described.

3) Others:

  • UAV images span three altitude levels and varying spatial coverage, while satellite imagery uses three scales and multiple temporal instances.The dataset uses these variations to represent changes in spatial coverage and temporal disparities.Source: 3) Others · p. 4

1) Sampling Scheme for UAV-View Images:

  • UAV-view images are sampled at 80, 90, and 100 meters, under varied weather and daylight, with a downward-facing camera and 20-meter spacing.Waypoint positioning controls coordinate error within 1 meter; adjacent sampling points provide overlapping coverage.Source: 1) Sampling Scheme for UAV-View Images · p. 4, Figure 2 · p. 5

2) Acquisition of Satellite-View Images:

  • Satellite-view images use Google Maps level-20 imagery from 2020 and 2022, with three image scales.The temporal instances and scales are included to address spatial change over time and variation in satellite-image scale.Source: 2) Acquisition of Satellite-View Images · p. 4

3) Dataset Composition:

  • The gallery covers all fourteen universities and contains 27,297 images from 3,033 sampling points.Training uses 6,768 UAV-view and 13,536 satellite-view images; the test query set contains 2,331 UAV-view images and 4,662 satellite-view images.Source: 3) Dataset Composition · p. 4, Table II · p. 4

IV. EVALUATION METRICS

  • The evaluation section introduces Recall@K and then presents SDM@K to assess retrieval and positioning performance together.SDM@K is introduced as a metric considering both retrieval and positioning performance.Source: §IV · p. 4

A. Spatial Discrete Index (Recall@K)

Recall@K counts correct category matches, while SDM@K incorporates spatial distance to evaluate retrieval and localization together. The distinction reflects that small positioning deviations and large errors should not be treated alike.

A. Spatial Discrete Index (Recall@K)

  • Recall@1 averages binary correct-match indicators, counting a query as correct when its nearest gallery image has the same category.The indicator is 1 for a category match and 0 otherwise.Source: A. Spatial Discrete Index (Recall@K) · p. 5

B. Spatial Continuity Index (SDM@K)

  • SDM combines retrieval with localization by scoring the spatial distances of the top-K retrieved images and normalizing the result.The top-K images are weighted by rank, and the spatial distance uses longitude and latitude; the paper sets amplification factor s to 5×10^3.Source: B. Spatial Continuity Index (SDM@K) · p. 5
  • Unlike Recall@1's binary decision, SDM is continuous and distinguishes small positioning errors from large deviations that should count as wrong positioning.The metric is more sensitive to small errors and gives scores close to zero for very large errors.Source: B. Spatial Continuity Index (SDM@K) · pp. 5–6
  • SDM@K uses the cosine-similarity rank as the weight and includes the top K recalled images in its calculation.The metric lies between 0 and 1, with larger values indicating better localization performance.Source: B. Spatial Continuity Index (SDM@K) · pp. 5–6

V. THE PROPOSED BASELINE MODEL

The baseline uses a shared-weight two-view network with augmentation, feature extraction and integration, and supervised training losses. Its augmentations target flight direction and satellite-view orientation.

V. THE PROPOSED BASELINE MODEL

  • The shared-weight UAV and satellite branches produce features for cosine-similarity ranking at inference and use class-, feature-, and distribution-level supervision during training.Training includes representation learning, metric learning, and mutual learning.Source: §V · p. 6

A. Data Augmentation

  • Four augmentation methods expand UAV and satellite training data because real-scene drone data collection is costly.The methods address variation in direction, illumination, and spatial appearance.Source: A. Data Augmentation · p. 6

1) UAV Flight Direction:

  • Random rotate simulates UAV flight directions by cropping a square from the original image’s largest inscribed circle after selecting a rotation angle.The approach also removes edge information so the model can prioritize central image information.Source: 1) UAV Flight Direction · p. 6, Figure 5 · p. 7

2) Satellite View Direction:

  • Random affine augmentation addresses uncertain satellite-view orientation and introduces empty edge regions that encourage learning spatial context.The authors distinguish it from random rotate, which is used for UAV flight-direction variation.Source: 2) Satellite View Direction · p. 7

3) Light Intensity:

The baseline addresses illumination and spatial variation through augmentation, uses ViT-S as its backbone, and compares pooling- and chunking-based prediction heads alongside three levels of supervision.

3) Light Intensity:

  • Random brightness augmentation targets illumination variation across UAV images captured at different times and in different weather.The augmentation is intended to improve robustness to illumination changes.Source: 3) Light Intensity · p. 7

4) Spatial Differences:

  • Random erasing simulates spatial changes associated with temporal offsets and encourages attention to more general, robust features.The paper identifies temporal spatial differences as a major self-positioning challenge.Source: 4) Spatial Differences · p. 7

B. Backbone Network

  • The authors report that Transformer backbones outperform CNN-based backbones in their experiments and select ViT-S to balance performance and inference speed.They attribute the observed difference primarily to domain, perspective, and temporal spatial-information discrepancies that require global context.Source: B. Backbone Network · p. 7

1) Pooling-based:

  • Pooling-based heads compress ViT local tokens into one feature vector, with MaxPool, AvgPool, AvgMaxPool, and GemPool evaluated.The local tokens have dimensions (B, N, C), with pooling applied across the patch dimension.Source: 1) Pooling-based · pp. 7–8

2) Chunking-based:

  • Chunking-based heads use spatial partitioning for fine-grained features, adapting LPN to ViT-S tokens and considering FSRA’s adaptive partitioning.The local tokens are reconstructed into a height-by-width format for the LPN head.Source: 2) Chunking-based · p. 8

D. Loss Functions

  • The baseline combines representation learning, metric learning, and mutual learning, calculating losses at class, feature, and distribution levels.Source: D. Loss Functions · p. 8

1) Representation Learning:

The baseline combines classification-based representation learning with metric learning and bilateral learning between UAV and satellite views. The methods address feature discrimination and cross-view consistency through distinct training objectives.

1) Representation Learning:

  • The baseline uses Cross-Entropy loss for representation learning.Classification loss maps data into a feature space under supervision.Source: 1) Representation Learning · p. 8

2) Metric Learning:

  • Soft-weighted triplet loss is selected as the baseline’s feature-measurement loss after experiments with four triplet-loss variants.Metric learning supervises feature-level distances used to measure image similarity during inference.Source: 2) Metric Learning · p. 8

3) Mutual Learning:

  • The bilateral mutual-learning method encourages UAV and satellite category-vector distributions for the same category to be consistent.The two views’ output distributions are represented separately as Od and Os.Source: 3) Mutual Learning · p. 8

A. Implementation Details

The experiments assess dense versus non-dense training data and several augmentations, including transformations for flight direction, satellite-view geometry, lighting, and spatial differences.

A. Implementation Details

  • The implementation uses 224×224 inputs, 512-dimensional global representation vectors, and 120 training epochs.Training uses SGD with an initial learning rate of 0.003 and batch size 8; backbone weights use 0.3× the other weights’ learning rate.Source: A. Implementation Details · p. 8

B. Non-Dense Features v.s. Dense Features

  • Dense-data training significantly improves UAV self-positioning, with overlapping adjacent frames encouraging more robust spatial-distribution features.The comparison uses DenseUAV for dense data and Place and University-1652 for non-intensive geographic-task data.Source: B. Non-Dense Features v.s. Dense… · p. 8, Table III · p. 9

C. Evaluation on Data Augmentation

1) UAV Flight Direction:

  • Random rotation of UAV views increases R@1 by nearly 9% and SDM@1 by 6%.The authors present it as an augmentation of UAV flight-direction data.Source: 1) UAV Flight Direction · p. 9

2) Satellite View Direction:

  • Satellite-view random affine augmentation raises R@1 by 1.3% and SDM@1 by 1.2%, while slightly degrading UAV-view performance.The authors attribute the UAV-view degradation to the transform’s poor fit for capturing real drone rotation.Source: 2) Satellite View Direction · p. 9

3) Light Intensity:

  • Random brightness has minimal impact, which the authors attribute to the dataset’s random time-based collection already covering brightness variation.They conclude that lighting is not the task’s main challenge or performance factor.Source: 3) Light Intensity · p. 9

4) Spatial Difference:

  • Random erasing generally improves positioning accuracy, with satellite-view R@1 and SDM@1 each increasing by about 2.5%.The authors connect this gain to training with spatial information loss, which may help address changes associated with temporal offsets.Source: 4) Spatial Difference · p. 9

D. Evaluation on Backbone Network

The backbone experiments favor Transformers over CNNs and identify ViT-S as a balance of accuracy and inference speed. Head comparisons report gains from adaptive pooling over the baseline.

D. Evaluation on Backbone Network

1) Pooling-based Heads:

  • Global, the paper’s baseline, improves over AvgPool and MaxPool; GemPool adaptively learns weights for local tokens and improves R@1 by 2.3% and SDM@1 by 0.6% over Global.Source: 1) Pooling-based Heads · p. 10

2) Chunking-based Heads:

The experiments report gains from chunking-based heads and from adding metric and mutual learning to the baseline. They also compare representation losses and identify data-source factors for evaluation.

2) Chunking-based Heads:

  • LPN improves R@1 by 2.9% and SDM@1 by 1.8% over the baseline, while LPN and FSRA both improve final performance and slightly increase inference time.The chunking-based approach aligns images from different perspectives by blocks, with two blocks used for alignment.Source: 2) Chunking-based Heads · p. 10

F. Evaluation on Loss Function

  • The baseline combines representation learning, metric learning, and mutual learning as its three supervised methods.The loss-function experiments examine these methods as components of the baseline.Source: F. Evaluation on Loss Function · p. 10

1) Representation Learning:

  • Cross-Entropy loss is more suitable for UAV self-positioning than Focal Loss in the reported comparison.The two evaluated classification losses are CE and Focal Loss.Source: 1) Representation Learning · p. 10

2) Metric Learning:

  • Adding triplet loss to representation learning improves R@1 by 2.9% and SDM@1 by 2.4%; soft-weighted triplet loss adds 2% and 1.6% over standard triplet loss, respectively.The experiments compare Triplet, Hard-Mining Triplet, Same-Domain Triplet, and Soft-Weighted Triplet losses.Source: 2) Metric Learning · pp. 10–11

3) Mutual Learning:

  • KLLoss adds a further 1.8% in R@1 and 1.2% in SDM@1.It is introduced with CE and soft-weighted triplet loss in a bidirectional mutual-learning approach where the branches exchange knowledge and representations.Source: 3) Mutual Learning · p. 11

G. The Impact of Data Source

  • The data-source analysis considers flight height, image scale, and the satellite image’s time node.The passage identifies these as factors affecting UAV self-positioning performance.Source: G. The Impact of Data Source · p. 11

1) Impact of Flight Altitude:

Across the data-source experiments, increasing flight altitude and using multi-scale or multi-time satellite imagery improve reported positioning metrics. The paper also reports persistent matching challenges and limits DenseUAV’s coverage to low-altitude urban scenes.

1) Impact of Flight Altitude:

  • R@1 and SDM@1 increase with flight altitude, which the authors attribute to a wider field of view and richer spatial information.The study tested 80 m, 90 m, and 100 m; higher altitudes remain unexplored because of limited flight altitude.Source: 1) Impact of Flight Altitude · p. 11

2) Impact of the Scale:

  • Using three satellite scales improves R@1 by 0.9% and SDM@1 by 1.0% over the middle scale alone.The middle scale performs best among single-scale settings; small and large scales are each nearly 3 points lower in R@1.Source: 2) Impact of the Scale · p. 11, Table VIII · p. 12

3) Impact of the Time:

  • Multi-time satellite images improve R@1 by nearly 9% and SDM@1 by approximately 7 points over a single time node.The satellite images were collected in 2020 and 2022, while UAV images were acquired in 2021.Source: 3) Impact of the Time · p. 11, Table VIII · p. 12

H. Visualization

  • Some test queries still fail to match corresponding satellite images amid UAV–satellite domain differences and variation in brightness and shooting height.The paper notes that perspective transformation and GAN methods may also help address domain differences.Source: H. Visualization · p. 12, Figure 8 · p. 12

VII. CONCLUSION

  • The conclusion reports that Transformer models significantly outperform CNNs for this task and describes the proposed baseline as robust.The authors attribute the comparison to UAV–satellite modality differences and the characteristics of UAV self-positioning.Source: §VII · p. 12
  • DenseUAV covers only low-altitude urban scenes; other scenarios have not been studied.The authors identify scene coverage and altitude range as dataset limitations.Source: §VII · p. 12
Loading 2201.09201v2…