Source-linked AI summary
SUES-200: A Multi-height Multi-scene Cross-view Image Benchmark Across Drone and Satellite
Runzhe Zhu, Ling Yin, Mingze Yang, Fei Wu, Yuncheng Yang, Wenbo Hu
TL;DR
Cross-view matching needs datasets that represent drone imagery across heights, scenes, and real-world variation. SUES-200 provides such data with a dedicated evaluation pipeline and baseline analysis, and the experiments report increasing matching accuracy with drone height while identifying remaining limitations in samples, viewpoints, winter robustness, and inference speed.
Problem
Existing public cross-view datasets lacked drone imagery at different heights and represented relatively homogeneous scenes, limiting assessment under complex changing conditions.
Method
The paper constructs SUES-200 from real drone and satellite imagery across multiple scenes and heights, then evaluates matching models using metrics for height robustness, uncertainty robustness, and inference speed.
Results
Matching accuracy and precision increase with drone height, and the best overall feature extractor is released as the SUES-200 baseline.
Takeaways & Limitations
SUES-200 supports evaluation of cross-view matching models on height variation, uncertainties, and inference speed rather than conventional matching accuracy alone.
Takeaways & Limitations
SUES-200 still has a small number of samples and limited viewpoints, while wintertime invariant-feature extraction and ViT inference speed remain challenging.
Abstract
from arXiv · showhide
Cross-view image matching aims to match images of the same target scene acquired from different platforms. With the rapid development of drone technology, cross-view matching by neural network models has been a widely accepted choice for drone position or navigation. However, existing public datasets do not include images obtained by drones at different heights, and the types of scenes are relatively homogeneous, which yields issues in assessing a model's capability to adapt to complex and changing scenes. In this end, we present a new cross-view dataset called SUES-200 to address these issues. SUES-200 contains 24120 images acquired by the drone at four different heights and corresponding satellite view images of the same target scene. To the best of our knowledge, SUES-200 is the first public dataset that considers the differences generated in aerial photography captured by drones flying at different heights. In addition, we developed an evaluation for efficient training, testing and evaluation of cross-view matching models, under which we comprehensively analyze the performance of nine architectures. Then, we propose a robust baseline model for use with SUES-200. Experimental results show that SUES-200 can help the model to learn highly discriminative features of the height of the drone.
I. INTRODUCTION
SUES-200 addresses limitations in prior cross-view matching datasets by introducing real drone imagery across four heights and diverse scenes, alongside an evaluation pipeline and baseline model. Experiments show height-aware performance differences and provide a ViT-based benchmark baseline.
- Motivation: Prior cross-view datasets lacked realistic multi-height drone imagery and diverse scenes, limiting evaluation across changing scenarios.University-1652 primarily used synthetic drone views, single-type scenes, and did not distinguish drone heights.
- Dataset: SUES-200 contains real drone imagery from 150m, 200m, 250m, and 300m across parks, schools, lakes, public buildings, and other scenes.The dataset includes 200 target scenes, with 120 for training and 80 for testing.
- Evaluation: The evaluation system measures robustness across heights, robustness to uncertainties, and inference speed alongside Recall@K and AP.The accompanying pipeline is designed to improve training, testing, and evaluation efficiency.
- Experiments: The study trains and evaluates feature extractors, compares multi-angle fusion and transfer learning, and performs ablations before releasing the strongest overall model as the baseline.The baseline is selected from comprehensive evaluation results.
- Results: 59.32, 62.30, 71.35, and 77.17 are the baseline Drone →Satellite Recall@1 accuracies at 150m, 200m, 250m, and 300m, respectively.For Satellite →Drone, the corresponding Recall@1 values are 82.50, 85.00, 88.75, and 96.25.
II. RELATED WORK
Prior cross-view datasets and methods largely paired street or bird’s-eye imagery with satellite views and often relied on hand-crafted or shared backbone features. The paper emphasizes the need to evaluate feature extractors under realistic drone uncertainty and height variation.
- Cross-view Datasets: Existing datasets commonly pair street, bird’s-eye, aerial, or drone views with satellite imagery from the same locations.Examples include panoramic street–satellite datasets and multi-source datasets such as University-1652.
- Cross-view Methods: Traditional hand-crafted descriptors such as SIFT, SURF, and ORB are vulnerable to lighting, occlusion, and large drone–satellite appearance differences.These conditions can produce false or missing matches.
- Cross-view Methods: Most existing approaches use the same backbone network, whereas this work evaluates VGG, ResNet, and DenseNet for feature extraction at different heights.The comparison is conducted through the paper’s matching pipeline.
III. SUES-200 DATASET
SUES-200 is a multi-source, multi-height, multi-scene cross-view dataset pairing satellite images with drone imagery collected at four heights. Its design includes realistic scene overlap and supports retrieval-based evaluation across training and testing splits.
- Dataset characteristics: SUES-200 collects satellite and drone images from 200 locations across parks, schools, lakes, and public buildings.Drone imagery was collected at 150m, 200m, 250m, and 300m.
- Data collection: Each location includes one satellite image and 50 drone images captured along a curved flight path at different heights.The flight path provides multi-angle information about target scenes.
- Dataset challenges: Drone imagery can include overlapping information from nearby scenes because multiple satellite images are selected consecutively within the same area.This creates a challenge for models to focus on the main scene feature rather than overlapping regions.
- Dataset organization: The dataset contains 120 training scenes and 80 testing scenes, with drone images at 1080 × 1080 resolution and satellite images at 512 × 512.The testing setup includes query and gallery datasets for both drone and satellite views.
- Evaluation scope: SUES-200 supports evaluation across four drone heights and includes continuous scenes that reflect surrounding-scene interference in practical environments.At greater heights, drone images become increasingly similar to satellite views.
B. Evaluation Protocol
The SUES-200 evaluation protocol extends conventional retrieval metrics with measures for height robustness, uncertainty robustness, and inference speed. It evaluates degradation across drone heights and simulated disturbances while accounting for combined query-and-gallery inference time.
- Traditional Metrics: Recall@K measures whether a correct match appears among the top-k results, while AP summarizes precision across all true-matched images.Recall@1 is especially sensitive to the rank of the first true match.
- Robustness at Different Heights: The height-robustness protocol uses Recall@1 at 300m as a baseline and measures degradation at lower heights through RDR.The overall RDR from 300m to 150m directly reflects robustness to changing drone height.
- Robustness to Uncertainties: The uncertainty protocol applies flip, block, fog, snow, and rain disturbances to drone queries and measures average precision degradation across four heights.RDP compares disturbed AP with the model’s original AP and averages degradation over the evaluated heights.
- Inference Speed: Inference speed is normalized to a fast base model with benchmark value 1.00 and includes the combined inference time of query and gallery images.This broader measure differs from evaluations that consider only query-image inference time.
IV. METHOD
The SUES-200 pipeline standardizes model training, testing, and evaluation by connecting dataset input, model selection, task execution, and metric output. Users can replace or customize the backbone while retaining the pipeline’s evaluation flow.
- Pipeline Overview: The pipeline takes a cross-view dataset as input and outputs values for each evaluation index.The model is divided into backbone and classification components within the pipeline.
- Training and Testing: During training, images are sent to the selected model, after which the model with the best parameters is selected for testing in Task 1 or Task 2.An evaluation module processes the test results into an evaluation table.
- Model Configuration: Selecting a feature extractor from the Model List replaces the backbone, while users may also customize the network structure.The classification network remains part of the pipeline’s model decomposition.
B. Network Architecture and Loss Function
The method uses a two-branch network to extract and align drone and satellite features for cross-view matching. Cross-entropy trains the branches, while cosine distance compares their test-time feature vectors.
- Network Architecture: A two-branch DNN extracts robust, view-invariant features from drone and satellite images and maps them into a high-dimensional space.The resulting features are used for the subsequent matching task.
- Backbone Networks: ResNet and variants including SE-ResNet, ResNeSt, and CMAB-ResNet are considered as CNN-based feature-extraction backbones.These architectures are presented as developed alternatives for cross-view matching.
- Loss Function: The model uses cross-entropy because training is formulated as a multi-classification task, with lower cross-entropy indicating better classification ability.Each branch produces a loss value that contributes to joint optimization.
- Network Architecture: The backbone and pooling layer produce feature maps, FC1 unifies their dimensions, and a classifier network supports training.The two branches use cross-entropy loss during training.
- Testing and Matching: At test time, the classification network is removed and the backbone outputs drone and satellite feature vectors compared by cosine distance.The system searches the gallery for the most similar feature vectors.
V. EXPERIMENT
The experiments evaluate multiple feature extractors through the SUES-200 pipeline, including multi-query, transfer-learning, and classical-model comparisons. The protocol visualizes retrieval performance and accuracy loss across heights and directions.
- Experimental Design: The experimental study evaluates multiple feature extractors and examines multi-query matching, transfer learning, classical cross-view models, and baseline components.The experiments are organized around the comprehensive evaluation pipeline.
- Training Setup: All backbone feature extractors use ImageNet pre-trained weights, while grid search tunes learning rate, dropout rate, and weight decay.Images are resized to 384 × 384 before training.
- Height Evaluation: The Recall@1 and AP curves compare Drone →Satellite and Satellite →Drone retrieval at 150m, 200m, 250m, and 300m.The figure reports both retrieval directions and both evaluation measures across heights.
- Robustness Evaluation: The robustness plot reports total recall-accuracy loss from 300m to 150m and separates losses for each height interval.Separate panels cover Drone →Satellite and Satellite →Drone matching.
- Uncertainty Testing: Testing augments drone images with unfavorable elements to assess performance under disturbed conditions.The implementation uses imgaug during the testing stage.
B. Evaluation of Different Extractors
The evaluation compares feature extractors on SUES-200 across matching accuracy, robustness to height and uncertainty, and inference speed. ViT performs strongly across accuracy and robustness measures, but has the highest inference cost.
- ViT achieves 59.57%, 62.30%, 71.35%, and 77.15% Recall@1 across four heights for Drone →Satellite, and 82.50%, 85.00%, 88.75%, and 96.25% for Satellite →Drone.
- As drone height increases, captured images are less affected by surroundings and the camera field of view, improving similarity to satellite images and matching performance.
- Robustness at different heights: ViT accuracy decreases by only 23.13% and 14.28% on the two tasks from 300m to 150m, indicating robustness to changing target-scene size.
- Robustness to uncertainties: ViT and SE-ResNet resist uncertainty factors better than other models, while snow is the most difficult condition and prevents any model from reaching the original 50% AP.
- Inference speed: ViT inference takes 2.45 times the baseline in Task1 and 2.48 times in Task2, despite having fewer parameters than VGG.
C. Multiple Queries
Multiple drone-view queries improve matching because they provide more complete scene information, while transfer-learning experiments compare initialization sources and related architectures on SUES-200.
- Multiple queries: Using multiple query images improves Recall@K and AP, with 50-image average features generally increasing Recall@1 by 15% over a single query.
- Transfer learning: University-1652 transfer learning outperforms ImageNet initialization, but its ability to learn features across heights remains limited because heights are not distinguished in that dataset.
- Transfer learning: Training from scratch extracts features less effectively than ImageNet-based training, while University-1652 initialization performs better at the beginning of training than ImageNet initialization.
- Classical models: LPN achieves competitive performance on SUES-200, especially in Task 1, by partitioning features to extract global image information rather than focusing only on the center.
B. Effects of sharing weights
The experiments examine whether shared backbone weights and alternative loss or distance choices improve cross-view matching. Unshared weights perform better overall, although their advantage narrows at greater heights.
- Effects of sharing weights: Recall@1 without shared weights is always higher than with shared weights, but the gap decreases as drone height rises.
- Effects of sharing weights: As height increases, drone and satellite images become more similar, offering a possible explanation for the narrowing shared-versus-unshared performance gap.
- Loss functions: The loss-function experiments show that contrastive loss, triplet loss, and the baseline loss each have advantages and disadvantages across Recall@K and AP.
- Distance measurement: Cosine distance is used in the baseline because of its performance in image retrieval, while Manhattan distance performs worst among the compared measurements.
E. Effects of Distractors in Gallery
The ablations test distractors, loss coupling, and ensemble strategies. Removing distractors improves retrieval, jointly optimized branches help performance, and numerical averaging is the strongest tested fusion strategy.
- Effects of distractors: Removing training images from the gallery improves model performance because the absence of distractors makes the correct match easier to find.
- Effects of adding the losses: Training two branches independently with separate losses causes both tasks’ performance to decline considerably.
- Effects of adding the losses: The authors attribute the benefit of adding losses to mutual branch constraints that jointly optimize an FC layer for features usable by both views.
- Ensemble strategies: Numerical averaging outperforms max pooling and voting for 50-query fusion on both tasks, while voting improves as height rises and predictions become more accurate.
VII. VISUALIZATION
Visualizations show how retrieval and feature attention vary across heights, tasks, and model families on SUES-200. They also connect these results to the benchmark’s broader findings on height effects, feature extraction, robustness, and future model needs.
- Qualitative retrieval: ViT retrieves correct results in some visually similar scenes under Rank 5 across heights and both tasks.Figure 13 presents qualitative drone-view target localization and drone navigation retrieval results.
- Heatmap visualization: CNN-based models focus attention on the main target, while ViT is also visualized through heatmaps on drone and satellite views.Figure 14 compares heatmaps generated by ResNet, Dense, and ViT-based models.
- Height effects: 150m and 200m footage is more affected by surrounding environment and camera pose, making drone and satellite images more different and lowering accuracy.As height increases, redundant information has less influence and matching accuracy gradually increases.
- Feature extractors: Most existing approaches use ResNet, while the ViT-based model shows better robustness across heights and uncertainties.The authors identify feature extraction as a bottleneck and evaluate feature extractors through the complete pipeline.
- Evaluation and future work: SUES-200 evaluates robustness at different heights, robustness to uncertainties, and inference speed, while future work targets redundant low-height information and lightweight Transformers.The benchmark contains images from 200 locations and supports comprehensive evaluation of these model properties.
- Benchmark results: The benchmark reports increasing matching accuracy with height, with baseline Drone →Satellite Recall@1 rising from 59.32 at 150m to 77.17 at 300m.The corresponding Satellite → Drone Recall@1 values are 82.50, 85.00, 88.75, and 96.25 at 150m, 200m, 250m, and 300m.