PaperlayerView exact revision on arXiv

Source-linked AI summary

ATRNet-STAR: A Large Dataset and Benchmark Towards Remote Sensing Object Recognition in the Wild

Yongxiang Liu, Weijie Li, Li Liu, Jie Zhou, Bowen Peng, Yafei Song, Xuying Xiong, Wei Yang, Tianpeng Liu, Zhen Liu, Xiang Li

arXiv:2501.13354v6cs.CV

TL;DR

Robust, accurate, and efficient open-world SAR target recognition remains unresolved, while MSTAR has been a prominent dataset in classification research. ATRNet-STAR introduces a large, fine-grained vehicle dataset and benchmarks 15 methods across 7 settings; the authors identify a trade-off between useful auxiliary information and increased instability and complexity.

  • Problem

    Accurate, robust, and efficient recognition of targets in an open world remains unresolved, and MSTAR is prominent among SAR classification datasets.

    Source: §1 · p. 2, §2.2 · p. 6

  • Method

    The authors introduce ATRNet-STAR, a dataset of 194,324 annotated object images from 40 target types, and derive classification and detection benchmarks.

    Source: §1 · p. 3

  • Results

    15 representative methods are comparatively evaluated across 7 experimental settings on ATRBench classification and detection benchmarks, with official baseline references provided.

    Source: §1 · p. 3

  • Takeaways & Limitations

    ATRNet-STAR and ATRBench are intended to facilitate diverse explorations and promote SAR ATR techniques.

    Source: §5 · p. 14

    Auxiliary complex and angle information may help distinguish targets from clutter under scene variation, but increasing information dimensionality may increase instability and complexity.

    Source: §4.3 · p. 14

Abstract

from arXiv · show

The absence of publicly available, large-scale, high-quality datasets for Synthetic Aperture Radar Automatic Target Recognition (SAR ATR) has significantly hindered the application of rapidly advancing deep learning techniques, which hold huge potential to unlock new capabilities in this field. This is primarily because collecting large volumes of diverse target samples from SAR images is prohibitively expensive, largely due to privacy concerns, the characteristics of microwave radar imagery perception, and the need for specialized expertise in data annotation. Throughout the history of SAR ATR research, there have been only a number of small datasets, mainly including targets like ships, airplanes, buildings, etc. There is only one vehicle dataset MSTAR collected in the 1990s, which has been a valuable source for SAR ATR. To fill this gap, this paper introduces a large-scale, new dataset named ATRNet-STAR with 40 different vehicle categories collected under various realistic imaging conditions and scenes. It marks a substantial advancement in dataset scale and diversity, comprising over 190,000 well-annotated samples, 10 times larger than its predecessor, the famous MSTAR. Building such a large dataset is a challenging task, and the data collection scheme will be detailed. Secondly, we illustrate the value of ATRNet-STAR via extensively evaluating the performance of 15 representative methods with 7 different experimental settings on challenging classification and detection benchmarks derived from the dataset. Finally, based on our extensive experiments, we identify valuable insights for SAR ATR and discuss potential future research directions in this field. We hope that the scale, diversity, and benchmark of ATRNet-STAR can significantly facilitate the advancement of SAR ATR.

1 INTRODUCTION

SAR ATR lacks large-scale standardized datasets, while accurate, robust, and efficient recognition in open-world settings remains unresolved. ATRNet-STAR addresses this gap with a large vehicle dataset and classification and detection benchmarks, whose results show that complex imaging conditions remain challenging.

1 INTRODUCTION

  • ATRNet-STAR focuses on fine-grained single-target recognition and includes horizontal bounding-box labels for detection under different conditions.Source: §1 · p. 2
  • 194,324 object images span 40 target types and 5 scenes, making ATRNet-STAR 10 times the size of any previous vehicle dataset.The dataset includes detailed annotations and varied imaging conditions.Source: §1 · p. 3
  • SAR ATR under complex conditions remains difficult: representative methods successful on previous datasets cannot solve image-angle and scene variations.The authors also report foundation-model effectiveness and transfer-learning results that contribute to feature representation of different ground targets.Source: §1 · p. 3
  • The ATRBench classification and detection benchmark evaluates 15 representative methods across 7 experimental settings for robust ATR, few-shot learning, and transfer learning.The evaluation offers baseline references for developing new approaches.Source: §1 · p. 3
  • The authors built a code library integrating publicly available representative methods to support reproducible, large-scale performance evaluations.Source: §1 · p. 3
  • The paper analyzes experimental results to identify SAR ATR issues and potential future research directions.Source: §1 · p. 3

2 RELATED WORK

The review surveys SAR target datasets and recognition methods, identifying limited diversity, access, and standardization as persistent dataset concerns. It also outlines how dataset composition and experimental settings shape algorithm development and evaluation.

2.1 Datasets for SAR target classification

  • Existing vehicle datasets vary in access, sample counts, target types, scenes, and acquisition conditions, while MSTAR methods have saturated common settings despite background correlation and ideal acquisition conditions.The review also notes that MSTAR does not release all target samples.Source: §2.1 · p. 3
  • Ship datasets face label noise, class imbalance, and limited imaging angles; aircraft datasets cover few types, while satellite detection datasets offer limited resolution for small vehicles in complex scenes.The review describes aircraft as harder to classify because their surfaces are streamlined and their scattering characteristics less obvious.Source: §2.1 · pp. 3–4
  • The review proposes dataset richness, standardization, and usability as key qualities, including diverse targets and conditions, metadata and formats, fair benchmarks, and accessible data.It notes that some datasets require additional SAR imaging and preprocessing.Source: §2.1 · p. 4
  • ATRNet-STAR is presented as a first step toward a new SAR target dataset system, with an open dataset and an official benchmark spanning 7 settings, 15 methods, and 2 data formats.The authors plan to expand beyond vehicles in future work.Source: §2.1 · p. 4

2.2 Review of SAR Target Classification Algorithms

  • The algorithm review groups recognition methods into traditional features, deep learning, feature fusion, and classification with metadata; traditional features include geometric, scattering, time-frequency, local-descriptor, and sparse-representation approaches.SAR-HOG uses ratio-based gradient computation, while scattering-feature extraction can rely on complex images with phase information.Source: §2.2 · pp. 4–5
  • Feature-fusion methods combine traditional and deep features or use scattering-center information to guide deep models, but many datasets provide only magnitude images and omit phase information.Examples include FEC’s combination of deep CNN and attributed scattering-center features and ESF’s use of those features to guide training.Source: §2.2 · p. 5
  • Metadata-based methods use imaging angles, target attributes, or multi-view sequences, motivating datasets with imaging parameters and detailed target characteristics.The review gives examples involving self-supervised rotational-pattern prediction, 3D CAD models, and target size and structure attributes.Source: §2.2 · pp. 5–6
  • Classification accuracy, confusion matrices, and ROC curves are among the stated evaluation metrics, while MSTAR’s popular settings and simulated occlusion may not reflect real acquisition conditions.MSTAR settings include angle, version/configuration, noise, occlusion, resolution, and background-clutter variation; the figure contrasts common simulated occlusion with real occlusion in ATRNet-STAR.Source: Figure 3 · p. 5, §2.2 · p. 6

3 BUILDING ATRNET-STAR: WHY, WHAT, HOW

ATRNet-STAR addresses the lack of large, diverse SAR vehicle datasets by targeting variation in vehicle classes, scenes, and sensors. Its design combines standardized acquisition and annotation with multiple data products and metadata to support SAR ATR research.

3.1 Construct program

  • The construction plan combines fine-grained vehicle categories with five scene types and balanced azimuth sampling across depression angles.Dense target settings such as car parks are outside this version’s single-target focus.Source: §3.1 · p. 6

3.2 Data acquisition

  • UAV radar data are focused into slant-range complex images, geocoded into ground-range images, and annotated with target boxes before target slices are extracted and offset.The products include original complex slant-range data and ground-range images, with target, scene, and sensor parameters in annotation files.Source: §3.2 · p. 7, Figure 4 · p. 7

3.3 Professional system

  • A hierarchical class–subclass–type taxonomy standardizes vehicle categories, while multiple image formats and metadata support varied recognition studies.The metadata include target attributes, scene and sensor parameters; raw echoes are stored but unavailable for download because of their size.Source: §3.3 · p. 8

3.4 Statistical Analysis

  • ATRNet-STAR contains balanced samples across 4 classes, 21 subclasses, and 40 vehicle types, with varied structures and sizes.The breadth of civilian vehicle categories introduces challenges for fine-grained recognition and supports diverse experimental settings.Source: §3.4 · p. 9, Figure 6 · p. 9
  • The dataset spans complex and simple scenes, where occlusion and layover can obscure targets or increase non-target scattering.Random target offsets also make recognition of non-centered objects more challenging.Source: §3.4 · pp. 9–10, Figure 8 · p. 10
  • Azimuth sampling is balanced across depression angles, but stripmap constraints and operating costs limit its density; band and polarization also affect target scattering.ATRNet-STAR provides two bands and quad polarization for studying these sensor effects.Source: §3.4 · p. 10, Figure 9 · p. 11, Figure 10 · p. 11

3.5 Dataset value

  • The dataset supports research on robustness across acquisition domains, few-shot learning, and incremental recognition of changing domains or classes.It also offers varied target samples for transfer learning and study of self-supervised learning and foundation models.Source: §3.5 · pp. 10–11
  • Its multiple data formats and acquisition metadata support work on complex phase, polarization, controllable target generation, and target-parameter estimation.The proposed uses extend beyond recognition to generation under varied imaging conditions.Source: §3.5 · p. 11

4 BENCHMARKS

ATRBench evaluates classification and detection across seven settings, including shifts in scenes, imaging conditions, and target state. The results show substantial recognition challenges, with transfer benefits for similar ground-target datasets but remaining robustness gaps.

4 BENCHMARKS

  • ATRBench covers seven experimental settings and 15 methods across classification and detection on magnitude images, plus classification on complex images.Its settings include similar-condition SOC tests and EOC tests with shifts in scene, depression angle, azimuth, band, or polarization.Source: §4 · p. 11, §4.1 · p. 11, Table 4 · p. 12

4.2 Results and analyses

  • Complex-image MS-CVNets and LDSF better distinguish targets from clutter in EOC-Scene; ConvNeXt is robust to depression shifts, while ViT is insensitive to azimuth changes.SARATR-X, a SAR-pretrained self-supervised HiViT model, significantly improves on HiViT; full updating of some low-layer Transformer weights tends to overfit.Source: §4.2 · p. 12, Table 5 · p. 13
  • Under SOC-50, test accuracy decreases linearly as training samples are reduced on a logarithmic scale; platform-resolution effects remain unexamined because of cost.The paper also reports that imaging-angle changes affect target characteristics more than band and polarization changes.Source: §4.2 · pp. 12–13, Figure 11 · p. 13
  • Two-stage detectors are more robust under most EOC conditions, and DiffDet4SAR outperforms other algorithms in EOC-Scene and EOC-Azimuth.CenterNet shows superior detection under EOC-Depression and EOC-Polarization; YOLO methods exhibit limitations under EOC conditions.Source: §4.2 · p. 13, Table 6 · p. 13
  • Detection can distinguish targets from background clutter in complex scenes, but its performance is slightly lower than classification, partly due to noisy labels and box offsets.Fine-grained detection also combines precise localization and classification, making it more complex than classification alone.Source: §4.2 · pp. 13–14
  • ATRNet-STAR pretraining benefits vehicle and airplane recognition in five-shot transfer experiments, while transfer to sea-scene ship classification is not obvious.The transfer evaluation initializes models with SOC-40 supervised pretraining weights and fine-tunes them on other target datasets.Source: §4.2 · p. 14, Figure 12 · p. 14
  • The authors conclude that the benchmarks provide a challenging basis for studying SAR target recognition across imaging conditions and fine-grained classes.They report that scene and imaging-angle robustness still has substantial room for improvement, and target-position changes are infrequent because of acquisition time and cost.Source: §4.2 · p. 14

4.3 Discussion

  • No single method effectively handles diverse perturbations, leaving robust feature extraction, learning from few labels, and use of complex and polarization information as research problems.The discussion also identifies interpretable, trustworthy recognition and incremental adaptation to changing conditions and target types as directions for study.Source: §4.3 · p. 14

5 CONCLUSION

The paper establishes ATRNet-STAR, a large-scale fine-grained SAR vehicle dataset, and ATRBench, a benchmark spanning seven settings and 15 methods. The authors conclude these resources will facilitate diverse explorations and promote SAR ATR techniques.

5 CONCLUSION

  • ATRNet-STAR contains varied vehicle types, scenes, and imaging conditions and is 10 times the size of previous datasets of the same type.ATRBench comprises seven experimental settings and 15 methods.Source: §5 · p. 14
Loading 2501.13354v6…