PaperlayerView exact revision on arXiv

Source-linked AI summary

What, Where and How to Transfer in SAR Target Recognition Based on Deep CNNs

Zhongling Huang, Zongxu Pan, Bin Lei

arXiv:1906.01379v1eess.SPcs.CV

TL;DR

SAR target recognition lacks abundant labeled data, while optical and SAR imagery differ substantially, making transfer choices uncertain. The paper analyzes networks, source tasks, and layers, then proposes multi-source transitive transfer with domain adaptation. Results show that transferability depends on network generalization and source-target task distance, with low-level features generally more transferable and the proposed strategies improving recognition in reported experiments.

  • Problem

    Limited labeled SAR data and the discrepancy between optical and SAR images leave the best transferable networks, source tasks, layers, and transfer strategy insufficiently established.

  • Method

    The paper analyzes layer generality across source tasks and proposes multi-source transitive transfer with domain adaptation to reduce source-target discrepancy.

  • Results

    Transferability depends on network generalization and source-target task distance; low-level features are generally more transferable, while reported methods improve OpenSARShip performance.

  • Takeaways & Limitations

    A larger network trained on large-scale data and a source task similar to SAR target recognition are preferred for transfer, with SAR scene pre-training identified as a good source when available.

Abstract

from arXiv · show

Deep convolutional neural networks (DCNNs) have attracted much attention in remote sensing recently. Compared with the large-scale annotated dataset in natural images, the lack of labeled data in remote sensing becomes an obstacle to train a deep network very well, especially in SAR image interpretation. Transfer learning provides an effective way to solve this problem by borrowing the knowledge from the source task to the target task. In optical remote sensing application, a prevalent mechanism is to fine-tune on an existing model pre-trained with a large-scale natural image dataset, such as ImageNet. However, this scheme does not achieve satisfactory performance for SAR application because of the prominent discrepancy between SAR and optical images. In this paper, we attempt to discuss three issues that are seldom studied before in detail: (1) what network and source tasks are better to transfer to SAR targets, (2) in which layer are transferred features more generic to SAR targets and (3) how to transfer effectively to SAR targets recognition. Based on the analysis, a transitive transfer method via multi-source data with domain adaptation is proposed in this paper to decrease the discrepancy between the source data and SAR targets. Several experiments are conducted on OpenSARShip. The results indicate that the universal conclusions about transfer learning in natural images cannot be completely applied to SAR targets, and the analysis of what and where to transfer in SAR target recognition is helpful to decide how to transfer more effectively.

I. INTRODUCTION

The paper examines transfer learning for SAR target recognition, where limited labeled data and differences between SAR and optical imaging make direct transfer uncertain. It asks what to transfer, where transferred features remain generic, and how to transfer more effectively.

  • Limited labeled remote-sensing data makes training deep networks from scratch difficult, motivating transfer learning from large source datasets.
  • Optical-image transfer methods do not directly generalize to SAR because the sensors use different imaging mechanisms.
  • Prior evidence was insufficient to determine whether optical images transfer effectively to SAR images.
  • The paper studies which networks and source tasks transfer best, which layers provide generic features, and how transfer can be improved for SAR target recognition.
  • Feature transferability generally decreases from low-level to high-level layers as representations become more task-specific.

3) How to Transfer:

The paper combines feature-transfer analysis with domain adaptation to improve transfer from heterogeneous source data to SAR targets. Its approach uses task-specific feature alignment and a SAR-specific pre-trained model to support recognition.

  • How to Transfer: Multi-kernel maximum mean discrepancy combines unsupervised and supervised learning to reduce source-target discrepancy in task-specific layers.
  • How to Transfer: The proposed adaptation increases the generality of task-specific features and improves target-data representation and recognition performance.
  • How to Transfer: A SAR-specific model pre-trained on a large-scale SAR land-cover and land-use dataset transfers well to MSTAR and OpenSARShip target-recognition datasets.
  • How to Transfer: STL and ITL are presented as approaches for making transferred features more effective through adaptation and transfer-loss constraints.

A. Generic or Specific

The paper evaluates feature generality by transferring and fixing layers from source-trained networks, then compares source-task sequences and layer choices for SAR target recognition. It proposes multi-source transitive transfer to bridge progressively different tasks.

  • A. Generic or Specific: Features are classified as generic when they represent other datasets or tasks, and specific when they remain closely tied to the source data or task.
  • A. Generic or Specific: The analysis asks whether each layer transfers to the target task and where feature transferability declines sharply.
  • A. Generic or Specific: A source layer is judged generic when transferring and fixing its preceding layers outperforms retraining the target network from scratch.
  • B. Transitive Transfer via Multi-Source: Comparing transferred networks from different source tasks identifies which source produces more general features at a given layer.
  • B. Transitive Transfer via Multi-Source: Multi-source transitive transfer fine-tunes a network sequentially across source tasks before evaluating layer transfer to the target task.

C. Transfer Learning with Domain Adaptation

The section introduces MMD and MK-MMD as distribution-discrepancy measures for domain adaptation, with kernel-based embeddings and an efficient mini-batch estimate.

  • The proposed transitive-transfer method applies domain adaptation to reduce discrepancy between source and target tasks.
  • MMD compares source and target data distributions through their embeddings in a reproducing kernel Hilbert space.The feature map represents distributions as RKHS embeddings, whose distance defines MMD.
  • The source and target distributions are represented by expectations of mapped samples, denoted as their RKHS embeddings.
  • MK-MMD uses a convex combination of multiple basis kernels to measure distribution discrepancy.The paper selects Gaussian kernels as the basis kernels in its method.
  • An unbiased linear-complexity MK-MMD estimate enables more efficient optimization with mini-batch stochastic gradient descent in CNNs.

2) Deep Domain Adaptation Based on Transitive Transfer with Multi-Source:

The method transfers knowledge through source tasks ordered by similarity to the SAR target, then adapts selected layers using either integrated or two-step training.

  • Source tasks are ordered from least to most similar to the target, enabling transitive transfer across multiple sources.Layer-wise transferability is analyzed to identify where generality drops sharply.
  • Because new SAR target categories are unseen in the source data, the classification layer must be retrained rather than directly reused.
  • ITL: ITL jointly back-propagates classification loss and transfer loss through adaptation layers.The transfer loss acts as a regularizer, with λ balancing the two losses and α_l weighting adaptation layers.
  • STL: STL first trains adaptation layers using transfer loss while freezing off-the-shelf layers, then trains the classifier and slightly fine-tunes those layers.Freezing the off-the-shelf layers relies on their general target representation and reduces optimization cost.

A. Datasets and Tasks Description

The experiments compare source networks and tasks—including natural-image classification, SAR reconstruction, and SAR land-cover classification—before evaluating transfer to OpenSARShip.

  • The experiments analyze feature transferability across networks and source tasks, then evaluate the proposed method on OpenSARShip recognition.
  • ImageNet for Classification: ImageNet supplies a large-scale natural-image classification source with 3.2 million labeled images across 5247 categories.A commonly used benchmark subset contains 1.2 million images in 1000 object classes.
  • Domain-adaptation training: ITL combines classification and transfer losses across adaptation layers, whereas STL separates adaptation, classifier training, and slight fine-tuning.
  • TerraSAR-X Images for Classification and Reconstruction: The SAR reconstruction task uses more than 50,000 unlabeled TerraSAR-X image slices to train stacked convolutional autoencoders.
  • TerraSAR-X Images for Classification and Reconstruction: The TerraSAR-X land-cover dataset contains annotated high-resolution SAR scenes spanning urban and non-urban areas worldwide.The dataset includes 288 full scenes and supports SAR land-cover classification.

3) MSTAR for SAR Target Recognition:

MSTAR provides a ten-category military-vehicle SAR recognition source, while OpenSARShip supplies a larger ship-recognition target with substantial class imbalance; the study compares transfer choices across these tasks.

  • MSTAR dataset: MSTAR contains ten categories of military vehicles collected by a Sandia National Laboratory SAR platform at 1-foot X-band resolution.
  • OpenSARShip dataset: OpenSARShip contains 17 ship types from Sentinel-1 imagery, with highly unbalanced class counts.Cargo has 8470 examples, whereas Towing has 4 in the cited example.
  • Source-task comparison: The study evaluates ImageNet classification, SAR reconstruction, SAR land-cover classification, MSTAR recognition, and multi-source transitive transfer as source tasks.

1) What Network:

Network architecture and source-task choice both shape transferability to SAR target recognition. Deeper networks can learn more general features, but source-task distance determines which layers transfer effectively.

  • Network comparison: A ConvNet, H Net, and AlexNet Conv are compared as candidate networks for SAR target recognition.AlexNet Conv uses only AlexNet’s convolutional layers because more than 90% of AlexNet’s parameters are fully connected.
  • Network comparison: As networks become deeper and wider, scratch-training performance on SAR targets decreases, while H Net(M) and AlexNet Conv(M) improve performance through transfer learning.The smaller A ConvNet offers a feature space better suited to limited data, whereas deeper networks learn features that can help related target tasks.
  • Source tasks: Natural-image features generalize well in the first layer but become increasingly specific in higher layers, whereas SAR-trained features transfer more robustly.AlexNet Conv(S) performs better in higher layers because it uses abundant SAR information and a task similar to target classification.
  • Source tasks: MSTAR recognition is more similar to OpenSARShip than unlabeled SAR reconstruction, producing more general H Net(M) features and less transferable H Net(S) features.MSTAR classification can bridge unlabeled SAR reconstruction and OpenSARShip recognition by improving feature generality.
  • Source tasks: Multi-source transitive transfer combines distant and similar datasets to obtain general low-level features and improve transferability in higher layers.The approach combines generic ImageNet low-level features with SAR-derived higher-layer knowledge while reducing source-target discrepancy.

C. Where and How to Transfer Effectively

The paper evaluates where and how to transfer features by comparing source-task scenarios and visualizing source-target feature distributions. It distinguishes transferable off-the-shelf layers from layers requiring adaptation.

  • Where to transfer: Different source-task scenarios produce different layer-wise transferability patterns for OpenSARShip recognition.Fig. 14 compares H Net and AlexNet Conv transferability across layers and source-task settings.
  • Where to transfer: AlexNet Conv(I ∗M) and AlexNet Conv(I ∗S ∗M) transfer strongly through layer 4 but decline sharply at layer 5, while H Net(M) and H Net(S ∗M) generalize better at layer 5.The paper uses t-SNE visualizations of MSTAR and OpenSARShip features in layers 4 and 5 to examine these differences.
  • How to transfer: The method combines transfer loss with classification loss through ITL or STL to reduce source-target feature discrepancy during transfer.The experiments apply both algorithms in AlexNet Conv and H Net scenarios.

1) AlexNet Conv:

For AlexNet Conv, the first four layers are treated as general off-the-shelf features and the fifth layer is adapted using source-target distribution alignment.

  • STL design: STL freezes AlexNet Conv layers 1–4 as off-the-shelf layers and uses layer 5 as the adaptation layer.This choice follows the strong transfer performance observed when freezing the first four layers.
  • STL design: Updating the adaptation layer makes source and target feature distributions more similar in AlexNet Conv(I ∗M) and AlexNet Conv(I ∗S ∗M).The off-the-shelf layers remain fixed while labeled and unlabeled OpenSARShip data are used in the unsupervised adaptation step.
  • Results: 1.44% and 1.96% are STL’s performance gains over common fine-tuning on AlexNet Conv(I ∗M) and AlexNet Conv(I ∗S ∗M), respectively.ITL also improves performance, but its gains are smaller: 0.57% and 0.48% in the same two scenarios.
  • Results: The transfer loss improves classification, but applying it to the first four AlexNet Conv layers is unnecessary because those layers already extract general OpenSARShip features.The results support distinguishing fixed off-the-shelf layers from adaptation layers.

2) H Net:

The H Net experiments show that transfer performance depends on both the adaptation strategy and the layer selected for transfer. ITL and STL can improve OpenSARShip recognition, but their effectiveness varies by scenario and layer.

  • H Net: 1.15% improvement over fine-tuning all layers was achieved by ITL in the reported experiments.The result is attributed to transitive transfer generalizing lower-layer features to OpenSARShip.
  • H Net: 2.02% improvement over fine-tuning all layers was achieved by STL when layers 4 and 5 were adapted in H Net(M).The layer choice reflects declining feature generalization in higher layers.
  • H Net: H Net features in layers 4 and 5 were more distributionally similar to MSTAR than in AlexNet Conv scenarios, yet both tested transfer strategies performed worse.The authors attribute this to AlexNet(I) having stronger generic feature-extraction ability than H Net(S) and H Net(M).
  • H Net: In H Net(S∗M), ITL improved OpenSARShip recognition by 1.44%, while STL improved it by 0.87%.The paper therefore identifies ITL as the better choice for this scenario.
  • H Net: The study concludes that transferability depends on network generalization capacity and source-target task distance, favoring larger networks and source tasks similar to SAR recognition.A deep network pretrained on a large-scale annotated SAR scene dataset is preferred when available; transitive transfer from unlabeled SAR images is another option.
Loading 1906.01379v1…