Source-linked AI summary
Predicting Gradient is Better: Exploring Self-Supervised Learning for SAR ATR with a Joint-Embedding Predictive Architecture
Weijie Li, Yang Wei, Tianpeng Liu, Yuenan Hou, Yuxuan Li, Zhen Liu, Yongxiang Liu, Li Liu
TL;DR
SAR target recognition lacks a generic representation across tasks, while limited pretraining data and SAR-specific small-target and speckle challenges complicate SSL. SAR-JEPA predicts multi-scale SAR gradient features for masked local patches, and the authors report improved performance over other SSL methods with performance increasing as SAR data grows. They conclude that SSL shows potential for SAR recognition across targets, scenes, and sensors, while noting that pretraining data must scale extensively to exploit that potential.
Problem
The paper asks whether a generic representation can support varied SAR target-recognition tasks, where costly data collection and annotation limit dataset diversity and large-scale pretraining remains underexplored.
Source: §1 · p. 1Method
SAR-JEPA uses local masked patches to predict multi-scale SAR gradient features, combining masked-image modeling with gradient-ratio target features.
Source: §1 · pp. 2–3Results
SAR-JEPA achieves improved performance over other SSL methods, with performance increasing as pretraining data volume grows.
Source: §1 · p. 3Takeaways & Limitations
The experiments highlight SSL’s potential for SAR target recognition across diverse targets, scenes, and sensors.
Source: §5 · p. 13The pretraining dataset needs extensive scaling, and approaching ImageNet’s million-image scale is challenging.
Source: §5 · p. 13
Abstract
from arXiv · showhide
The growing Synthetic Aperture Radar (SAR) data has the potential to build a foundation model through Self-Supervised Learning (SSL) methods, which can achieve various SAR Automatic Target Recognition (ATR) tasks with pre-training in large-scale unlabeled data and fine-tuning in small labeled samples. SSL aims to construct supervision signals directly from the data, which minimizes the need for expensive expert annotation and maximizes the use of the expanding data pool for a foundational model. This study investigates an effective SSL method for SAR ATR, which can pave the way for a foundation model in SAR ATR. The primary obstacles faced in SSL for SAR ATR are the small targets in remote sensing and speckle noise in SAR images, corresponding to the SSL approach and signals. To overcome these challenges, we present a novel Joint-Embedding Predictive Architecture for SAR ATR (SAR-JEPA), which leverages local masked patches to predict the multi-scale SAR gradient representations of unseen context. The key aspect of SAR-JEPA is integrating SAR domain features to ensure high-quality self-supervised signals as target features. Besides, we employ local masks and multi-scale features to accommodate the various small targets in remote sensing. By fine-tuning and evaluating our framework on three target recognition datasets (vehicle, ship, and aircraft) with four other datasets as pre-training, we demonstrate its outperformance over other SSL methods and its effectiveness with increasing SAR data. This study showcases the potential of SSL for SAR target recognition across diverse targets, scenes, and sensors.Our codes and weights are available in \url{https://github.com/waterdisappear/SAR-JEPA.
1. Introduction
The paper investigates whether self-supervised learning can support generalized SAR target recognition despite limited data, small targets, and speckle noise. It proposes SAR-JEPA and reports improved performance over other SSL methods, with performance increasing as pretraining data grows.
1. Introduction
- The paper asks whether a generic representation can serve varied SAR target-recognition tasks, while costly data collection and annotation limit dataset diversity.Existing work is divided among vehicle, ship, and aircraft recognition.Source: §1 · p. 1
- SAR SSL faces small targets in large remote-sensing scenes and speckle noise that can degrade learned representations.The introduction notes that augmentation may not simulate multiplicative speckle noise accurately, while HOG gradient computation can produce false results under it.Source: §1 · p. 2
- SAR-JEPA combines local masked-image modeling with gradient-by-ratio target features, using local masks and multi-scale features for varied small-target scales.The study also constructs a pretraining dataset of approximately 100,000 SAR magnitude images from four open-source datasets.Source: §1 · pp. 2–3
- SAR-JEPA improves performance over other SSL methods, and its performance increases with pretraining-data volume.The authors present this scaling as potential for a foundation model across SAR ATR targets, scenes, and sensors.Source: §1 · p. 3
2. Related Work
The related work reviews masked-image modeling choices in computer vision and the distinct constraints of SAR and remote sensing. It motivates local masking and SAR-domain target features as responses to small targets, limited pretraining data, and speckle noise.
2.1. Self-supervised learning in computer vision
- Computer-vision masked-image modeling varies masking strategies and prediction targets; local masking is proposed to learn context around small remote-sensing targets.Prior work also uses pixel, token, HOG, deep, and frequency targets; I-JEPA’s learnable target encoder is reported to collapse under SAR image noise, motivating gradient-ratio features.Source: §2.1 · pp. 3–4
2.2. Self-supervised learning in remote sensing
- SAR target-recognition SSL remains sparsely studied, with no unified benchmark setting and pretraining datasets that cover limited targets and scenarios.The potential of foundation models for SAR target recognition had not yet been investigated.Source: §2.2 · p. 4
- Remote-sensing images pair large scenes with small targets, while SAR pixel-level masking is unsuitable because individual pixels contain multiplicative noise.The authors therefore prefer local patches over whole-image or pixel-level masking.Source: §2.2 · p. 4
- SAR speckle noise limits SSL features; the authors favor gradient-ratio features for target shapes over differential-gradient HOG under multiplicative noise.Prior remedies discussed include augmentation and despeckling, while the authors distinguish target-level shape extraction from scene-level features.Source: §2.2 · p. 4
3. Approach
SAR-JEPA predicts SAR-domain features for unseen regions from locally masked patches. Its design combines local context with gradient-ratio features at multiple scales, then fine-tunes the encoder for downstream recognition.
3. Approach
- SAR-JEPA uses masked local patches to predict SAR-domain feature representations of unseen regions, learning contextual relationships between target parts and their surroundings.The pretraining framework uses a MIM encoder and predictor alongside a target encoder that extracts SAR features.Source: §3 · p. 4, Figure 2 · p. 5
3.1. Local Masked Patches
- Local patches focus masking on the small areas where remote-sensing target context usually remains.The method follows LoMaR and adds mask tokens and relative positional encoding to the encoder input and MAE structure.Source: §3.1 · p. 5
3.2. Target Encoder
- The target encoder uses ratio-based gradients because differential gradients are not constant-false-alarm-rate under multiplicative speckle noise.Opposite-side area averages are used to compute directional ratios, and logarithms yield horizontal and vertical gradients; kernel sizes vary to construct multi-scale features.Source: §3.2 · p. 6
3.3. Implementation
- The implementation trains masked-patch predictions against multi-scale SAR features with mean squared error, while downstream tasks use the DNN encoder and a task layer.The target encoder processes the full SAR image; the encoder and predictor use learnable relative positional encoding, and the decoder uses an eight-layer ViT rather than a linear layer to reduce noise disturbance.Source: §3.3 · p. 6
4. Experiments
The experiments evaluate self-supervised pretraining on diverse SAR data through few-shot recognition tasks, then examine the architecture’s components, comparisons, attention patterns, and scaling. Results highlight gains and trade-offs associated with target features, model design, and available data.
4.1. Dataset and Experimental Settings
- The four pretraining datasets span targets, scenes, and sensors; pretrained models are evaluated with few-shot fine-tuning and linear probing on three target-recognition datasets.The evaluation covers vehicle, ship, and aircraft recognition, with shared downstream training settings.Source: §4.1 · pp. 6–8
4.2. Ablation Study
- LoMaR-SAR, which replaces LoMaR’s linear decoder with MAE’s eight-layer decoder, gives the best result in the MIM-framework comparison.Local masks are favored for small remote-sensing targets, while the linear decoder can expose encoder features to SAR noise through the loss.Source: §4.2 · p. 8, Table 3 · p. 9
- SAR-HOG’s ratio-gradient calculation improves over HOG, but HOG sampling can discard small-target scattering points; the method therefore emphasizes gradient edges.The comparison varies target features while holding other settings fixed; the reported issue with HOG is noise points in small target regions.Source: §4.2 · pp. 8–9, Table 4 · p. 9
4.3. Comparisons with other methods
- SAR-JEPA’s comparisons show that SAR-pretrained MAE exceeds ImageNet with fine-tuning, while HOG-based FG-MAE degrades and I-JEPA exhibits feature collapse under speckle noise.LoMaR’s local reconstruction improves fine-tuning on vehicles and ships versus MAE, but its linear decoder is more vulnerable to speckle and unstable.Source: Table 6 · p. 11, §4.3 · pp. 11–12
4.2. Ablation Study
- MIM and PGCA are reported as equally important and mutually reinforcing: MIM supplies contextual prediction, while PGCA adds domain knowledge to target features.Together, they learn contextual features for noisy SAR data.Source: §4.2 · pp. 10–11, Table 5 · p. 10
4.3. Comparisons with other methods
- The proposed method performs better on aircraft, though its results remain below those for vehicles and ships; SAR-ACD 10-shot fine-tuning also trails linear probing.The authors attribute the gap to aircraft scattering structure and limited aircraft samples in pretraining.Source: §4.3 · p. 12
4.4. Visualization
- Attention analysis shows that SAR-JEPA models local and global features; adding MIM shifts attention toward high-level global context relative to PGCA.LoMaR-SAR models local information, while MAE mainly emphasizes global information in deep layers.Source: §4.4 · p. 12, Figure 6 · p. 12
4.5. Scaling Experiment
- Increasing dataset size significantly improves small-sample classification, while insufficient data can make larger models and longer training overfit.ViT-Large overfits with insufficient data, and overly long training also overfits ViT-Base; the default is 200 epochs.Source: §4.5 · p. 13, Figure 7 · p. 13
5. Conclusion
SAR-JEPA applies local patch masking and SAR-domain gradient features in masked image modeling, and the authors report effective representations across target categories and image conditions. They conclude that exploiting SSL's potential will require much larger pretraining data and identify data, architecture, target-feature, and downstream-task extensions as future work.
5. Conclusion
- SAR-JEPA combines architectures for masked image modeling in a gradient feature space, using local patch masking and SAR-domain features to obtain representations effective across target categories and image conditions.The authors plan to extend the data and computational resources to pursue a foundation model for downstream SAR ATR tasks.Source: §5 · p. 13
- The pretraining dataset needs to scale extensively, like ImageNet, to exploit SSL's potential for SAR ATR, although approaching ImageNet's million-image scale is challenging.The authors identify additional vehicle, ship, aircraft, and building datasets as possible additions.Source: §5 · p. 13
- Future work includes improving data collection and model architectures, exploring new pretraining target features, and evaluating tasks such as detection and segmentation.The authors also point to simulated data and architectures such as ConvNeXt as directions for further study.Source: §5 · p. 13