Source-linked AI summary
Multiple instance dense connected convolution neural network for aerial image scene classification
Qi Bi, Kun Qin, Zhili Li, Han Zhang, Kai Xu
TL;DR
Aerial scene classification is challenging because geospatial objects may occur anywhere and spatial arrangements are complex, while CNNs tend to emphasize global semantics. The paper combines dense connectivity with attention-based multiple-instance pooling and reports the highest overall accuracy on all three evaluated datasets.
Problem
CNNs tend to focus on global semantics, while aerial scenes have complex spatial arrangements and objects can occur anywhere, making local semantics relevant to scene labels.
Source: §1 · p. 1Method
The paper proposes MIDCCNN, combining a dense connected convolutional backbone with attention-based multiple-instance pooling trained with scene-level bag labels.
Source: §1 · p. 2Results
The proposed DCCNN and MIDCCNN achieve the highest OA on all three aerial image datasets, with significant OA improvement on AID and NWPU but not UCM.
Source: §3.3 · p. 4Takeaways & Limitations
The authors attribute the gains to dense connections that propagate features with fewer parameters and attention pooling that highlights important local semantics.
Source: §3.3 · p. 4
Abstract
from arXiv · showhide
With the development of deep learning, many state-of-the-art natural image scene classification methods have demonstrated impressive performance. While the current convolution neural network tends to extract global features and global semantic information in a scene, the geo-spatial objects can be located at anywhere in an aerial image scene and their spatial arrangement tends to be more complicated. One possible solution is to preserve more local semantic information and enhance feature propagation. In this paper, an end to end multiple instance dense connected convolution neural network (MIDCCNN) is proposed for aerial image scene classification. First, a 23 layer dense connected convolution neural network (DCCNN) is built and served as a backbone to extract convolution features. It is capable of preserving middle and low level convolution features. Then, an attention based multiple instance pooling is proposed to highlight the local semantics in an aerial image scene. Finally, we minimize the loss between the bag-level predictions and the ground truth labels so that the whole framework can be trained directly. Experiments on three aerial image datasets demonstrate that our proposed methods can outperform current baselines by a large margin.
1. INTRODUCTION
Aerial scene classification remains challenging because scenes have complex spatial patterns and CNNs tend to emphasize global semantics, despite objects appearing across the image. The introduction motivates preserving middle- and low-level features and local semantics, then proposes dense connectivity with attention-based multiple-instance learning.
1. INTRODUCTION
- Low- and middle-level feature methods are described as inflexible for aerial scenes with intra-class similarity, inter-class variety, and complex spatial arrangements.Source: §1 · p. 1
- CNNs emphasize global semantics, although aerial scenes require attention to local semantics because geospatial objects may appear anywhere and in varied orientations.The introduction also notes that gradients can vanish and middle- and low-level features can be lost in large CNNs.Source: §1 · p. 1
- Dense connectivity feeds each layer's feature maps to subsequent layers, preserving middle- and low-level features, alleviating gradient vanishing, and enabling feature reuse.The introduction also says this structure enables deeper CNNs without severe overfitting problems.Source: §1 · p. 1
- Multiple instance learning treats each training sample as a bag of instances, with densely sampled image patches as instances and relevant ones determining the scene label.Source: §1 · p. 2
- The proposed MIDCCNN pairs a 23-layer DCCNN backbone with attention-based MIL pooling that highlights relevant local semantics and is optimized with the CNN under bag-label supervision.Source: §1 · p. 2
2. METHODOLOGY
MIDCCNN combines a 23-layer dense-connected CNN backbone with attention-based multiple-instance pooling, then trains the full framework using bag-level scene labels. Its design refines the backbone for aerial images and aggregates local predictions into a scene-level prediction.
2.1. Dense Connected Convolution Neural Network
- The 23-layer DCCNN modifies DenseNet by reducing 3×3 convolution layers, refining rather than reducing features in transition layers, and replacing the fourth dense block with a 1×1 convolution layer.The added 1×1 layer further refines extracted convolution features before classification.Source: §2.1 · p. 2
2.2. Multiple Instance Learning Pooling
- Attention-based MIL pooling uses normalized weights to combine instance probability vectors, addressing the spatial arrangement and object-distribution limits of max and average pooling.The weighted combination is described as a convex combination of the instance probability distributions.Source: §2.2 · p. 3
2.3. MIDCCNN Framework
- MIDCCNN uses DCCNN features, an instance classifier, MIL pooling, and cross-entropy on bag-level predictions to optimize the full framework.The instance classifier uses N 1×1 convolution filters, with N equal to the number of scene categories.Source: §2.3 · p. 3
3. EXPERIMENT AND ANALYSIS
The experiments evaluate MIDCCNN on three aerial-image datasets using a repeated overall-accuracy protocol and compare its attention-based pooling with common MIL pooling alternatives. The reported results favor DCCNN and MIDCCNN across datasets, with attention-based pooling also outperforming mean and max pooling.
3.1. Dataset
- Experiments used the UCM, AID, and NWPU aerial-image scene-classification datasets.The datasets and their corresponding training ratios are listed in Table 2; the ratios are not supplied here.Source: §3.1 · p. 3
3.2. Model initialization and training
- Training used Adam with a two-stage schedule: learning rate 0.001 for 100 epochs on AID and NPWU or 40 on UCM, then divided by 10 until it reached 1e-6.Weights were randomly initialized and biases set to 0.001; L2 normalization used a parameter of 1e-6, with dropout rate 0.2.Source: §3.2 · pp. 3–4
3.3. Results and discussion
- DCCNN and MIDCCNN achieved the highest OA on UCM, AID, and NWPU; attention-based pooling also outperformed mean and max MIL pooling.OA was averaged over ten repetitions; Fig. 3 demonstrates attention-based MIL pooling.Source: §3.3 · p. 4, Figure 3 · p. 4
4. CONCLUSION
The paper concludes that MIDCCNN combines a 23-layer dense-connected CNN backbone with attention-based MIL pooling for aerial scene classification. Future work will study feature reuse and explore aerial instance segmentation and object detection.
4. CONCLUSION
- MIDCCNN uses a 23-layer DCCNN backbone and attention-based MIL pooling for aerial image scene classification.The authors attribute its effectiveness to dense connections, relatively few parameters, and 1×1 convolution layers.Source: §4 · p. 4
- The proposed model is intended to highlight local scene semantics while retaining middle- and low-level convolution features.The conclusion reports that experiments on three aerial-image datasets outperformed state-of-the-art methods.Source: §4 · p. 4
- Future work will study feature reuse and apply the framework to aerial instance segmentation and object detection.Source: §4 · p. 4