PaperlayerView exact revision on arXiv

Source-linked AI summary

A survey of active learning algorithms for supervised remote sensing image classification

Devis Tuia, Michele Volpi, Loris Copa, Mikhail Kanevski, Jordi Munoz-Mari

arXiv:2104.07784v1cs.CV

TL;DR

Remote sensing classification needs compact, representative training sets, but labeling can be costly and affected by redundancy, noise, and complex feature spaces. This paper reviews and tests committee-, large margin-, and posterior probability-based active-learning methods across challenging remote-sensing scenarios. It reports that active-learning heuristics outperform random selection at convergence and identifies large-margin methods with diversity as especially appropriate in the discussed setting.

  • Problem

    Remote sensing classification requires representative training data, but collecting labels is costly and limited, while redundant or noisy samples can impair class-statistics representation.

  • Method

    The paper reviews and compares active-learning heuristic families, including committee, large margin, and posterior probability approaches, across remote-sensing applications.

  • Results

    At convergence, all heuristics outperformed random pixel selection, and large margin-based methods with diversity criteria appeared most appropriate in the discussed setting.

  • Takeaways & Limitations

    Heuristic selection should be problem-oriented and depend on the user’s needs when choosing an active-learning architecture.

Abstract

from arXiv · show

Defining an efficient training set is one of the most delicate phases for the success of remote sensing image classification routines. The complexity of the problem, the limited temporal and financial resources, as well as the high intraclass variance can make an algorithm fail if it is trained with a suboptimal dataset. Active learning aims at building efficient training sets by iteratively improving the model performance through sampling. A user-defined heuristic ranks the unlabeled pixels according to a function of the uncertainty of their class membership and then the user is asked to provide labels for the most uncertain pixels. This paper reviews and tests the main families of active learning algorithms: committee, large margin and posterior probability-based. For each of them, the most recent advances in the remote sensing community are discussed and some heuristics are detailed and tested. Several challenging remote sensing scenarios are considered, including very high spatial resolution and hyperspectral image classification. Finally, guidelines for choosing the good architecture are provided for new and/or unexperienced user.

I. INTRODUCTION

Remote sensing classification depends heavily on representative, compact training sets, yet collecting labels is costly or redundant across field surveys and image interpretation. Active learning addresses this challenge by iteratively selecting uncertain, informative pixels for labeling, and the paper reviews remote-sensing applications and methods.

  • Motivation: Training-set representativeness strongly affects supervised classification, making manual dataset generation difficult and expensive.Labels may require extensive visual inspection, field surveys, and successive image labeling.
  • Motivation: Field surveys limit the number of obtainable labels because discovering new labels requires time-consuming and costly terrain campaigns.The resulting constraint motivates compact and informative training sets.
  • Motivation: Visual interpretation can produce redundant neighboring samples and noisy pixels, slowing training and misrepresenting class statistics.These effects may lead to poor classification performance and overfitting.
  • Motivation: A desirable training set samples discriminative pixels that represent class boundaries, especially in high spatial- or spectral-resolution imagery with complex feature spaces.Such scenarios often provide only limited training information.
  • Active learning: Active learning seeks a small, well-chosen training set that can match randomly selected larger sets while reducing computational cost.Its core interaction returns uncertain pixels to the user for labeling and adds them to the training set.
  • Paper scope: The paper presents a general active-learning framework, reviews remote-sensing methods, and organizes the discussion around its application-specific scope.The survey excludes general machine-learning developments and outlines datasets, numerical comparisons, user guidelines, and conclusions.

II. ACTIVE LEARNING: CONCEPTS AND DEFINITIONS

Active learning iteratively enlarges a labeled training set with pixels selected for their expected benefit to model performance. Its heuristics rank unlabeled candidates, support user labeling, and are commonly grouped into committee-, large margin-, posterior probability-, and separately discussed cluster-based approaches.

  • Framework: Active learning adapts a classifier by repeatedly adding newly labeled pixels selected for their expected benefit to model performance.The process continues until a stopping criterion is met.
  • Framework: At each iteration, the algorithm trains on the current set, evaluates candidates, ranks them with a user-defined heuristic, and selects a batch for labeling.The batch size q determines how many pixels are added per iteration.
  • Framework: Selected pixels are added to the current training set and removed from the candidate pool before the process repeats.This update reduces model uncertainty in regions where confidence is low.
  • Interaction: The user supplies labels and class knowledge, while the model identifies relevant pixels and exposes discrepancies requiring resolution.The ranking strategy is central because heuristics differentiate the subsequent algorithm families.
  • Heuristic families: The main heuristic families are committee-based, large margin-based, and posterior probability-based.Cluster-based heuristics instead prune hierarchical clusters using an unsupervised model and are not detailed because their aim differs.
  • Heuristic families: Cluster-based heuristics prune a hierarchical clustering tree until its clusters are consistent with user-provided labels.They rely on an unsupervised rather than predictive model and therefore fall outside the detailed families.

III. COMMITTEE BASED ACTIVE LEARNING

Committee-based active learning measures uncertainty through disagreement among multiple classifiers. In query-by-bagging, entropy ranks candidates, while normalized variants and feature-space views adapt the committee to multiclass and high-dimensional data.

  • Committee design: Committee approaches can use pixel subsets, feature-space subsets, boosting, or bagging to construct alternative classifier committees.The reviewed literature includes binary and remote-sensing applications of these constructions.
  • Committee principle: Committee methods select pixels whose predicted labels show maximal disagreement among multiple classification models.The committee represents different hypotheses about the classification problem.
  • Query-by-bagging: Bagging committees train classifiers on labeled samples drawn with replacement and obtain multiple predictions for every candidate pixel.Entropy of the prediction distribution provides the query-by-bagging heuristic.
  • Query-by-bagging: Normalized entropy bounds uncertainty relative to the number of committee-predicted classes and reduces overlap-region uncertainty hotspots.This modification was introduced for the normalized entropy query-by-bagging heuristic.
  • Query-by-bagging: Entropy maximization naturally supports multiclass selection: unanimous predictions have null entropy, whereas maximal disagreement has maximum entropy.Candidates with maximum disagreement are expected to contribute substantial information when labeled.

B. Adaptive maximum disagreement (AMD)

Large margin methods use classifier distance and support-vector behavior to identify uncertain samples, while multiview committee methods extend disagreement-based selection to high-dimensional data. The paper compares these strategies and emphasizes that heuristic choice depends on the problem and user needs.

  • B. Adaptive maximum disagreement (AMD): Multiview entropy evaluates predictions from classifiers trained on disjoint feature-space views, with adaptive weights reflecting class-specific discrimination abilities.Weights are updated using labels from previously sampled pixels and normalized across views.
  • B. Adaptive maximum disagreement (AMD): Adaptive maximum disagreement selects candidates for which the multiview committee predicts the highest number of classes.Restricting selection to maximally uncertain candidates reduces computational load.
  • IV. LARGE MARGIN BASED ACTIVE LEARNING: Large margin heuristics estimate confidence from the absolute SVM decision-function value, treating samples near the separating boundary as more uncertain.Samples far from the boundary have higher confidence and are less interesting for sampling.
  • IV. LARGE MARGIN BASED ACTIVE LEARNING: Margin sampling targets unlabeled pixels likely to become support vectors because they lie within the current model’s margin.Later modifications address the risk of selecting points that will not become support vectors.
  • IV. LARGE MARGIN BASED ACTIVE LEARNING: Because margin sampling lacks cross-sample information, subsequent work introduces diversity criteria for selecting batches.These modifications aim to improve batch diversity rather than evaluating candidates independently.
  • IV. LARGE MARGIN BASED ACTIVE LEARNING: Figure 1 visualizes large-margin uncertainty for a three-class toy example, with black regions marking maximal uncertainty for the illustrated heuristics.The figure also shows per-class absolute distances in its bottom row.

A. Margin sampling (MS)

Margin-based active learning uses SVM geometry to select candidates near decision boundaries, with multiclass extensions comparing the two most probable classes and alternatives based on support-vector structure.

  • Margin sampling (MS): Margin sampling selects candidates by minimizing their distance to class-specific SVM hyperplanes.In multiclass one-against-all classification, each class has a corresponding hyperplane distance.
  • Multiclass level uncertainty: Multiclass level uncertainty compares distances to the margins of the two most probable classes rather than using only the most uncertain class.A small difference indicates unreliable classification, while a high value indicates greater confidence in the leading class.
  • Multiclass level uncertainty: MCLU returns maximal uncertainty where three classes mix, outperforming MS in the corresponding top-right region of the toy feature space.The comparison evaluates uncertainty using all three class decision values in that mixed region.

D. On the need for a diversity criterion

Diversity criteria complement uncertainty-based selection by reducing redundancy among queried pixels. Proposed methods use feature-space angles, support-vector structure, clustering, and hierarchical partitioning to diversify batches.

  • Motivation: Diversity is desirable because it lets the model reject highly ranked candidates that are redundant with already selected samples.The criterion is intended for applicative settings where selected samples should provide complementary information.
  • Angle-based diversity: Most ambiguous and orthogonal constrains margin sampling with feature-space angles and iteratively selects samples from the uncertain candidate set.MAO was among the first remote-sensing heuristics to introduce explicit diversity.
  • Angle-based diversity: MCLU-ABD combines MCLU uncertainty with angle-based diversity, while extending MAO through feature-space normalization for general kernels.Selection is performed within a subset of candidates maximizing the MCLU criterion.
  • Support-vector diversity: Support-vector-based diversification constrains margin sampling using different closest support vectors, but does not guarantee that selected samples are mutually separated.Two nearby samples can still be associated with different support vectors.
  • Clustering-based diversity: MCLU-ECBD partitions uncertain samples into q kernel k-means clusters and selects one pixel per cluster.This creates a diverse batch while retaining MCLU as the uncertainty-selection stage.
  • Hierarchical diversity: hMCS-i hierarchically partitions uncertain candidates together with previously bounded support vectors until q clusters without bounded support vectors are obtained.The procedure controls redundancy across iterations while preserving informativeness.

V. POSTERIOR PROBABILITY BASED ACTIVE LEARNING

Posterior-probability methods rank unlabeled pixels using estimated class-membership probabilities and focus sampling on candidates whose inclusion most changes or equalizes those distributions.

  • Posterior probability framework: Posterior-probability heuristics use p(y|x) to estimate class-assignment confidence and rank candidates by uncertainty.They can examine changes in the overall posterior distribution or in per-class distributions.
  • Heuristics: The reviewed posterior-probability section details KL-max and Breaking ties as two uncertainty-based heuristics.Both are presented as strategies for selecting informative candidates from posterior estimates.
  • KL-max: KL-max selects the candidate whose addition maximizes the Kullback-Leibler divergence between posterior distributions before and after inclusion.Each candidate is temporarily added with the label having maximal posterior probability, and the resulting distributions are compared.
  • Computational limitation: These methods are most suitable for inexpensive classifiers because each iteration trains u+1 models, limiting use with computationally demanding methods such as SVM.They also do not support batch selection of pixels.

B. Breaking ties (BT)

Breaking ties selects candidates with nearly equal posterior probabilities, while the experiments evaluate active heuristics across three challenging remote-sensing image scenarios and varied data distributions.

  • Posterior estimation: Posterior probabilities can be obtained from probabilistic neural networks, maximum likelihood classifiers, or SVM outputs calibrated with Platt’s estimation.Platt’s approach fits a sigmoid function to the SVM decision function for each class.
  • Breaking ties: Breaking ties selects samples with the smallest difference between their two highest posterior probabilities.Close top probabilities indicate low classifier confidence in multiclass settings.
  • Experimental scenarios: The paper tests active heuristics on three challenging remote-sensing classification scenarios whose data distributions are presented separately.The experiments include a hyperspectral 1.3m spatial-resolution image among the described scenarios.
  • Pavia dataset: The Pavia dataset contains five selected classes and 206‘009 visually inspected labeled pixels, with 20‘000 pixels allocated to training and candidate sets.The classes are Buildings, Roads, Water, Vegetation and Shadows.
  • Experimental setup: Ten independent experiments begin with 25 labeled pixels, while LDA experiments start with 150 pixels because of the dimensionality requirement.The standard initialization uses 5 pixels per class, whereas LDA uses 30 pixels per class.
  • Data distribution: Pavia’s five classes are well separated using spectral information alone, and its data manifold can be handled by linear and nonlinear models.This characterization is based on the mean spectral profiles and the resulting data manifold.

2) Hyperspectral MR:

The section describes three remote-sensing datasets spanning hyperspectral, high-resolution, and spatial-feature-enhanced classification, each presenting distinct class-separation challenges.

  • Hyperspectral datasets: A second dataset exhibits high nonlinearity and strongly overlapping classes, so linear classifiers are excluded from its experiments.Its data manifold and spectral mean profiles illustrate the separation difficulty.
  • Hyperspectral datasets: The Indian Pines AVIRIS benchmark contains 145 × 145 pixels, 16 classes, and 10‘366 labeled pixels.It is a challenging crop-classification benchmark because class signatures are strongly mixed.
  • Hyperspectral datasets: After removing classes with fewer than 100 labeled pixels, Indian Pines becomes a 12-class problem with 10‘171 labeled pixels.Seven thousand labeled pixels were assigned to the X and U sets, with 60 initial training pixels per experiment.
  • High-resolution datasets: The third dataset is a 4-band image whose complexity is marked by strong overlap between asphalt and soil classes.The experiments use 20‘000 extracted pixels and begin with 45 training pixels, five per class.
  • High-resolution datasets: Spatial features improve class differentiation in the third dataset, including opening features for vegetation and closing features for asphalt.These features help despite spectral overlap between some classes.

B. Experimental setup

The experiments compare active-learning heuristics across SVM and LDA classifiers, using random sampling and standard full-data models as references. Results favor large-margin methods for SVMs, while LDA heuristics improve subspace estimation and diversity can help with little added cost.

  • Experimental setup: SVM classifiers use an RBF kernel, while LDA classifiers are also evaluated.SVM free parameters are optimized by five-fold cross-validation using accuracy.
  • Experimental setup: Standard SVM/LDA models are trained on X ∪ U, and Random Sampling adds equal numbers of randomly selected pixels at each epoch.Random Sampling serves as the lower-bound reference.
  • Experimental setup: Each heuristic is run ten times with different initial training sets, and plots report the mean and standard deviation.This repeated evaluation addresses variation caused by initialization.
  • Comparisons: The tested heuristics are nEQB, MS, MCLU, MCLU-ABD, and BT, compared with Random Sampling, standard SVM/LDA, and one another.The comparison is intended to expose differences among active-learning architectures rather than identify one universally best heuristic.
  • SVM results: For SVMs, MS generally performs better than nEQB and BT because it uses the SVM decision function directly without further estimations.The smaller initial training set limits committee and posterior-probability estimates, making their early performance similar to random sampling.
  • SVM results: When using SVMs, the paper identifies a large-margin heuristic as the most logical family choice.MS is the representative large-margin approach in these comparisons.
  • Large-margin results: MS and MCLU describe candidate uncertainty similarly, while adding diversity slightly improves results and MCLU-ABD increases performance with little added computational cost except on AVIRIS.No diversity-based solution degrades the corresponding uncertainty-only solution.
  • LDA results: For LDA on the Pavia image, nEQB and BT perform similarly, and active sampling helps estimate a subspace that better separates the data.From 330 pixels onward, active-learning training sets improve the standard LDA result by producing more harmonious solutions.

VIII. DISCUSSION

The experiments indicate that active-learning heuristic choice should be problem-oriented, balancing image characteristics, classifier properties, sampling logistics, computational cost, and batch size.

  • Nearly all compared algorithms showed fast convergence toward the Standard SVM/LDA upper bounds.
  • Heuristics outperformed random pixel selection, while diversity criteria provided a slight advantage in experiments.
  • Large margin methods with diversity criteria are recommended for SVM, while committee heuristics allow users to exploit preferred classifier models.
  • Heuristic selection should depend on user needs, including time, complexity, and batch size, rather than classifier model alone.
  • For hyperspectral images, multi-view and spectral-diversity heuristics such as AMD and ECBD-ABD directly exploit data structure.
  • With very small initial training sets, posterior-probability heuristics and bagging-based committees should be avoided because estimates depend strongly on class-statistics quality.
  • Small batches and many iterations favor complex diversity heuristics, whereas large field batches favor simple heuristics such as MCLU, BT, or EQB.
  • For sequential moving-sensor sampling, computationally inexpensive models such as BT and KL-max can rapidly provide the next sampling location.

IX. CONCLUSION

The paper compares active learning approaches for remote sensing classification, analyzes their advantages and drawbacks on challenging multispectral and hyperspectral datasets, and offers recommendations for improvement. It also identifies open issues including limited contextual information, noise robustness, and the review’s non-exhaustive scope.

  • The study classifies active learning heuristics into four families and details selected heuristics from each family.
  • The selected heuristics are applied to three challenging remote sensing datasets for multispectral and hyperspectral classification.
  • Advantages and drawbacks of each method are analyzed, supporting recommendations for further improvement.
  • The review is not exhaustive, and active learning research in remote sensing remains ongoing.
  • Future heuristics should incorporate contextual information because existing approaches mainly use spectral criteria and few directly consider position or texture.
  • Current uncertainty-based heuristics are ineffective for highly noisy images such as SAR.
Loading 2104.07784v1…