PaperlayerView exact revision on arXiv

Source-linked AI summary

Half a Percent of Labels is Enough: Efficient Animal Detection in UAV Imagery using Deep CNNs and Active Learning

Benjamin Kellenberger, Diego Marcos, Sylvain Lobry, Devis Tuia

arXiv:1907.07319v1cs.CV

TL;DR

Models trained on one UAV acquisition may not transfer well to another, making it difficult to find rare animals in a new dataset. The paper proposes Transfer Sampling, which uses Optimal Transport to transfer source rankings to target candidates for active-learning queries. After five iterations, it reports finding 443 of 565 animals (78.4%) with 250 queries, and the experiments use simulated oracles.

  • Problem

    A detector trained on one dataset can lose accuracy on another, while rare animals among many false alarms make target-domain labeling difficult.

    Source: §I · p. 2

  • Method

    Transfer Sampling uses Optimal Transport to transfer source-domain rankings to target candidates, which are queried iteratively so oracle labels can fine-tune the target model.

    Source: §II · p. 3, A. Transfer Sampling · p. 3

  • Results

    After five active-learning iterations, Transfer Sampling found 443 of 565 animals (78.4%) with 250 oracle queries, and its recall exceeded the baselines through most of the process.

    Source: §IV · p. 9

  • Takeaways & Limitations

    The approach prioritizes likely true positives to locate rare animals efficiently while iteratively adapting the detector to the target domain.

    Source: §II · p. 3, A. Transfer Sampling · p. 3

    The experiments use ground-truth-based simulated oracles rather than human annotators.

    Source: §II · p. 3

Abstract

from arXiv · show

We present an Active Learning (AL) strategy for re-using a deep Convolutional Neural Network (CNN)-based object detector on a new dataset. This is of particular interest for wildlife conservation: given a set of images acquired with an Unmanned Aerial Vehicle (UAV) and manually labeled gound truth, our goal is to train an animal detector that can be re-used for repeated acquisitions, e.g. in follow-up years. Domain shifts between datasets typically prevent such a direct model application. We thus propose to bridge this gap using AL and introduce a new criterion called Transfer Sampling (TS). TS uses Optimal Transport to find corresponding regions between the source and the target datasets in the space of CNN activations. The CNN scores in the source dataset are used to rank the samples according to their likelihood of being animals, and this ranking is transferred to the target dataset. Unlike conventional AL criteria that exploit model uncertainty, TS focuses on very confident samples, thus allowing a quick retrieval of true positives in the target dataset, where positives are typically extremely rare and difficult to find by visual inspection. We extend TS with a new window cropping strategy that further accelerates sample retrieval. Our experiments show that with both strategies combined, less than half a percent of oracle-provided labels are enough to find almost 80% of the animals in challenging sets of UAV images, beating all baselines by a margin.

I. INTRODUCTION

UAV imagery and CNN detectors can make repeated wildlife censuses faster, but models trained on one acquisition can lose accuracy on another. The paper proposes active learning to retrieve rare true positives in a new acquisition while using oracle labels to improve the detector.

I. INTRODUCTION

  • Repeated censuses need animal-counting tools that are fast, reliable, and suitable for repeated use over time.Up-to-date population densities and locations can inform grazing estimates and anti-poaching efforts.Source: §I · p. 1
  • UAV-based CNN detectors can scan thousands of images, but acquisition differences can sharply reduce accuracy when a model is applied to another dataset.The source-trained model may retain high recall while producing many false alarms, lowering precision.Source: §I · p. 1
  • Because animals are rare among background imagery, finding a few target labels by manually sifting through images is inefficient and can cause fatigue and erroneous labels.Active learning lets a model query an oracle for ground truth on selected samples.Source: §I · p. 2
  • Conventional active-learning criteria prioritize samples expected to improve target-model performance, often using uncertainty or model-specific rankings.Examples include Breaking Ties, SVM margin sampling, expected model change, and Bayesian CNNs.Source: §I · p. 2
  • For animal censuses, the priority is locating animals, so sampling very confident true-positive candidates can be more useful than querying uncertain predictions.The paper also identifies two objectives: rapid localization and model improvement through iterative updates rather than one-shot sampling.Source: §I · p. 2
  • On a UAV dataset with two distinct acquisitions characterized by domain shifts, the proposed smart sampling strategy retrieved 80% of the animals by screening only half a percent of the acquired dataset.Source: §I · p. 2

II. PROPOSED METHOD

The method uses a source-trained detector and a current target-domain CNN to produce candidates, then matches source and target feature distributions with Optimal Transport. It transfers source rankings to target candidates for oracle queries, whose labels support iterative fine-tuning.

II. PROPOSED METHOD

  • Optimal Transport matches source and target CNN-feature distributions, allowing source ground-truth labels to be transferred to target samples for expert verification.The workflow assumes a source dataset and an object detector trained on it.Source: §II · p. 3
  • Transfer Sampling ranks source samples with an SVM and transports that ranking to target candidates, which anchor patches submitted as oracle queries.Candidates are predicted using the original source CNN and the current target CNN.Source: Figure 2 · p. 3
  • Oracle labels for likely true positives iteratively fine-tune the model to the target domain, while window cropping is proposed to facilitate and accelerate manual annotation.Experiments use simulated, ground-truth-based oracles.Source: §II · p. 3

A. Transfer Sampling

Transfer Sampling ranks target candidates by transferring confidence from likely true positives in the source domain through Optimal Transport correspondences. It prioritizes confident predictions to retrieve animals quickly despite domain shift and class imbalance.

A. Transfer Sampling

  • TS starts with source-model candidates predicted at 10% or more animal confidence and seeks those most likely to be true positives in the target domain.The approach assumes true-animal predictions cluster in the source CNN feature space and a similarly relevant region exists in the target.Source: A. Transfer Sampling · p. 3
  • An SVM ranks source candidates by distance from its separating hyperplane, prioritizing samples farthest from the decision boundary as the most trustworthy.This adapts margin-based ranking to select the most certain positive-class regions rather than uncertain samples.Source: A. Transfer Sampling · p. 4
  • Optimal Transport maps source and target candidates by finding sparse correspondences that minimize global cost between their CNN feature-space distributions.Links may be one-to-many or many-to-one, depending on the relative numbers of source and target samples.Source: A. Transfer Sampling · pp. 4–5
  • 51 target true positives (~10% of target true positives) and 759 target false positives (~2.6% of target false positives) were linked from source true positives, with globally consistent correspondences despite class imbalance.The figure illustrates the correspondences; the counts and percentages are reported in the text.Source: Figure 4 · p. 5, A. Transfer Sampling · p. 5
  • TS transfers source SVM distances along nonzero OT links and averages linked source scores to rank each target candidate.The approach ranks the target samples for priority sampling after transferring those scores.Source: A. Transfer Sampling · pp. 5–6

B. Window cropping for patch-based labeling

Window cropping expands a selected candidate into a patch that an oracle labels, rather than querying candidates individually. The patch balances candidate coverage, overlap with earlier queries, and centering the selected candidate.

B. Window cropping for patch-based labeling

  • Window cropping labels a fixed-size patch around each Transfer Sampling candidate instead of querying only that prediction.The oracle returns animal positions within the patch and labels other locations in the image as background.Source: B. Window cropping for patch-based… · pp. 6–7
  • Patches can capture neighboring candidates, potentially speeding animal retrieval and providing false-positive labels that reduce later false predictions.The authors note that nearby animals may occur in herds.Source: B. Window cropping for patch-based… · p. 6
  • Figure 5 contrasts a multi-candidate patch with per-candidate queries, which the authors report are slower and less intuitive for annotators.Providing spatial context also makes animals easier for humans to recognize.Source: Figure 5 · p. 6, B. Window cropping for patch-based… · p. 6
  • The rectangle-selection objective favors including candidates, avoiding overlap with queried patches, and keeping the selected anchor near the window center.The overlap term uses the maximum intersection with previously queried rectangles; centering matters especially when no other candidates or prior windows are present.Source: B. Window cropping for patch-based… · pp. 6–7
  • A 1000 × 1000-pixel patch, approximately 60 × 60 m, was used with a simulated oracle as a compromise between labeling difficulty and query efficiency.Larger patches were harder to label, while smaller patches used the query budget faster and provided less context.Source: B. Window cropping for patch-based… · p. 7

III. EXPERIMENTS

The experiments section introduces the two datasets and the detector and active-learning parameters used to evaluate the method.

III. EXPERIMENTS

  • The experiments test the proposed workflow using data and parameters described in the following subsections.The section points to dataset descriptions in III-A and detector CNN and active-learning parameters in III-B.Source: §III · p. 7
  • Section III-A describes the two datasets used in the evaluation.Source: §III · p. 7
  • Section III-B covers the detector CNN and active-learning routine parameters.Source: §III · p. 7

A. Study Area and Data

The study uses two UAV image acquisitions over Kuzikus wildlife reserve, Namibia, with substantial acquisition and label-space differences. The 2014 data are divided into source splits, while the 2015 data use a validation set but no test set.

A. Study Area and Data

  • The 2014 and 2015 UAV campaigns produced 654 and 3254 images, respectively, using a SenseFly eBee with an RGB compact digital camera.The 2014 labels began with crowdsourcing and were refined by the authors; the authors labeled the 2015 images.Source: A. Study Area and Data · p. 7
  • The 2014 and 2015 Kuzikus datasets differ in acquisition conditions and class imbalance, with the larger 2015 set having a lower animal proportion.The campaigns occurred in different months and years, with varying weather, lighting, cameras, and flight altitudes; 2014 animal-to-background pixel imbalance was 1:10’000.Source: A. Study Area and Data · p. 7
  • The 2014 source dataset was divided into training, validation, and test sets by assigning entire images to one set, with animal-containing images allocated to yield animal-count shares of 70%, 10%, and 20%, respectively.Assigning entire images to a single set was intended to avoid autocorrelation effects.Source: A. Study Area and Data · p. 7
  • The 2015 target data had no test set because sampling used the oracle, but a small validation set was retained for hyperparameter tuning.The paper identifies Tables I and II as overviews of the datasets and their split properties, respectively.Source: A. Study Area and Data · p. 8, Table I · p. 8, Table II · p. 8

B. Model Setup

The model-setup section outlines the deep CNN detector and the active-learning framework.

B. Model Setup

  • The model-setup section presents the deep CNN used for animal detection and the active-learning framework.Source: B. Model Setup · p. 8
  • One subsection explains the deep CNN detector and its parameters.Source: B. Model Setup · p. 8
  • A separate subsection provides details on the active-learning framework.Source: B. Model Setup · p. 8

1) CNN Training:

The detector adapts a ResNet-18-based architecture for animal, background, and border predictions on image patches. Training follows recommendations for heavily imbalanced census data, with lower-rate fine-tuning on the growing target dataset during active learning.

1) CNN Training:

  • Training follows animal-census recommendations for imbalanced data: class weights, a border class, curriculum learning, and hard negative mining.The border class is placed in the 8-neighborhood of animal locations; hard negative mining emphasizes the four most confidently predicted false alarms after epoch 80.Source: 1) CNN Training · p. 8
  • The detector uses a pretrained ResNet-18 and two MLPs to predict background, animal, or border labels on a 32 × 32 grid from 512 × 512 patches.The 512-dimensional feature vectors from the last residual block support further adaptation, and the feature vector after MLP 1 is used by Optimal Transport within Transfer Sampling.Source: 1) CNN Training · p. 8, Figure 7 · p. 8
  • For target-domain fine-tuning, the learning rate drops from 10^-6 to 10^-7, curriculum learning is disabled, and training uses the full growing active-learning dataset.Other training procedures, including hard negative mining, remain as in the source model.Source: 1) CNN Training · pp. 8–9

2) AL Loop:

The active-learning loop starts from source-model predictions, then iteratively queries target patches and fine-tunes the detector. The evaluation compares three sampling baselines with Transfer Sampling, both with and without model updates.

2) AL Loop:

  • The source-trained detector supplies candidates at animal confidence of 0.1 or higher, with non-maximum suppression reducing duplicate predictions.Suppression uses a radius of two prediction-grid cells and retains the highest-confidence candidates in a 4-neighborhood.Source: 2) AL Loop · p. 9
  • The experiment runs ten iterations, querying 50 patches of 1000 × 1000 pixels each and fine-tuning the CNN for 12 epochs per iteration.Larger queried patches enable additional training augmentation through random crops of labeled areas.Source: 2) AL Loop · p. 9
  • Transfer Sampling is compared with random selection, Breaking Ties, and max confidence, with each strategy evaluated with and without iterative fine-tuning and candidate re-prediction.The no-update scenario uses source-model predictions once and samples only from those candidates.Source: 2) AL Loop · p. 9

IV. RESULTS AND DISCUSSION

Transfer Sampling retrieves many target animals with few queries and leads the baselines through most of the process. Its feature-space sampling, combined with model updates and window cropping, supports retrieval in highly imbalanced settings.

IV. RESULTS AND DISCUSSION

  • 443 of 565 animals (78.4%) were found after five iterations and 250 queries; after ten iterations, the total reached 473 (83.72%).The authors estimate that the five-iteration result required reviewing around half a percent of 54,324 queryable patches.Source: §IV · p. 9
  • Transfer Sampling had higher recall than all baselines through almost the entire process, reaching max confidence’s roughly same level one iteration earlier.The TS and max-confidence examples show TS sampling the feature-space region where true positives concentrate, rather than relying on confidence ranking alone.Source: §IV · pp. 9–10, Figure 9 · p. 10
  • In an example image, only Transfer Sampling located every animal with few queries; the other strategies found some true positives, mainly through window cropping.Breaking Ties sampled transition areas between shadow and ground, while max confidence missed the center-right hotspot.Source: §IV · p. 10
  • Across strategies, model updates and candidate re-predictions significantly boosted results over static one-shot sampling.Window cropping increases the image data and information gain per patch by including more predicted candidates, both true and false positives.Source: §IV · p. 10
  • After five iterations, the TS-trained CNN produced more candidates than the baselines; in fully automated evaluation, the additional predictions could also mean more false positives.The authors report that TS can filter predictions efficiently to localize animals even in increasingly imbalanced settings.Source: §IV · pp. 10–11

V. CONCLUSION

The conclusion presents Transfer Sampling and window cropping as an interactive approach to adapting animal detectors for repeated UAV censuses. The authors identify human annotation and broader geographic testing as directions for future work.

V. CONCLUSION

  • In the Kuzikus reserve, Transfer Sampling retrieved 78.4% of animals in 250 queries, with the oracle reviewing less than half a percent of the dataset.The method combines Optimal Transport-based transfer of source-detector knowledge with window cropping to increase detections labeled per query.Source: §V · p. 11
  • Future work could test human annotators and acquisitions from other geographic areas, beyond new acquisitions over the same reserve.The authors also suggest incorporating users’ confidence in providing ground truth.Source: §V · p. 11

Related papers

Loading 1907.07319v1…