PaperlayerView exact revision on arXiv

Source-linked AI summary

Detecting Mammals in UAV Images: Best Practices to address a substantially Imbalanced Dataset with Deep Learning

Benjamin Kellenberger, Diego Marcos, Devis Tuia

arXiv:1806.11368v1cs.CV

TL;DR

Large wildlife censuses need scalable detection because traditional surveys are costly and risky, while prior wildlife studies often used datasets too small or balanced for realistic deployment. The paper develops CNN training recommendations and census-oriented evaluation protocols, achieving fewer false detections and substantially less manual verification at 90% recall.

  • Problem

    Prior wildlife detection studies often rely on small or balanced datasets, whereas realistic reserves contain few animals amid abundant, heterogeneous background.

  • Method

    The paper trains CNN animal detectors using recommendations for imbalanced, large-area UAV data and evaluates detections by census counts and required verification tiles.

  • Results

    Less than one third of baseline image tiles required verification at 90% recall, while the recommended CNN substantially improved high-recall precision.

  • Takeaways & Limitations

    The approach reduces the manual screening burden for UAV-based wildlife censuses while retaining detection of almost all animals at the stated recall level.

  • Takeaways & Limitations

    Models trained on small subsets do not provide trustworthy results when extrapolated to larger areas, and one evaluated model failed to exceed approximately 74% recall.

Abstract

from arXiv · show

Knowledge over the number of animals in large wildlife reserves is a vital necessity for park rangers in their efforts to protect endangered species. Manual animal censuses are dangerous and expensive, hence Unmanned Aerial Vehicles (UAVs) with consumer level digital cameras are becoming a popular alternative tool to estimate livestock. Several works have been proposed that semi-automatically process UAV images to detect animals, of which some employ Convolutional Neural Networks (CNNs), a recent family of deep learning algorithms that proved very effective in object detection in large datasets from computer vision. However, the majority of works related to wildlife focuses only on small datasets (typically subsets of UAV campaigns), which might be detrimental when presented with the sheer scale of real study areas for large mammal census. Methods may yield thousands of false alarms in such cases. In this paper, we study how to scale CNNs to large wildlife census tasks and present a number of recommendations to train a CNN on a large UAV dataset. We further introduce novel evaluation protocols that are tailored to censuses and model suitability for subsequent human verification of detections. Using our recommendations, we are able to train a CNN reducing the number of false positives by an order of magnitude compared to previous state-of-the-art. Setting the requirements at 90% recall, our CNN allows to reduce the amount of data required for manual verification by three times, thus making it possible for rangers to screen all the data acquired efficiently and to detect almost all animals in the reserve automatically.

1. Introduction

Large-mammal censuses are important but traditional approaches are costly and risky, motivating UAV-based CNN detection that can scale to realistic, highly imbalanced datasets.

  • Manual surveys expose operators to wild animals and poachers and require substantial labor, while manned aircraft remain expensive and operator-dependent.
  • UAVs reduce operator risk and acquisition costs while providing access to otherwise inaccessible areas from a safe distance.
  • Automated detection can preselect likely animal-containing images for human review, reducing the fraction of data requiring screening despite possible false negatives.
  • Prior wildlife detectors often used small, confined, or balanced datasets, limiting assessment of performance when scaled to larger areas.
  • CNNs jointly learn classifiers and task-specific features, making them suitable for heterogeneous animal appearances and background conditions.
  • An order-of-magnitude reduction in false detections and up to 90% recall were achieved, while detections required significantly less manual verification.

2. Data

The study uses UAV imagery from a Namibian wildlife reserve where animals are rare within extensive, heterogeneous backgrounds, making annotation, detection, and large-area generalization difficult.

  • 2.1. Study area and ground truth: The Kuzikus reserve covers approximately 103 km^2, contains more than 3000 large mammals across over 20 species, and supports Black Rhino conservation.
  • 2.1. Study area and ground truth: Five UAV campaigns acquired 654 RGB images of 3000 × 4000 pixels between May 12 and May 15, 2014.
  • 2.1. Study area and ground truth: Crowdsourced convex-hull annotations required revision because they sometimes omitted animals, misplaced them, or combined multiple individuals.
  • 2.1. Study area and ground truth: The dataset was split image-wise into roughly 70% training, 10% validation, and 20% testing, with three training-validation splits and one shared test set.
  • 2.2. The challenges of covering large areas: Larger areas increase background heterogeneity and animal-background confusion, so results from small subsets do not transfer reliably to large study areas.
  • 2.2. The challenges of covering large areas: Most images contain no animals, creating severe class imbalance, while animals are difficult to locate amid heterogeneous savanna backgrounds.

3. Addressing realistic and imbalanced datasets

The paper outlines training and evaluation practices for semi-automated animal censuses on realistic UAV datasets, emphasizing CNN training under imbalance and large-area conditions.

  • The methodology reviews CNNs, presents training practices for learning varied backgrounds alongside scarce animals, and defines a census-oriented evaluation protocol.

3.1. Working principle of Convolutional Neural Networks

CNNs learn hierarchical, image-local features through convolutional processing and optimize them with a differentiable loss and backpropagation.

  • CNNs learn hierarchical features optimized for a task, forming the core of many computer-vision pipelines.
  • Learned features help address difficult scenes where small animals resemble background objects such as tree trunks and dirt mounds.
  • Convolutions extract the same local feature at each image location using neighboring pixels, producing activation maps for subsequent processing.
  • Nonlinear functions between convolutions enable richer input-output relationships, while downsampling is commonly applied after convolutional layers.
  • The CNN uses cross-entropy loss to compare predictions with true class labels and can weight classes differently during learning.
  • Backpropagation computes parameter gradients through the network and updates learnable parameters using gradient-based optimization.

3.2. Training deep object detectors on imbalanced datasets

Training CNNs on UAV wildlife imagery requires addressing severe class imbalance, background diversity, and spatial label spillover. The paper recommends staged training, controlled augmentation, and a dedicated border class to reduce confusion around animal locations.

  • Class imbalance: The background class overwhelmingly exceeds the positive class, so unmodified CNN training can miss most animals.The imbalance concerns both sample quantity and sample complexity.
  • Class imbalance: Class weighting reduces the background class’s influence without duplicating scarce animal instances.The authors report that oversampling caused overfitting to the small number of animal instances.
  • Curriculum learning: Curriculum learning begins with animal-containing patches, then switches to the full dataset to establish a more balanced representation.The animal-containing subset is used for five epochs before training includes patches without animals.
  • Data augmentation: Extensive rotational augmentation can impose overly strong perturbations because it substantially changes the training distribution.The paper notes that 50% random rotation can expose the model to up to 1.5 times as many data points.
  • Border class: Receptive-field spillage can label transition areas incorrectly and is especially problematic when animals stand close together.Reducing the receptive field risks overly fine prediction grids, duplicate alarms, and incomplete animal representations.
  • Border class: A dedicated border class separates locations containing animal portions from true centers, allowing border predictions to be discarded at test time.In this implementation, the eight neighboring pixels around an animal center receive the border label.

3.3. Census-oriented evaluation protocol

The census-oriented protocol evaluates detections primarily by whether they produce reliable animal counts, while allowing reasonable positional error. It rejects duplicate or unsupported predictions and constrains true positives to the number of animals present.

  • Protocol objective: The protocol prioritizes animal abundance over exact pixel localization when judging detection quality.This reflects the final census objective, in which positional accuracy is secondary.
  • Matching rules: Predictions qualify as candidate matches only when they fall within a circular distance range around a ground-truth center.The distance range is the protocol’s manually set parameter.
  • Matching rules: Multiple predictions within one animal’s range count as one true positive and n −1 false alarms.This prevents duplicate detections from inflating census performance.
  • Matching rules: A prediction near multiple animals counts as at most one true positive when at least one matched animal lacks another prediction.If every involved animal is already covered, the prediction is discarded as a false positive.
  • Census consistency: The protocol ensures that true positives cannot exceed the actual number of animals.Its best score occurs when prediction count matches animal count and positions are reasonably close.
  • Detector output: The detector operates on 512 × 512 patches and produces a 32 × 32 prediction grid of animal probability scores.The grid is described as part of the detector’s working principle.

4. Experiments and results

The experiments compare baseline and CNN models, test training recommendations through ablations, examine evaluation distance thresholds, and report full-dataset performance using the best configuration.

  • Experimental design: Experiments evaluate baseline and CNN models before testing training recommendations, distance thresholds, and full-dataset performance.Rotational augmentation receives a dedicated ablation study.
  • Experimental design: The final full-dataset results are based on the best configuration identified through the preceding experiments.The passage describes this as the final stage of the evaluation sequence.
  • Evaluation analysis: Evaluation distance thresholds are examined separately to assess their effects on census-oriented scoring.This analysis follows the ablation studies of the CNN training recommendations.

4.1. Models setup

The experiments use a simplified detection-only CNN, evaluate recommendations through held-out ablations, and apply the model over overlapping image patches to produce coarse prediction grids. A state-of-the-art object-proposal system serves as the baseline.

  • CNN model: The CNN uses a simplified ResNet-18 feature extractor fine-tuned with added nonlinear convolutional blocks and softmax output.The model detects animals but does not regress bounding boxes.
  • Ablation design: Held-out ablations train models with one of five recommendations disabled, while the Full Model enables all recommendations.Rotational augmentation is analyzed separately.
  • Inference setup: At test time, each 4000×3000 image is divided into 8×6 overlapping 512×512 patches, producing 32 × 32 prediction grids.Overlapping border areas are eventually averaged.
  • Baseline model: The baseline follows the Kuzikus state-of-the-art object-proposal pipeline, replacing its exemplar SVM with a random forest classifier.The authors found similar performance to the previously reported baseline.

4.2. Ablation studies

The ablation study finds that combining the recommended CNN training practices produces the strongest precision, especially at high recall, while class balancing, curriculum learning, border classes, and hard negative mining address different failure modes.

  • The full model combining all recommendations provides the best precision, particularly beyond 80% recall.Individual recommendations improve precision, but their effects depend on one another rather than scaling linearly.
  • Curriculum learning is important because training directly on the full imbalanced dataset performs poorly at high recall.The model trained without curriculum learning performs worst, while adding background data alone is insufficient.
  • 19 false alarms per positive example and 5% precision occur when the border class is omitted at high recall.The border class helps distinguish animal centers from surrounding locations within the CNN’s receptive field.
  • Hard negative mining raises precision from around 5% to 20% at 90% recall and above, reducing false alarms from 19 to 4 per detected animal.It complements curriculum learning by acting during the final fine-tuning stage.
  • Without class-weight balancing, the model reaches almost 100% precision but cannot exceed approximately 74% recall.The authors hypothesize substantial overfitting, making this model unsuitable for real census scenarios despite high precision at lower recall.

4.3. Rotational augmentation

Rotational augmentation has stage-dependent effects: early use provides little benefit or worsens accuracy, whereas late fine-tuning improves precision with only a slight recall cost.

  • Rotational augmentation enabled from the start or at epoch 150 produces similar or worse final accuracy than the model without rotations.The study compares schedules using augmentation from the beginning, from epoch 150, or from epoch 300.
  • Late rotational fine-tuning from epoch 300 with a lower learning rate substantially improves precision while slightly reducing achievable recall.The authors therefore identify late-stage, gentler-rate rotation as the useful augmentation schedule.

4.4. Evaluation of the distance thresholds

Distance thresholds substantially affect census-oriented evaluation: small thresholds penalize positional imprecision, while excessive thresholds admit distant or duplicate detections and reduce precision.

  • At small thresholds such as 10 pixels, both models perform weakly because their detections are not positioned precisely at ground-truth points.Imprecise ground-truth centers, shadow detections, and related annotation issues may also contribute.
  • The baseline performs best around 50 pixels, whereas the CNN has its best high-recall precision at 25 pixels.At identical recall levels, the CNN has higher absolute precision, indicating spatially closer predictions.
  • The subsequent results use a distance threshold of 50 pixels, approximately 2m.

4.5. Results on the full dataset

On the full dataset, the proposed CNN substantially reduces false alarms relative to the baseline while preserving high recall and concentrating verification on fewer relevant image tiles.

  • The full-model test evaluation uses hyperparameters selected for best validation-set performance and compares three cross-validated models.
  • 4.5.1. Animal instance-based evaluation: At 80% recall, false positives fall from over 2500 for the baseline to fewer than 450 for the full CNN model.The comparison uses the census-oriented evaluation protocol and averages performance across three cross-validation splits.
  • 4.5.1. Animal instance-based evaluation: At 90% recall, the full model produces 870 detections compared with 20’688 for the baseline.
  • 4.5.2. Tile-based evaluation: At tile-based recall of 0.9, the CNN requires verification of 190 tiles, while the baseline produces detections in 779 of 1536 tiles.The test set contains 97 tiles with ground-truth objects and 1536 total tiles.
  • 4.5.2. Tile-based evaluation: Figure 14 evaluates the number of detected tiles against tile-based recall, while Figure 15 relates the number of tiles to recall over individual animals.The CNN curve in Figure 15 slightly plateaus at high recall levels.

5. Conclusion

The paper addresses large-mammal censuses in UAV campaigns dominated by empty images, where detectors can produce many false positives. It combines CNN training recommendations with census-oriented evaluation protocols, reducing verification workload while retaining high recall.

  • CNN training on mostly empty UAV imagery requires class-weighting, curriculum learning, hard negative mining, and a border class around animals.An ablation study found every technique necessary for the best performance.
  • On an animal-free test image at 90% recall, baseline false alarms covered seven 1000 × 1000 tiles, whereas CNN false alarms covered one tile.Both models still produced false alarms, but the CNN predictions were more spatially confined.
  • The proposed evaluation protocols prioritize detection counts and the number of image tiles requiring verification over pixel-precise localization.The tile-based protocol measures suitability for a subsequent manual verification stage.
  • At 90% recall, the CNN reduced image tiles requiring verification to less than one third of the baseline.The experiment was conducted in the Namibian game reserve of Kuzikus and compared the CNN with a state-of-the-art detector.
  • The conclusion frames these methods as necessary steps toward realistic UAV censuses containing countless images without recorded wildlife.The training recommendations are described as model-agnostic and straightforward to apply to deep-learning object detectors.
Loading 1806.11368v1…