PaperlayerView exact revision on arXiv

Source-linked AI summary

A deep active learning system for species identification and counting in camera trap images

Mohammad Sadegh Norouzzadeh, Dan Morris, Sara Beery, Neel Joshi, Nebojsa Jojic, Jeff Clune

arXiv:1910.09716v1cs.LGcs.CVeess.IVstat.ML

TL;DR

Camera-trap surveys generate valuable wildlife data, but manual review is slow and projects often lack enough labeled images for transferable deep learning. The paper combines transfer learning, object detection, and active learning to accelerate animal identification and counting, matching reported accuracy with over 99.5% less manual annotation.

  • Problem

    Manual review limits large-scale camera-trap studies, while few labeled images and background overfitting hinder transferable deep-learning applications.

  • Method

    The pipeline combines transfer learning, object detection, lower-dimensional embeddings, and active learning to select informative images for labeling.

  • Results

    The method matched 90.9% species-classification accuracy using 99.5% less manually annotated training data, while counting was exact for 72.4% of images.

  • Takeaways & Limitations

    The pipeline reduces annotation effort and can support efficient information extraction from large camera-trap datasets.

  • Takeaways & Limitations

    The study uses fixed, well-known hyperparameters, only ResNet-50 embeddings and a simple two-layer classifier, and does not fine-tune the pretrained detector.

Abstract

from arXiv · show

Biodiversity conservation depends on accurate, up-to-date information about wildlife population distributions. Motion-activated cameras, also known as camera traps, are a critical tool for population surveys, as they are cheap and non-intrusive. However, extracting useful information from camera trap images is a cumbersome process: a typical camera trap survey may produce millions of images that require slow, expensive manual review. Consequently, critical information is often lost due to resource limitations, and critical conservation questions may be answered too slowly to support decision-making. Computer vision is poised to dramatically increase efficiency in image-based biodiversity surveys, and recent studies have harnessed deep learning techniques for automatic information extraction from camera trap images. However, the accuracy of results depends on the amount, quality, and diversity of the data available to train models, and the literature has focused on projects with millions of relevant, labeled training images. Many camera trap projects do not have a large set of labeled images and hence cannot benefit from existing machine learning techniques. Furthermore, even projects that do have labeled data from similar ecosystems have struggled to adopt deep learning methods because image classification models overfit to specific image backgrounds (i.e., camera locations). In this paper, we focus not on automating the labeling of camera trap images, but on accelerating this process. We combine the power of machine intelligence and human intelligence to build a scalable, fast, and accurate active learning system to minimize the manual work required to identify and count animals in camera trap images. Our proposed scheme can match the state of the art accuracy on a 3.2 million image dataset with as few as 14,100 manual labels, which means decreasing manual labeling effort by over 99.5%.

1 Introduction

Camera traps provide inexpensive, non-intrusive wildlife observations, but manual image review limits large-scale studies. Deep learning offers fast extraction, yet limited labels and background dependence constrain adoption; the proposed active-learning approach addresses these barriers.

  • Camera traps collect large image volumes for wildlife population studies, but manually reviewing every image is slow and limits large-scale surveys.These observations support modeling population sizes, distributions, and environmental interactions.
  • Deep convolutional neural networks can rapidly extract species labels, counts, and behavior from camera trap images.Recent systems can process hundreds of images in seconds, potentially reducing conservation biologists’ time and costs.
  • State-of-the-art deep networks typically require millions of labeled images, a volume unavailable to most camera trap projects.Existing labels are often only image- or sequence-level and lack animal location information.
  • 99.5% less manually annotated training data matched Norouzzadeh et al.’s 90.9% species-classification accuracy on a 3.2-million-image dataset.The proposed method combines object detection, transfer learning, and active learning, while filtering background pixels to improve prospective transferability.

2 Background and related work

The background reviews deep learning, image classification, and object detection for camera-trap analysis. It emphasizes that classification struggles with multiple species, counting, and transfer across locations, while detection offers richer localization but requires costly box labels.

  • Deep learning: Deep learning uses multilayer artificial neural networks whose trained weights transform inputs into outputs through successive nonlinear computations.Convolutional layers connect locally and learn useful visual patterns.
  • Image classification: Image classification assigns an image to predefined classes or estimates each class’s probability, such as species presence in camera-trap images.Classification can use image-level labels but does not specify animal locations.
  • Image classification: Image classification typically selects one most-probable species, so it cannot handle images containing more than one species.
  • Image classification: Classification performs worse on counting tasks and learns background information, which reduces accuracy when applied to new locations.A cited example found lower accuracy when a model trained in the United States identified the same species in Canada.
  • Object detection: Object detection outputs bounding-box coordinates and class probabilities, naturally handling multiple object classes within an image.The paper hypothesizes that explicit region focus may also reduce background sensitivity.
  • Object detection: Training object detectors requires bounding-box and class labels for each animal, which are costly and rarely available in camera-trap projects.This makes detector training impractical for many projects despite its suitability for multi-species images.

2.4 Transfer learning

Transfer learning reuses general features from related tasks when target labels are scarce, while active learning iteratively selects informative samples for human labeling. The paper combines these ideas with detection and learned embeddings to scale camera-trap analysis.

  • Transfer learning: Early network layers learn general patterns such as edges, while later layers become increasingly specific to the training dataset and task.
  • Transfer learning: Transfer learning first trains a network on a large related dataset, then fine-tunes it on a smaller target dataset with limited labels.This reuses general features to learn the target task more efficiently.
  • Active learning: Active learning alternates model training with oracle labeling, selecting informative unlabeled samples while minimizing labeling requests.The procedure repeats until a labeling budget or acceptable accuracy level is reached.
  • Embedding learning: The paper compares embeddings learned with cross-entropy loss against embeddings learned with triplet loss, which brings same-class samples closer together.

2.7 Datasets

The experiments use four camera-trap datasets spanning millions of images, multiple locations, and diverse animal categories, with varying proportions of empty images.

  • The study uses Snapshot Serengeti, eMammal, NACTI, and Caltech Camera Traps for model training and evaluation.
  • Snapshot Serengeti contains 3.2 million images in 1.2 million sequences, with labels for 48 animal categories and approximately 75% empty images.
  • eMammal provides over 450,000 images covering more than 270 species from diverse locations worldwide.
  • NACTI contains 3.7 million images from five United States locations, labeled across 28 animal categories, with approximately 12% empty images.
  • Caltech Camera Traps contains 245 thousand images from 140 Southwestern United States cameras, spanning 22 animal categories and approximately 70% empty images.

3 Methods

The pipeline combines pretrained object detection, transferable low-dimensional embeddings, and active learning to reduce background effects, support counting, and select informative labels efficiently.

  • The proposed pipeline addresses transferability, multi-species images, inaccurate counting, and limited labeled data using transfer learning and active learning.
  • A pretrained Faster-RCNN detector uses a single animal class and a 90% confidence threshold to identify detections.
  • Images without detections above 90% confidence are marked empty, while animals are counted by summing qualifying detections.
  • Detected animals are cropped to reduce background pixels, then resized to 256 × 256 pixels before classification processing.
  • The method embeds target crops into a 256-dimensional feature space using a pretrained model, enabling transfer learning and faster whole-dataset query evaluation.
  • Active learning begins with 1,000 randomly labeled images, then iteratively selects 100 unlabeled images for human annotation and model training.
  • Algorithm 2 summarizes the pipeline as detection, crop embedding, initial labeling, classification training, and iterative active selection.

4 Experiments and results

Experiments on Snapshot Serengeti evaluate detection, counting, embedding, active querying, and crop-based classification within the proposed three-stage pipeline.

  • The experiments train embeddings on eMammal and evaluate the complete pipeline on Snapshot Serengeti.
  • 4.1 Empty vs. animal: The pretrained detector achieves 91.71% accuracy, 84.47% precision, and 84.23% recall without target-dataset manual labels.
  • 4.2 Counting: The detector counts animals exactly in 72.4% of images and exactly or within one bin in 86.8% of images.
  • 4.3.1 Embedding spaces: With 25,000 labels, cross-entropy features reach 85.23% accuracy, whereas triplet-loss features reach 91.37% accuracy.
  • 4.3.1 Embedding spaces: Triplet loss imposes constraints across combinations of labeled samples, unlike cross-entropy loss, which uses each sample independently.
  • 4.3.2 Active learning strategies: All active-learning strategies outperform random selection, with k-Center reaching 92.2% accuracy at 25,000 labels and matching Norouzzadeh et al. at 14,000 labels.
  • 4.3.3 Crops versus full images: Using animal crops produces significantly better species-classification results than using full images, likely because crops remove background pixels.

5 Further improvement

The authors identify several avenues for improving the pipeline, including broader tuning, detector fine-tuning, and combining classification with detection.

  • The pipeline’s parameters and algorithmic design choices were not explored extensively, leaving multiple opportunities for improvement.
  • Hyperparameter tuning and broader architecture searches may improve results beyond the fixed ResNet-50 embedding and simple two-layer classifier.
  • Collecting bounding-box information from the oracle could enable fine-tuning of the pretrained detector alongside the embedding and classification models.
  • After enough labels are collected, combining classification and detection into one multi-class detector may improve accuracy and computational efficiency.

6 Conclusion

The proposed pipeline reduces the annotation bottleneck in camera-trap analysis by combining object detection, triplet embeddings, and active learning to reduce human effort.

  • 99.5% annotation reduction can facilitate deploying large camera-trap arrays and improve efficiency in wildlife monitoring projects.The stated scope includes wildlife biology, zoology, ecology, and animal behavior applications.
  • Object detection supports multiple species and removes background pixels, helping detectors generalize better than image-classification models to other datasets.
  • Triplet-loss embeddings outperform cross-entropy embeddings when labeled data are limited.
  • Active learning selects examples for labeling to use human expertise more efficiently and reduce the effort needed to extract information from camera-trap datasets.

S1 Triplet loss

Triplet loss trains an embedding space by bringing same-label samples together and separating different-label samples, while excluding already-satisfied examples from optimization.

  • Triplet loss arranges labeled examples into triplets so same-label samples are nearby and different-label samples are distant in embedding space.
  • Easy triplets have zero loss, whereas semi-hard and hard triplets violate the desired margin relationship to different degrees.The paper omits easy triplets when arranging training triplets.
  • The margin hyperparameter specifies the minimum acceptable difference between d(A, P) and d(A, N).
  • This paper uses randomly selected semi-hard negatives when forming triplets.

S2 Active learning selection criteria

The paper compares uncertainty- and density-based active-learning criteria, finding benefits from triplet features, k-Center selection, and object crops in the Snapshot Serengeti experiments.

  • Uncertainty-based selection: Uncertainty methods select samples whose model outputs indicate low confidence or a small gap between the two most likely classes.
  • Density-based selection: Density-based methods query representative samples from dense input-space regions, making them more resilient to outliers.
  • Embedding comparison: Triplet-loss features produce better active-learning accuracy than softmax cross-entropy features, although the gap closes as queries increase.
  • Strategy comparison: k-Center achieves the best accuracy at 30,000 queries when using triplet-loss features on Snapshot Serengeti.
  • Crop versus full-image inputs: Using object crops instead of full images substantially increases accuracy for k-Center active learning with triplet-loss features.
Loading 1910.09716v1…