PaperlayerView exact revision on arXiv

Source-linked AI summary

The iNaturalist Species Classification and Detection Dataset

Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, Serge Belongie

arXiv:1707.06642v2cs.CV

TL;DR

Natural-world species recognition requires handling visually similar categories with highly imbalanced data, a setting underrepresented by conventional benchmarks. The paper introduces and evaluates iNat2017, a large citizen-science dataset for classification and detection. Baselines achieve 67.74% single-crop top-1 accuracy with Inception ResNetV2 SE, while detection reaches 43.5 comprehensive AP, demonstrating substantial difficulty.

  • Problem

    Existing benchmarks often use approximately uniform class distributions, while real-world species data are imbalanced and visually challenging.

    Source: §1 · p. 1

  • Method

    The paper constructs iNat2017 from citizen-science observations and evaluates state-of-the-art classification and detection models on its fine-grained species data.

    Source: §1 · p. 2, §4.1 · p. 5, §4.2 · p. 6

  • Results

    Inception ResNetV2 SE reaches 67.74% single-crop top-1 accuracy on iNat2017, while Faster-RCNN achieves 43.5 comprehensive AP on detection.

    Source: §4.1 · p. 5, §4.2 · p. 6

  • Takeaways & Limitations

    iNat2017 provides an experimental platform for studying imbalanced, fine-grained recognition under real-world conditions.

    Source: §1 · p. 2, §5 · p. 7

    Bounding boxes were not collected for Plantae, Fungi, Protozoa, or Chromista because individual instances are difficult to box.

    Source: §3.3 · p. 4

Abstract

from arXiv · show

Existing image classification datasets used in computer vision tend to have a uniform distribution of images across object categories. In contrast, the natural world is heavily imbalanced, as some species are more abundant and easier to photograph than others. To encourage further progress in challenging real world conditions we present the iNaturalist species classification and detection dataset, consisting of 859,000 images from over 5,000 different species of plants and animals. It features visually similar species, captured in a wide variety of situations, from all over the world. Images were collected with different camera types, have varying image quality, feature a large class imbalance, and have been verified by multiple citizen scientists. We discuss the collection of the dataset and present extensive baseline experiments using state-of-the-art computer vision classification and detection models. Results show that current non-ensemble based methods achieve only 67% top one classification accuracy, illustrating the difficulty of the dataset. Specifically, we observe poor results for classes with small numbers of training examples suggesting more attention is needed in low-shot learning.

1. Introduction

iNat2017 addresses the need for visual biodiversity monitoring by introducing a large, imbalanced dataset of fine-grained species images collected and verified through citizen science.

1. Introduction

  • Existing classification benchmarks are nearing saturation, whereas reducing training images per category typically harms performance and makes class imbalance a practical challenge.The paper argues that acquiring more data for rare classes is often impractical or impossible.Source: §1 · p. 1
  • Several million species exist, and visual identification can be difficult because many categories are visually similar, motivating automated tools for global biodiversity monitoring.The paper notes that approximately 1.2 million species have been formally described.Source: §1 · p. 1, Figure 1 · p. 1
  • iNaturalist observations combine dates, locations, images, and species labels, and species identities may be confirmed by multiple annotators.By November 2017, the site had collected over 6.6 million observations from 127,000 species.Source: §1 · pp. 1–2
  • iNat2017 targets robust classification and detection across more than 5,000 imbalanced species using 675,000 training/validation images, 183,000 test images, and over 560,000 bounding boxes.Its images span visually similar species, varied situations, and locations worldwide, with labels verified by multiple citizen scientists.Source: §1 · pp. 1–2

2. Related Datasets

Prior fine-grained datasets cover many domains but often use roughly uniform class distributions and web-sourced images, unlike the real-world variation and consensus annotations emphasized by iNat2017.

2. Related Datasets

  • Fine-grained recognition requires distinguishing many potentially similar categories, often with few training examples and limited domain expertise.The related-work discussion identifies smaller class sizes and fewer capable experts as characteristic of finer-grained classification.Source: §2 · p. 2
  • Existing datasets span birds, dogs, airplanes, flowers, leaves, food, trees, and cars, with ImageNet also containing groups of fine-grained classes.The cited examples demonstrate broad domain coverage but do not by themselves represent natural-world species diversity.Source: §2 · p. 2
  • Many datasets were designed to have an approximately uniform distribution of images across their different categories.Table 1 defines imbalance as the number of images in the largest class divided by the number in the smallest.Source: §2 · p. 2, Table 1 · p. 2
  • Web-crawled datasets can contain incorrect images, while iNat2017 uses consensus annotations from informed enthusiasts and captures natural variation caused by species, age, sex, and environment.Clean validation and test sets remain important for quantifying performance even when models tolerate some training-time label noise.Source: §2 · p. 2

3. Dataset Overview

iNat2017 is a citizen-science biodiversity dataset designed for imbalanced, fine-grained classification and detection, with observer-partitioned splits and extensive bounding-box annotations. Its construction yields verified image splits across 5,089 taxa and annotations concentrated on animal super-classes.

3.1. Dataset Collection

  • iNat2017 uses citizen-science observations and selects 5,089 taxa with at least 20 observations from 20 unique observers.The archive was processed from research-grade labels collected through iNaturalist and GBIF.Source: §3.2 · p. 3, §3.1 · p. 3

3.2. Dataset Construction

  • The training distribution is strongly imbalanced: the top 1% of species contain over 16% of training images.Each class has at least 8 training images and generally 4–30 validation images, but the resulting class frequencies remain unequal.Source: Figure 2 · p. 3, §3.2 · p. 4
  • Observer-based partitioning produces 579,184 training, 95,986 validation, and 182,707 test images while limiting user-specific cues from crossing splits.Observers are assigned to splits before their photographs, with test observers selected to increase variability.Source: §3.2 · pp. 3–4

3.2.1 Test Set Verification

  • Test-set verification found fewer than 1.1% non-instance images for Aves and Reptilia, while Mammalia images containing footprints or feces were filtered from testing.Training and validation images were not filtered.Source: §3.2.1 · p. 4

3.3. Bounding Box Annotation

  • Bounding boxes were omitted for Plantae, Fungi, Protozoa, and Chromista because individual instances are difficult to delineate in these super-classes.The paper suggests specialized pixel annotations as a potentially more appropriate alternative.Source: §3.3 · p. 4
  • Bounding boxes cover 2,854 classes across nine super-classes, yielding 561,767 boxes from 449,313 training and validation images.Annotations target super-classes rather than fine-grained taxa, allow up to 10 instances per image, and classify 5.7% of instances as small, 23.6% medium, and 70.7% large by COCO conventions.Source: §3.3 · p. 4

4. Experiments

Experiments show that iNat2017 remains difficult for state-of-the-art classification and detection models, with performance varying substantially across fine-grained classes and super-classes. Detection also degrades when fine-grained labels must support coarser recognition and localization.

4.1. Classification Results

  • 67.74% top-1 and 87.89% top-5 accuracy were achieved by Inception ResNetV2 SE on iNat2017, versus 80.2% and 95.21% on ILSVRC 2012.Inception ResNetV2 SE was the best-performing classification model, but iNat2017 was comparatively harder.Source: §4.1 · p. 5, Table 3 · p. 5
  • Classification difficulty varied sharply across super-classes: Reptilia reached 45.87% top-1 accuracy, whereas Protozoa reached 89.19%.These values are averages across species within each super-class.Source: §4.1 · p. 5, Table 4 · p. 5
  • Accuracy generally increased with training images per class, but substantial variance remained among classes with similar data amounts.The result indicates opportunities for improvement in both low-data and high-data regimes.Source: §4.1 · p. 5, Figure 5 · p. 5

4.2. Detection Results

  • Faster-RCNN achieved 43.5 comprehensive average precision across the 2,854-class detection subset, with AP ranging from 21.3 for Reptilia to 49.4 for Insecta.The corresponding AUC values were 0.315 for Reptilia and 0.677 for Insecta.Source: §4.2 · p. 6, Table 5 · p. 6
  • Detection performance was highly variable across super-classes, challenging detectors to work across broad groups of fine-grained classes.Evaluation used validation images containing a single instance from the 2,854 classes with bounding boxes.Source: §4.2 · p. 6, Figure 6 · p. 6
  • Training with finer-grained labels reduced performance at coarser evaluation granularities, especially for the 2,854-class detector.The findings identify room for end-to-end fine-grained detection methods that preserve precision across label granularities.Source: §4.2 · p. 6, Table 6 · p. 6, Figure 7 · p. 7

5. Conclusions and Future Work

The paper presents iNat2017 as a realistic, expanding benchmark for long-tailed fine-grained recognition, with value for conservation and field biology. Its sparse-data challenges are expected to become more relevant as the community adds species, while future work targets richer annotations and open-world evaluation.

5. Conclusions and Future Work

  • iNat2017 is presented as an unbiased, representative, long-tailed dataset useful for conservation and field biology.Its images were collected for a defined purpose by non-computer-vision researchers rather than through typical researcher-driven selection.Source: §5 · p. 7
  • The dataset can grow with the iNaturalist community, with another species reaching the 20-observer threshold approximately every 1.7 hours.Species available for inclusion had already increased from 5,000 to 12,000 by November 2017.Source: §5 · p. 7
  • As the dataset expands, its long-tail and sparse-data challenges are expected to become more relevant.The paper links this trend to the continuing addition of species with sufficient observer coverage.Source: §5 · p. 7
  • Future work includes adding sex, life-stage, habitat, and pixel-level annotations, especially for four difficult-to-annotate super-classes.The authors also plan to study open-world testing, taxonomic-rank-aware error measures, and models of human learning.Source: §5 · p. 7

Related papers

Loading 1707.06642v2…