PaperlayerView exact revision on arXiv

Source-linked AI summary

AP-10K: A Benchmark for Animal Pose Estimation in the Wild

Hang Yu, Yufei Xu, Jing Zhang, Wei Zhao, Ziyu Guan, Dacheng Tao

arXiv:2108.12617v2cs.CV

TL;DR

Animal pose estimation lacks broadly diverse benchmarks despite its relevance to behavior understanding and wildlife conservation. The paper introduces AP-10K, a taxonomically organized mammal dataset and benchmark spanning supervised learning, cross-domain transfer, and domain generalization. Results provide empirical evidence that learning from diverse animal species benefits accuracy and generalization, while the dataset also exposes scope and long-tail challenges for future work.

  • Problem

    Existing animal pose studies and datasets cover limited species diversity, making generalization and transfer questions difficult to evaluate comprehensively.

  • Method

    The paper constructs AP-10K with 10,015 labeled images from 23 families and 54 species, then benchmarks pose models across supervised, cross-domain transfer, and domain-generalization tracks.

  • Results

    Experiments provide empirical evidence that learning from diverse animal species improves pose-estimation accuracy and generalization ability.

  • Takeaways & Limitations

    AP-10K supports research on diverse-animal pose estimation and enables further study of semi-supervised, self-supervised, and few-shot learning.

  • Takeaways & Limitations

    AP-10K covers far fewer species than exist in the real world, so model generalization on it requires careful attention.

Abstract

from arXiv · show

Accurate animal pose estimation is an essential step towards understanding animal behavior, and can potentially benefit many downstream applications, such as wildlife conservation. Previous works only focus on specific animals while ignoring the diversity of animal species, limiting the generalization ability. In this paper, we propose AP-10K, the first large-scale benchmark for mammal animal pose estimation, to facilitate the research in animal pose estimation. AP-10K consists of 10,015 images collected and filtered from 23 animal families and 54 species following the taxonomic rank and high-quality keypoint annotations labeled and checked manually. Based on AP-10K, we benchmark representative pose estimation models on the following three tracks: (1) supervised learning for animal pose estimation, (2) cross-domain transfer learning from human pose estimation to animal pose estimation, and (3) intra- and inter-family domain generalization for unseen animals. The experimental results provide sound empirical evidence on the superiority of learning from diverse animals species in terms of both accuracy and generalization ability. It opens new directions for facilitating future research in animal pose estimation. AP-10k is publicly available at https://github.com/AlexTheBad/AP10K.

1 Introduction

Animal pose estimation matters for behavior understanding and applications such as wildlife conservation, but existing datasets and studies have limited species diversity and generalization. AP-10K addresses this gap with a taxonomically organized mammal benchmark and evaluations spanning supervision, transfer, and generalization.

  • Animal pose estimation identifies the category and location of body keypoints and supports activity recognition, behavior understanding, and human-object interaction.
  • Existing animal pose studies focus on specific species, while prior multi-species datasets include few species and lack family-species taxonomic structure.
  • 10,015 images from 23 animal families and 54 species form AP-10K, with manually labeled and double-checked keypoints organized by taxonomic rank.
  • AP-10K benchmarks supervised learning, cross-domain transfer from human pose estimation, and intra- and inter-family domain generalization for unseen animals.
  • The reported results provide empirical evidence that learning from diverse animal species improves pose-estimation accuracy and generalization ability.

2 Related work

Human pose estimation has benefited from large, diverse benchmarks and strong bottom-up and top-down methods. Animal pose estimation remains constrained by species-specific datasets, while the effectiveness of transfer learning with large animal datasets remains unclear.

  • 2.1 Human pose estimation: Human pose methods are commonly categorized as bottom-up or top-down, with top-down methods generally achieving higher benchmark accuracy.
  • 2.1 Human pose estimation: Large human pose datasets such as COCO and MPII provide diversity in posture, illumination, scale, occlusion, and person count.
  • 2.2 Animal pose estimation: Animal pose datasets often target individual species and remain limited in posture, texture, habitat, or taxonomic diversity.
  • 2.2 Animal pose estimation: The five-species Animal Pose Dataset lacks family-species organization, preventing study of intra- and inter-family generalization.
  • 2.3 Transfer learning: Transfer learning typically uses pretraining followed by finetuning, but its effectiveness from ImageNet or human pose datasets remains unclear when large animal data are available.

3 Dataset

AP-10K combines broad taxonomic coverage with manually checked animal-pose annotations, including labeled and unlabeled images organized by family and species. Its long-tail distribution reflects animal frequency in the wild and creates a challenging benchmark for pose estimation.

  • Data collection: AP-10K collects 59,658 candidate animal images from public datasets, removes duplicates and mislabeled images, and retains high-quality species labels.Duplicate removal uses aHash followed by manual double-checking.
  • Data organization: The dataset reorganizes images by family and species, using biological similarity in appearance and pose distributions as a prior for evaluation.The organization supports model generalization evaluation across related and unseen animal groups.
  • Data annotation: Thirteen annotators labeled 10,015 images with 17 animal keypoints, followed by three rounds of cross-checking and correction.The annotation process took about three months.
  • Dataset statistics: AP-10K covers 23 animal families and 54 species, with 10,015 labeled images and 13,028 instances across complex animal-pose distributions.The dataset is described as richer in species diversity and larger in labeled content than previous animal-pose datasets.
  • Dataset statistics: An additional 50k images have family and species labels without keypoint annotations, enabling semi-supervised and self-supervised animal-pose research.The taxonomic organization makes these unlabeled images usable for such research.
  • Dataset statistics: Family image counts follow a long-tail distribution, ranging from 1,913 labeled images for Felidae to 200 for Procyonidae.This imbalance reflects animal commonness or rarity in the wild and also supports few-shot learning studies.

4 Experiment

The experiments evaluate representative pose models across supervised learning, cross-domain transfer, and intra- and inter-family generalization. Results favor diverse-species training, while transfer effectiveness and generalization depend on training duration and biological relatedness.

  • The study benchmarks pose estimation models across supervised learning, cross-domain transfer from humans, and intra- and inter-family domain generalization.
  • Supervised learning track: HRNet generally outperforms SimpleBaseline and Hourglass, while ImageNet pretraining improves performance across models.The authors select HRNet-w32 as the default model by trading off model complexity and performance.
  • Supervised learning track: Longer training reduces the performance gap between ImageNet pretraining and training from scratch.The authors attribute pretraining’s benefit primarily to faster convergence from a better initialization.
  • Cross-domain transfer learning track: 0.753 mAP after 210 epochs exceeds 0.738 mAP with ImageNet pretraining, whereas human-pose transfer performs worse after 20, 30, or 40 epochs.The reported short-schedule results are 0.606, 0.642, and 0.667 mAP, respectively.
  • Intra-family domain generalization: Models retain good performance on unseen species within the same family, but performance is lower than on seen species and can vary with appearance diversity.Dog and Cat are highlighted as difficult cases because of wide appearance variation associated with human cultivation.
  • Inter-family domain generalization: Bovidae-trained models generalize best to Cervidae, decline on Equidae, and generalize poorly to Hominidae, consistent with biological relatedness.Cervidae shares the Artiodactyla order with Bovidae, whereas Hominidae belongs to the Primates clade.
  • Inter-family transfer and few-shot learning: With only 20 finetuning images, performance on other species increases quickly, especially for Zebra, whose poses resemble Bovidae despite different textures.
  • Cross-animal pose dataset evaluation: AP-10K-trained models generalize well to the Animal Pose dataset, while Animal Pose-trained models do not generalize conversely.The comparison evaluates mAP on the 17 keypoints shared by both datasets.

5 Discussion

The discussion reports empirical support for AP-10K’s value and identifies remaining challenges in transfer, domain generalization, and rare-species learning. It also outlines future uses for unlabeled images and few-shot learning.

  • Results across the SL, CD-TL, and DG tracks provide empirical evidence supporting the value of AP-10K.
  • Future work should improve cross-domain transfer efficiency, address domain gaps across families and species, and handle rare species in the long-tail distribution.
  • About 50k animal images with family and species labels but no keypoint annotations could support semi-supervised and self-supervised pose estimation.
  • AP-10K is the largest dataset in this area but is 10× smaller than human pose datasets such as COCO.

6 Conclusion

AP-10K provides a diverse, taxonomically organized benchmark for mammal pose estimation and supports analysis of model representation and generalization. Its real-world coverage remains incomplete, requiring caution when interpreting model generalization.

  • AP-10K is a large-scale mammal pose estimation dataset organized by taxonomic rank, with diversity in posture, scale, occlusion, and species.The dataset is presented as enabling new research questions in animal pose estimation.
  • The benchmark examines model representation, pretraining effects, diverse-species training, and intra- and inter-family generalization.These analyses use representative pose estimation methods evaluated on AP-10K.
  • AP-10K can potentially benefit animal behavior understanding, zoology, and wildlife conservation.
  • AP-10K covers much fewer species than exist in the real world, so model generalization on it requires careful attention.

A.1 More quantitative and qualitative results

The appendix reports expanded supervised-learning evaluations across models, species, and training settings. These results indicate benefits from advanced architectures, ImageNet pretraining, and training on more diverse species.

  • Table S1 reports complete validation-set results for multiple representative models on the supervised-learning track.The evaluated models include HRNet variants, SimpleBaseline with ResNet backbones, and Hourglass.
  • HRNet outperforms other backbones on most categories across evaluation metrics, while ImageNet pretraining accelerates convergence and improves performance over training from scratch.
  • Per-species test results compare HRNet-W32 with ImageNet pretraining, training from scratch, and an expanded 56-animal test set.
  • 0.761 v.s. 0.663 mAP for Sheep illustrates that more diverse species can improve feature representation and performance with the same training-data amount.The 0.761 score comes from training only on the Bovidae family, whereas 0.663 is the comparison reported for more diverse-species training.

A.2 Motivation

AP-10K was created to facilitate animal pose estimation research by enabling study of model performance, diverse-species training, and pretraining effects. The supplied motivation materials frame these as open questions for the dataset.

  • AP-10K was created to facilitate research in animal pose estimation.
  • The dataset targets questions about representative human pose models, large-scale diverse-species training, and pretraining effects.The motivation identifies these as challenging questions to study when more diverse animal training data are available.
  • Existing datasets and the associated questionnaire materials are included alongside the motivation, but the supplied passages provide no additional substantive rationale.

A.3 Composition

AP-10K consists of animal images categorized by family and species, with 17 keypoint annotations for each animal instance. Its composition therefore combines taxonomic organization with structured pose labels.

  • The composition description concerns animal-image instances and their keypoint annotations rather than multiple interacting instance types.
  • AP-10K comprises images covering 54 animal species categorized according to taxonomic rank, specifically family and species.
  • Each animal instance has 17 annotated keypoints covering facial, torso, front-limb, hind-limb, and tail locations.

2. How many instances are there in total (of each type, if appropriate)?

AP-10K contains 10,015 images and 13,028 annotated animal instances, plus 49,643 additional images with family and species labels but no keypoint annotations. It is a real-world sample spanning 23 families and 54 species, with incomplete keypoint information possible for some instances because of occlusion, blur, or small scale.

  • 10,015 images and 13,028 instances have keypoint annotations in AP-10K.
  • AP-10K covers 23 typical animal families and 54 species as a real-world sample, not all wild-animal instances.
  • The dataset includes each animal’s bounding box, family and species labels, keypoint annotations, and unprocessed image data.
  • COCO-style targets include instance and image identifiers, category information, box area, crowd status, keypoint counts, and detailed keypoint locations and categories.
  • Some instances lack complete keypoint annotations because of occlusion, blur, or small scale; affected locations and visibility are marked zero.

A.4 Collection Process

AP-10K was assembled from publicly available animal datasets, cleaned and categorized, then manually annotated and cross-checked before public release. The dataset is intended for animal pose estimation and related machine-learning research, but annotations may contain residual noise.

  • Data acquisition: AP-10K images were obtained from publicly available datasets and downloaded from their websites.
  • Cleaning and labeling: Replicated images were detected with aHash and manual checking, while heavily occluded images and logos were removed manually.
  • Cleaning and labeling: Images were categorized by family and species, with annotator double-checking used to ensure dataset image quality.
  • Annotation: Three rounds of manual and automatic cross-checking and correction were conducted to reduce mislabeling.
  • Uses and maintenance: AP-10K can support animal pose estimation, few-shot learning, domain generalization, and self-supervised learning research.
  • Uses and maintenance: The dataset will be publicly available through GitHub and distributed under the MIT license.
  • Uses and maintenance: The dataset will not be updated; a very few labeling errors may remain and can be treated as noise.
Loading 2108.12617v2…