PaperlayerView exact revision on arXiv

Source-linked AI summary

Cross-Domain Adaptation for Animal Pose Estimation

Jinkun Cao, Hongyang Tang, Hao-Shu Fang, Xiaoyong Shen, Cewu Lu, Yu-Wing Tai

arXiv:1908.05806v2cs.CV

TL;DR

Animal pose estimation lacks broad labeled coverage, making training across diverse species difficult. The paper builds a five-class animal pose dataset and transfers knowledge from human and labeled-animal data to unlabeled species using WS-CDA and PPLO, achieving 65.7 mAP on the test set and 50+ mAP on unseen classes without pose labels.

  • Problem

    Animal pose estimation lacks well-labeled datasets, and labeling all animal species is impractical despite their varied poses and appearances.

  • Method

    The paper builds a five-class animal pose dataset and uses WS-CDA with human, labeled-animal, and box-labeled unseen-animal data, followed by PPLO pseudo-label optimization.

  • Results

    65.7 mAP is achieved on the test set, while unseen animal classes achieve 50+ mAP without pose-labeled data for those classes.

  • Takeaways & Limitations

    Human and animal pose priors can be leveraged to estimate poses for unseen animal classes through cross-domain adaptation.

  • Takeaways & Limitations

    Experiments restrict data sources to COCO2017 human data and the built animal dataset, with pose-unlabeled animal samples drawn from COCO2017 detection data.

Abstract

from arXiv · show

In this paper, we are interested in pose estimation of animals. Animals usually exhibit a wide range of variations on poses and there is no available animal pose dataset for training and testing. To address this problem, we build an animal pose dataset to facilitate training and evaluation. Considering the heavy labor needed to label dataset and it is impossible to label data for all concerned animal species, we, therefore, proposed a novel cross-domain adaptation method to transform the animal pose knowledge from labeled animal classes to unlabeled animal classes. We use the modest animal pose dataset to adapt learned knowledge to multiple animals species. Moreover, humans also share skeleton similarities with some animals (especially four-footed mammals). Therefore, the easily available human pose dataset, which is of a much larger scale than our labeled animal dataset, provides important prior knowledge to boost up the performance on animal pose estimation. Experiments show that our proposed method leverages these pieces of prior knowledge well and achieves convincing results on animal pose estimation.

1. Introduction

The paper addresses animal pose estimation despite scarce labeled data by combining human and animal priors through cross-domain adaptation and pseudo-label optimization. It introduces an animal pose dataset and reports strong performance on both seen and unseen animal classes.

  • Motivation: Animal pose estimation is difficult because well-labeled datasets are scarce, while labeling all animal species is impractical.The paper identifies pose variation and the diversity of animal appearances as central data challenges.
  • Motivation: Human–animal pose similarity, animal appearance data, and pose knowledge shared across anatomically similar species provide complementary priors.These priors target four-legged mammals and support transfer to animal classes without annotations.
  • Method: WS-CDA combines pose-labeled human data, box-labeled animal data, and a small pose-labeled animal dataset using adversarial feature adaptation.Its feature extractor, domain discriminator, and keypoint estimator learn cross-domain common features.
  • Method: PPLO progressively uses confidence-selected pseudo-labels from unseen species and alternating training to incorporate target-domain data with reduced noise.The method exploits shared limb proportions and recurring gestures across animal classes.
  • Dataset: The Animal-Pose dataset contains five four-legged mammal classes—dog, cat, horse, sheep, and cow—with annotations aligned to a popular human pose format.The dataset is intended for model training and evaluation and for fusing human and animal pose knowledge.
  • Results: 65.7 mAP is achieved on the test set with limited pose-labeled animal data, while unseen animal classes reach 50+ mAP without pose labels.The reported results are close to state-of-the-art human pose accuracy and demonstrate cross-domain animal pose estimation.

2. Related Work

Related work establishes animal pose estimation as a sparsely studied problem shaped by annotation scarcity and severe domain shift. Existing adaptation and image-transfer approaches motivate a method specialized for the greater pose variance across animal species.

  • Animal Pose Estimation: Animal skeleton detection is rarely studied, and manually labeling pose data across diverse animal classes is labor-intensive and unrealistic.Prior animal work discussed here includes face landmarks, but not large-scale animal skeleton datasets.
  • Domain Adaptation: Large-scale supervised learning creates data demands, motivating techniques that exploit shared feature distributions across related datasets.Domain adaptation has been applied to detection, classification, and segmentation.
  • Domain Adaptation: Domain adaptation becomes difficult under severe domain shift when no extra information is available to align representations.Prior approaches use weight-shared modules or feature alignment to extract more common features.
  • Limitations of Existing Approaches: Animal pose adaptation is harder than detection or classification because pose variance across animals exceeds texture or style differences.The paper therefore proposes a method that improves established adaptation ideas for this task.

3. Preliminaries

The paper constructs a five-class animal pose dataset and formulates estimation on unlabeled animal species as cross-domain adaptation from labeled human and animal domains. It characterizes domain shift through skeleton configuration and defines the target as uniform keypoint prediction.

  • Animal Pose Dataset: The dataset extends prior annotations to 5,517 instances across dog, cat, horse, sheep, and cow, using a human-keypoint-compatible format.It provides pose-labeled animal data for training and evaluation with limited additional labor.
  • Animal Pose Dataset: Animal pose domain shift is primarily associated with skeleton configuration rather than texture, and 18 defined bones quantify relative length proportions across classes.Some animal classes have much smaller skeleton discrepancies from one another than from humans.
  • Problem Statement: The task uses large-scale human pose data and a handful of labeled animal samples to estimate poses for unseen animals through domain adaptation.The target is pose estimation on unlabeled animal samples with knowledge transferred from pose-labeled domains.
  • Problem Statement: A pose-labeled dataset contains one human domain and m animal species, with the human dataset substantially larger than the animal datasets.The formulation distinguishes labeled source instances from unlabeled target animal samples.
  • Problem Statement: Each labeled instance has ordered keypoint coordinates in R^d×2, while the model predicts corresponding latent keypoints for unlabeled animal images.The predicted pose uses a uniform format across source and target domains.

4. Proposed Architecture

The proposed architecture combines weakly and semi-supervised cross-domain adaptation with progressive pseudo-label optimization to transfer pose knowledge across human, labeled-animal, and unseen-animal domains.

  • WS-CDA: WS-CDA addresses data imbalance and domain shift by learning shared features from large pose-labeled human data, smaller pose-labeled animal data, and unseen-class animal samples.The architecture uses semi-supervision because only some animal samples have pose annotations, and weak supervision because many animal samples have only bounding-box labels.
  • WS-CDA: The WS-CDA network contains a feature extractor, domain discriminator, keypoint estimator, and domain adaptation network.The discriminator distinguishes input domains, while the estimator learns pose from labeled samples and the adaptation network improves feature representations for animal pose estimation.
  • WS-CDA: The pose loss weights animal and human supervision to compensate for the much larger number of pose-labeled human samples.Without w2 > 1, the model tends to behave almost as if it were trained only on human samples.
  • WS-CDA: Adversarial domain-discrimination and pose-estimation losses encourage domain confusion while preserving pose-learning capability.The domain discrimination loss uses cross-entropy, and human and animal pose-labeled samples jointly provide HPEL and APEL supervision.
  • Overall framework: Figure 3 depicts feature flows through WS-CDA, while Figure 4 shows the combined WS-CDA and PPLO process.The pipeline uses DDL for domain discrimination and APEL/HPEL for animal/human pose estimation.
  • PPLO: PPLO adapts the model to unseen animal classes by selecting high-confidence predictions as pseudo-labels and progressively incorporating them during training.The method uses self-paced selection, relaxes the confidence threshold during optimization, and alternates source-domain and target-domain updates to reduce pseudo-label noise and data-imbalance effects.

5. Evaluation

The evaluation compares pose-estimation and domain-adaptation settings under controlled data sources, then tests WS-CDA on unseen animal classes. Results support human priors, adaptation, and PPLO, while GAN-based augmentation performs poorly because transformations can invalidate joint labels.

  • 5.1. Experiment Settings: Experiments restrict training sources to full COCO2017 human data, the built pose-labeled animal dataset, and pose-unlabeled animal samples from COCO2017 detection data.The human training set contains 100k+ instances, substantially more than the built animal pose dataset.
  • 5.2. Evaluation for WS-CDA: Human-only training fails on animal tests, whereas adding a small number of pose-labeled animal samples produces a large performance increase.Combining human and animal samples improves performance further, indicating shared but domain-shifted pose features.
  • 5.2. Evaluation for WS-CDA: The domain adaptation network and weighting factor reduce the negative effect of the dataset-size gap, while pose-unlabeled animal data helps only when weighting is enabled.Without the weighting factor, unlabeled animal samples may instead degrade the model.
  • 5.2. Evaluation for WS-CDA: The WS-CDA evaluation varies human data, domain adaptation, pose-unlabeled animal data, and weighting settings, using mAP and the number of labeled animal instances as key indicators.Table 1 defines H, DAN, UA, NA, and the weighting choice for RB.
  • 5.3. Evaluation on unseen species: For unseen-class evaluation, one of five animal classes is held out for testing while the other four provide training data, with comparisons against fine-tuning and other adaptation methods.The experiments also compare different pose-estimation frameworks and domain-adaptation techniques using mAP.
  • 5.3. Evaluation on unseen species: Human prior knowledge, WS-CDA, and PPLO improve unseen-animal pose estimation, and the proposed method outperforms the compared domain-adaptation techniques.GAN-based methods perform poorly because transformations can change joint locations while preserving segmentation masks, invalidating original pose labels and introducing label noise.

6. Conclusion

The paper introduces unseen-animal pose estimation with domain adaptation, combining WS-CDA and PPLO to transfer knowledge from human and labeled-animal data. It also provides an animal pose dataset and reports human-level pose-estimation accuracy on animals.

  • 6. Conclusion: The paper formulates pose estimation on unseen animals as a new domain-adaptation task.The task targets animal classes without pose-labeled training data.
  • 6. Conclusion: WS-CDA transfers pose knowledge from human and animal data, while PPLO progressively incorporates target-domain pseudo-labels through self-paced selection and alternating training.The dataset is designed to provide prior knowledge for related future tasks.
  • 6. Conclusion: Experiments support the proposed scheme and report human-level pose-estimation accuracy on animal pose estimation.The conclusion presents this as the overall outcome of the proposed approach.

A. Proposed Dataset

The proposed dataset adds bounding-box-labeled examples from seven novel animal categories to support domain adaptation experiments beyond the five pose-labeled mammals. It also defines 18 pre-defined bones for measuring keypoint domain shift.

  • A. Proposed Dataset: The dataset provides bounding-box-labeled data for otter, antelope, bear, chimpanzee, rhino, bobcat, and hippopotamus.These categories are intended to support domain adaptation to novel animal classes.
  • A. Proposed Dataset: The paper uses 18 pre-defined bones to evaluate domain shift between animal and human keypoints.Their definitions are presented in Figure 7.

B. Failure Cases of Proposed Methods

Failure cases arise from unusual appearance, low contrast, and unusual gestures. These conditions can make specific keypoints unrecognizable or disrupt global pose estimation.

  • B. Failure Cases of Proposed Methods: Unusual appearance can obscure keypoints, including withers hidden by a hedgehog’s spine or a dog’s clothes and face keypoints confused by a rhino’s horn.The examples show failures tied to distinctive animal-specific visual structures.
  • B. Failure Cases of Proposed Methods: Low contrast and unusual gestures create global-feature difficulties that also lead to estimation errors.The paper illustrates these problems with representative failure cases.

C. How ground truth helps model performance

Adding ground-truth annotations from the target domain substantially improves animal pose estimation, but the benefit varies across animal categories. This shows that domain shift remains a major performance constraint.

  • 200 labeled instances per target category were added to training to estimate an accuracy upper bound.
  • Introducing target-domain ground truth greatly boosts model performance compared with unsupervised domain adaptation.
  • The proposed methods improve unsupervised animal pose estimation, yet domain shift between animal categories still harms performance substantially.
  • The supervised–unsupervised performance gap varies considerably across categories, likely because domain shifts differ between domains.

D. Failure of GANs for Data Augmentation

GAN-based data augmentation was ineffective for this animal pose adaptation task because large appearance changes destabilize transfer and threaten label validity across animal classes.

  • GAN augmentation struggled when the target image differed substantially from the original image.The paper contrasts successful style transfer, such as white dogs to black dogs, with unstable transfer from dogs to cats.
  • CycleGAN transferred between visually similar dog domains more successfully than between dogs and cats.This comparison is presented as an example of the instability encountered when transferring across substantially different animal categories.
  • Augmentation must transfer pose-labeled samples to unseen animal classes while preserving the original pose labels.The task therefore requires both category transfer and label preservation, not merely visual style transfer.
Loading 1908.05806v2…