PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Physical Connections: Tree Models in Human Pose Estimation

Fang Wang, Yi Li

arXiv:1305.2269v1cs.CV

TL;DR

Human pose estimation commonly treated simple trees as insufficient for large appearance and pose variation, especially when combined parts are used. The paper learns tree models directly from observed single and combined parts, using visual categories for combined parts, and finds no latent variables are needed on LSP. The resulting method outperformed state of the art on human and animal pose-estimation evaluations, including cross-dataset testing.

  • Problem

    The paper asks whether simple tree models can adequately represent pose distributions and how to combine single and combined parts without sacrificing exact inference.

  • Method

    The method learns a tree directly from observed variables, combines single and combined parts, and uses visual categories to model combined-part appearance.

  • Results

    The method outperformed state of the art in human and animal pose estimation, including LSP evaluation and cross-dataset testing from PARSE to LSP.

  • Takeaways & Limitations

    On LSP, no latent variables were introduced, supporting a mixed single-and-combined-part representation within a simple tree model.

Abstract

from arXiv · show

Simple tree models for articulated objects prevails in the last decade. However, it is also believed that these simple tree models are not capable of capturing large variations in many scenarios, such as human pose estimation. This paper attempts to address three questions: 1) are simple tree models sufficient? more specifically, 2) how to use tree models effectively in human pose estimation? and 3) how shall we use combined parts together with single parts efficiently? Assuming we have a set of single parts and combined parts, and the goal is to estimate a joint distribution of their locations. We surprisingly find that no latent variables are introduced in the Leeds Sport Dataset (LSP) during learning latent trees for deformable model, which aims at approximating the joint distributions of body part locations using minimal tree structure. This suggests one can straightforwardly use a mixed representation of single and combined parts to approximate their joint distribution in a simple tree model. As such, one only needs to build Visual Categories of the combined parts, and then perform inference on the learned latent tree. Our method outperformed the state of the art on the LSP, both in the scenarios when the training images are from the same dataset and from the PARSE dataset. Experiments on animal images from the VOC challenge further support our findings.

1. Introduction

The paper argues that learned tree models can represent human pose distributions while retaining efficient exact inference, including when single and combined parts are used together. On LSP, latent-tree learning introduced no hidden variables, and the resulting method outperformed state of the art across same-dataset, cross-dataset, and animal-pose evaluations.

  • 1. Introduction: The paper argues that simple tree models can combine single and combined parts without sacrificing the benefits of exact inference.This directly addresses concerns that combined parts require loopy or otherwise more complex graphical models.
  • 1. Introduction: The paper targets uncertainty over whether skeleton-based tree structures are optimal and how combined parts should be integrated without complicating inference.Prior work introduced latent variables and loopy graphs to address perceived limitations of simplified trees.
  • 1. Introduction: The framework learns a tree directly from observed body-part locations while allowing additional observed combined parts with the same spatial state space.Single joints and body limbs are handled at the same inference level.
  • 1. Introduction: Latent-tree learning on LSP introduced no hidden variables, suggesting that mixed observed variables can approximate the joint distribution in a simple tree.The learned structure was similar to the human skeleton despite the dataset’s pose variation and size.
  • 1. Introduction: The approach uses types to model appearance, defining visual categories for combined parts rather than relying only on physical connections.The paper reports that using more visual categories facilitates pose estimation and broadens the framework beyond physical connections.
  • 1. Introduction: The method outperformed state of the art on LSP and maintained performance when trained on PARSE and tested on LSP, with further support from VOC animal images.The authors attribute efficient inference to the tree structure.

2. Related work

Prior pose-estimation work used appearance features, pairwise deformable models, loopy graphs, latent nodes, and hierarchical structures to address pose variation. This paper integrates these components through learned latent trees that approximate observed-variable joint distributions without requiring fixed physical constraints.

  • 2. Related work: Earlier methods used HOG features and poselets, with poselets representing distinctive combined parts in training images.These approaches motivate richer appearance modeling for pose estimation.
  • 2. Related work: Pictorial structures use pairwise relative distances for efficient message passing, while mixtures-of-parts models impose co-occurrence constraints on non-oriented parts.Both are examples of deformable-model formulations reviewed by the paper.
  • 2. Related work: Loopy, latent-node, and hierarchical graph models were proposed because richer graphical structures were widely hypothesized to improve pose-estimation accuracy.Loopy graphs may improve precision but require more effort to solve.
  • 2. Related work: The paper proposes learning latent tree structures directly from observations to integrate articulated-body detection components efficiently.The approach is presented as a framework combining appearance and deformable-model ideas.
  • 2. Related work: Latent-tree models approximate the joint distribution of observable variables, automatically constructing structures using information distances to guide node merging.Unlike Chow-Liu trees, latent-tree methods can introduce hidden variables when needed.
  • 2. Related work: The latent-tree formulation avoids making many assumptions about physical constraints while retaining a guarantee related to joint-distribution approximation.This provides the paper’s rationale for examining learned trees in pose estimation.

3. Latent tree models for pose estimation

The paper learns tree structures from single and combined body-part observations, using visual categories to represent combined parts while retaining exact tree inference. On LSP, the learned structures introduce no latent variables, supporting a mixed observable-part tree approximation.

  • Latent-tree structure: The model represents each part as a location plus a type, where combined-part types are visual-category labels and single-part types encode part morphologies.The framework permits combined parts when they are observable and share the state-space format of single parts.
  • Part representation: Combined parts may encode either physically connected limbs or data-driven semantic relations between physically separated parts.Examples include upper and lower arms as connected parts and symmetric arm poses as separated parts.
  • Latent-tree structure: No latent variables are introduced when latent trees are learned from either single parts alone or single and combined parts on LSP.The resulting joint distributions can therefore be approximated with simple tree structures whose nodes remain observable.
  • Visual categories: Visual categories model large combined-part appearance variation through appearance-based clustering and latent-SVM templates rather than semantic or heuristic configurations.The learned HOG templates characterize distinct appearances for combined parts and reduce the complexity of their appearance models.
  • Inference model: The final tree model combines appearance, deformable, and compatibility terms, and uses standard message passing for exact inference.Compatibility terms encode training-set co-occurrence and reduce the inference search space; deformable terms are computed efficiently by distance transform.

4. Experiment results

The experiments evaluate the method on LSP, cross-dataset human pose estimation, and VOC animal images using efficient tree-based inference. Performance is measured with endpoint-based detection accuracy.

  • The experiments cover LSP, cross-dataset human pose estimation, and animal pose estimation on VOC images.The study uses three evaluation settings to assess performance and generalization.
  • The model uses 14 single joints, limb-based combined parts, learned visual categories, and the procedure defined for training the full pose model.
  • A part is correctly detected when both endpoints lie within 50% of the corresponding ground-truth segment length.
  • Testing takes approximately 2s per image with non-optimized Matlab code, while computational complexity is reported to match the order of.

4.1. LSP dataset

On LSP, the method is compared with several pose-estimation systems and outperforms the reported state of the art under the paper’s evaluation protocol.

  • 4.1. LSP dataset: The method’s LSP outputs include detections for single and combined parts followed by fitted skeletons, alongside qualitative comparisons and failure examples.
  • 4.1. LSP dataset: 62.8% Total accuracy, with 74.0% on Upper Leg and 69.8% on Lower Leg, exceeds Yang & Ramanan.The method is also reported as superior to Johnson & Everingham.
  • 4.1. LSP dataset: Under Tian et al.’s five-model evaluation procedure, the method still marginally outperforms their approach.
  • 4.1. LSP dataset: The LSP evaluation compares four algorithms on matched training and testing data, with Table 1 highlighting the highest scores.
  • 4.1. LSP dataset: The experiment suggests that exploiting single–combined-part connections together with exact inference supports state-of-the-art performance.

4.2. Cross dataset validation

Cross-dataset validation trains on PARSE and tests on LSP to examine generalization across datasets. The method does not degrade and exceeds in several reported comparisons.

  • 4.2. Cross dataset validation: The results are presented as evidence that the model generalizes across datasets rather than overfitting to dataset-specific combined-part appearances.
  • 4.2. Cross dataset validation: The cross-dataset experiment trains on all 305 PARSE images and tests on the LSP dataset after manual relabeling for coordinate conventions.

4.3. Pascal VOC dog dataset

On a challenging subset of PASCAL 2009 dog images, the method achieves higher leg and total accuracy than, despite lower head performance.

  • 4.3. Pascal VOC dog dataset: The experiment uses 280 dog images with at least nine annotated keypoints, including facial and foreleg points.
  • 4.3. Pascal VOC dog dataset: The model uses three combined parts—head and two legs—with six visual categories per combined part; 100 images are used for training.
  • 4.3. Pascal VOC dog dataset: The dog subset is challenging because of large pose and viewpoint variation, and both methods obtain low overall accuracies.
  • 4.3. Pascal VOC dog dataset: 60.6% and 63.3% accuracy on the left and right forelegs yield approximately 6% higher average leg accuracy and 3% higher Total accuracy than.Head performance is lower than the comparison method.

5. Conclusion

The paper learns latent tree models that combine single and combined parts for inference, with empirical gains in human and animal pose estimation.

  • The approach uses latent tree models to approximate joint distributions of body-part locations.
  • Single and combined parts are integrated for effective inference without abandoning the tree-model framework.
  • Empirical results suggest the approach outperforms the state of the art in human and animal pose estimation.
Loading 1305.2269v1…