PaperlayerView exact revision on arXiv

Source-linked AI summary

Robust Estimation of 3D Human Poses from a Single Image

Chunyu Wang, Yizhou Wang, Zhouchen Lin, Alan L. Yuille, Wen Gao

arXiv:1406.2282v1cs.CV

TL;DR

Single-image 3D pose estimation must handle depth ambiguity, inaccurate 2D detections, and unknown cameras. The paper combines sparse skeleton bases, limb constraints, an L1 projection loss, and ADM, outperforming state-of-the-art methods on three benchmark datasets.

  • Problem

    Single-image 3D pose estimation is challenged by multiple 3D poses sharing one 2D projection and by inaccurate 2D joint estimates.

  • Method

    The method combines sparse basis representations, limb-length constraints, an L1 projection error, and alternating updates of 3D pose and camera parameters solved with ADM.

  • Results

    The approach outperforms state-of-the-art methods on three benchmark datasets.

  • Takeaways & Limitations

    The approach provides a single-image 3D pose estimator designed to tolerate inaccurate 2D detections while constraining skeleton plausibility.

  • Takeaways & Limitations

    Limb proportions are normalized using fixed proportions kept the same for all people.

Abstract

from arXiv · show

Human pose estimation is a key step to action recognition. We propose a method of estimating 3D human poses from a single image, which works in conjunction with an existing 2D pose/joint detector. 3D pose estimation is challenging because multiple 3D poses may correspond to the same 2D pose after projection due to the lack of depth information. Moreover, current 2D pose estimators are usually inaccurate which may cause errors in the 3D estimation. We address the challenges in three ways: (i) We represent a 3D pose as a linear combination of a sparse set of bases learned from 3D human skeletons. (ii) We enforce limb length constraints to eliminate anthropomorphically implausible skeletons. (iii) We estimate a 3D pose by minimizing the $L_1$-norm error between the projection of the 3D pose and the corresponding 2D detection. The $L_1$-norm loss term is robust to inaccurate 2D joint estimations. We use the alternating direction method (ADM) to solve the optimization problem efficiently. Our approach outperforms the state-of-the-arts on three benchmark datasets.

Abstract

The paper estimates 3D human poses from a single image alongside an existing 2D detector, addressing depth ambiguity and inaccurate 2D joints with structured, robust optimization. It reports outperforming state-of-the-art methods on three benchmark datasets.

  • 3D pose estimation is ambiguous because multiple 3D poses can project to the same 2D pose, while inaccurate 2D detections can distort estimation.
  • The method represents 3D poses as sparse linear combinations of bases learned from 3D human skeletons.
  • Limb length constraints eliminate anthropomorphically implausible skeletons during estimation.
  • An L1-norm projection error is used because it is robust to inaccurate 2D joint estimates.
  • ADM solves the optimization problem efficiently, and the approach outperforms state-of-the-art methods on three benchmark datasets.

1. Introduction

The introduction motivates single-image 3D pose estimation through depth ambiguity, detector noise, and unknown cameras, then presents an alternating, structured method to address these challenges.

  • Human pose estimation supports action recognition, whose applications include human-computer interaction and video surveillance.
  • Single-image 3D estimation is difficult because 2D detector accuracy, projection ambiguity, and unknown camera parameters complicate inference.
  • The method alternates camera-parameter estimation with 3D-pose updates, beginning from detected 2D joints and an initialized 3D pose.
  • Sparse basis coefficients encode a low-dimensional skeleton prior that reduces 3D ambiguities and provides robustness to missing joints.
  • L1 projection error tolerates inaccurate 2D joints, while eight limb-length constraints remove anthropomorphically implausible skeletons.

2. Related Work

Prior work differs by its required inputs, while this method uses only a single image and distinguishes itself through automatic 2D detection, robust objectives, stronger constraints, and ADM optimization.

  • Existing approaches are grouped by whether they require camera parameters, manually labeled video joints, manually labeled single-image joints, or only an image or image features.
  • Camera-parameter-based methods estimate pose using model projections and image discrepancies, including silhouette matching with MCMC.
  • Video-based methods use manually labeled 2D joints and may require human input to resolve depth ambiguities or impose rigid-body constraints.
  • Single-image methods have used manually labeled joints, basis combinations, exemplar matching, or silhouette-manifold mappings.
  • Compared with related basis methods, this approach detects 2D joints automatically, uses L1 penalties, enforces eight limb constraints, and applies ADM.

3. Our Approach

The approach alternates camera and 3D-pose estimation under a weak-perspective model. It combines sparse basis representations, robust L1 projection error, and limb-length constraints to address ambiguity and inaccurate 2D detections.

  • 3. Our Approach: The method models 2D and 3D poses as x = My under a weak perspective camera, then alternately estimates camera parameters and the 3D pose.Camera estimation and pose estimation are repeated until convergence or the iteration limit.
  • 3. Our Approach: Each 3D pose is represented as a linear combination of overcomplete bases and a mean pose, with L1-regularized coefficients encouraging only a few active bases.The representation encodes a low-dimensional structural prior for human skeletons.
  • 3.1. Robust 3D Pose Estimation: The pose objective minimizes L1 projection error between estimated 2D joints and projected 3D joints, while enforcing basis sparsity and eight limb-length constraints.The parameter θ balances projection loss and regularization.
  • 3.1. Robust 3D Pose Estimation: L1 loss is more tolerant of inaccurate 2D joint outliers than L2 loss, which can bias the inferred 3D pose toward a wrong configuration.Figure 2 illustrates this difference when the detected right-foot location is inaccurate.
  • 3.1. Robust 3D Pose Estimation: L1 sparsity reduces ambiguity and can prevent overfitting inaccurate 2D detections, whereas unconstrained bases can drive projection error to zero without guaranteeing a correct pose.The paper describes sparsity as a structural prior against incorrect or anthropomorphically implausible poses.
  • 3.1. Robust 3D Pose Estimation: The limb constraints normalize the right lower-leg length to one and set the other seven limb lengths according to fixed anthropometric proportions.The eight limbs are the left and right upper and lower arms and legs.

4. The Experimental Results

Experiments combine controlled evaluations of model components and real-image comparisons with state-of-the-art methods across three datasets. On CMU basis reconstruction, sparse bases achieve the lowest errors while activating fewer bases.

  • 4. The Experimental Results: The evaluation includes controlled tests of sparsity, anthropomorphic constraints, L1 penalty, inaccurate 2D detections, and human-camera angles, plus real-image comparisons with state-of-the-art methods.The datasets are CMU, HumanEva, and UvA 3D pose.
  • 4. The Experimental Results: The experiments use 12 body joints, 200 bases, approximately 6 activated bases per pose, and a maximum of 20 optimization iterations.The joints match the employed 2D pose detector.
  • 4.2. Basis Learning: The sparse bases consistently achieve the lowest CMU 3D pose reconstruction errors among sparse, PCA, and classwise PCA basis methods.PCA and classwise PCA use maximum basis counts of 36 and 144, respectively.
  • 4.2. Basis Learning: Fewer bases are activated with the L1-induced sparse bases than with the comparison methods, supporting their representative power.Figure 3 reports reconstruction errors and the cumulative distribution of activated-basis counts.

4.3. Controlled Experiments

Controlled experiments isolate the effects of sparsity, anthropomorphic constraints, L1 loss, inaccurate 2D poses, and human-camera angle on 3D pose estimation.

  • The evaluation assumes known ground-truth 2D poses and measures residual Euclidean error between estimated and ground-truth 3D poses.
  • Seven baselines vary the sparsity term, anthropomorphic constraints, and L1- versus L2-norm objectives.The baselines include L2NAWS, L2NANS, L2WANS, L2WAWS, L1NANS, L1NAWS, and L1WANS.
  • Baselines with sparsity outperform those without it, limb-length constraints improve performance, and L1-norm objectives outperform L2-norm objectives.Examples include L2WAWS outperforming L2NAWS and L1NAWS outperforming L2NAWS.
  • The proposed approach performs best among the baselines and outperforms Ramakrishna et al.’s method under noisy 2D-pose conditions.The noise experiment adds ten levels of random Gaussian noise, with the largest magnitude equal to the normalized right lower-leg length.
  • L1NANS outperforms L2NANS, demonstrating greater robustness to 2D pose errors, while sparsity also improves performance under inaccurate 2D poses.L2NANS and L2WANS have larger errors than L2NAWS and L2WAWS, respectively.
  • As the human-camera angle varies from 0 to 180 degrees, the proposed approach is more robust to viewpoint changes than the compared method.The compared method’s errors increase drastically from profile at 90 degrees toward frontal pose at 0 degrees.

4.4. Real Experiments

The experiments evaluate the method on UvA and HumanEva, including 3D estimation, camera initialization, and projected 2D pose quality. The method improves estimation against prior approaches and benefits from multiple pose initializations.

  • The pipeline detects 2D joints from an image and estimates the corresponding 3D pose.
  • Our approach achieves smaller estimation errors than a state-of-the-art method on every UvA joint, especially both hands.The authors interpret this as robustness to inaccurate 2D joint locations.
  • Our approach outperforms both compared state-of-the-art methods on the HumanEva dataset.The evaluation uses root mean square errors adopted from prior work.
  • Camera estimation usually converges within nine iterations, with estimated and groundtruth cameras differing subtly for 70% of cases.
  • Initializing with 30 clustered training-pose centers and retaining the lowest-error solution further improves performance.The 30 initializations are optimized in parallel.
  • Projected 2D results are evaluated using PCP and Euclidean pixel distance, with the approach best on six body parts and both criteria overall.For PCP, performance improves over the original 2D estimator from 0.714 to 0.741.

5. Conclusion

The conclusion presents a single-image 3D pose estimator that combines structural constraints, a robust projection loss, and efficient optimization. It reports stronger performance than state-of-the-art methods across three benchmark datasets.

  • The approach estimates 3D human poses from a single image alongside an existing 2D pose detector.
  • Sparse basis representation, anthropometric constraints, and an L1-norm projection error make the method robust to inaccurate 2D pose estimates.
  • An efficient alternating direction method solves the resulting optimization problem.
  • The approach outperforms state-of-the-art methods on three benchmark datasets.

6. Appendix: Optimization by ADM

The appendix sketches the major alternating direction method steps used for the optimization problems, with k and l denoting iteration counts.

  • The appendix only sketches the major ADM steps because of space limitations.
  • The symbols k and l denote the numbers of iterations in the optimization procedure.
  • The detailed optimization discussion concerns alternating updates across iterations.

6.1. 3D Pose Estimation

The appendix reformulates the pose-estimation optimization with auxiliary variables and solves its components by alternating minimization, convex programming, and closed-form updates.

  • Auxiliary variables β and γ rewrite the optimization problem with L1 penalties and constraints linking coefficients, projected error, and limb-length conditions.
  • ADM updates γ and β by minimizing the augmented Lagrangian after discarding terms independent of each variable.Both updates have closed-form solutions.
  • The α update introduces z and converts the constrained quadratic problem into a rank-one positive-semidefinite matrix formulation.The transformation uses Q = zz^T and linear trace constraints.
  • The matrix subproblem is split with an auxiliary P and solved by ADM under equality, positive-semidefinite, and rank constraints.
  • The Q update is convex and solved using CVX, while the P update has a closed-form eigenvalue-based solution.The solution uses the largest eigenvalue and corresponding eigenvector of a symmetric matrix.
  • For the P update, the minimizing scalar is max(ζ1, 0), with ν set to the eigenvector ν1 associated with the largest eigenvalue ζ1.

6.2. Camera Parameter Estimation

The camera-parameter problem is reformulated with an auxiliary variable and solved using ADM. The method updates the auxiliary variable and two additional variables, each through closed-form solutions.

  • An auxiliary variable R is introduced to rewrite the optimization problem.
  • ADM is used to solve the reformulated problem through an augmented Lagrangian.
  • The augmented Lagrangian uses H and ζ as Lagrange multipliers and τ > 0 as the penalty parameter.
  • R is updated after discarding terms in the augmented Lagrangian that are independent of R, yielding a closed-form solution.
  • The variables m1 and m2 are likewise updated by discarding terms independent of each variable, and both updates have closed-form solutions.
Loading 1406.2282v1…