Source-linked AI summary
Simple Baselines for Human Pose Estimation and Tracking
Bin Xiao, Haiping Wu, Yichen Wei
TL;DR
Pose estimation and tracking have become harder to analyze as their architectures and systems grow more complex. This paper introduces simple baselines using a deconvolution head for pose estimation and optical-flow-based propagation and similarity for tracking. The baselines achieve state-of-the-art benchmark results and are presented as tools for easier idea development and evaluation.
Problem
Growing algorithm and system complexity makes pose-estimation and tracking methods difficult to analyze and compare.
Method
The paper builds simple baselines with a ResNet deconvolution head for pose estimation and optical-flow-based pose propagation and similarity for tracking.
Results
The baselines achieve state-of-the-art results on challenging benchmarks, including 73.7 mAP for COCO pose estimation and 74.6 mAP with 57.8 MOTA for pose tracking.
Takeaways & Limitations
These simple baselines can support new idea development and simplify evaluation in pose estimation and tracking.
Takeaways & Limitations
The work provides no theoretical evidence and does not claim algorithmic superiority or complete fair comparisons with previous methods.
Abstract
from arXiv · showhide
There has been significant progress on pose estimation and increasing interests on pose tracking in recent years. At the same time, the overall algorithm and system complexity increases as well, making the algorithm analysis and comparison more difficult. This work provides simple and effective baseline methods. They are helpful for inspiring and evaluating new ideas for the field. State-of-the-art results are achieved on challenging benchmarks. The code will be available at https://github.com/leoxiaobin/pose.pytorch.
1 Introduction
As pose estimation and tracking improve, increasingly complex architectures and systems make algorithm analysis and comparison difficult. The paper addresses this by proposing simple, effective baselines that achieve strong benchmark results and support evaluation.
- Motivation: Pose estimation performance has advanced rapidly, while increasing architectural and experimental complexity makes algorithm analysis and comparison more difficult.MPII PCKH@0.5 rose from about 80% to above 90%, and COCO mAP rose from 60.5 to 72.1 in one year.
- Contribution: The paper asks how well a simple method can perform and provides baseline methods for both pose estimation and tracking.The authors intend these baselines to inspire ideas and simplify evaluation.
- Results: 73.7 mAP on COCO test-dev is achieved by the single pose-estimation model, exceeding the COCO 2017 challenge winner’s single and ensembled models.The improvements are 1.6% and 0.7%, respectively.
- Results: 74.6 mAP and 57.8 MOTA are achieved for pose tracking, improving over the ICCV’17 PoseTrack Challenge winner by 15% and 6% absolute.The paper reports these as new state-of-the-art results.
- Scope: The work relies on simple techniques and comprehensive ablation experiments rather than theoretical evidence, and does not claim algorithmic superiority or complete fair comparisons.The authors characterize the contribution as solid baselines for the field.
2 Pose Estimation Using A Deconvolution Head Network
The pose-estimation baseline adds a simple deconvolution head to ResNet to generate heatmaps from deep, low-resolution features. Relative to Hourglass and CPN, it combines upsampling and convolution in deconvolutional layers without skip connections.
- Baseline architecture: The baseline adds a few deconvolutional layers over ResNet’s final convolution stage, C5, to generate pose heatmaps.The authors describe this as a simple way to generate high-resolution heatmaps from deep features.
- Baseline architecture: Three deconvolutional layers with batch normalization, ReLU, 256 filters, 4 × 4 kernels, and stride 2 are used by default.A final 1 × 1 convolution produces heatmaps for all key points.
- Training: Mean Squared Error is used between predicted and targeted heatmaps, with each target heatmap generated from a 2D Gaussian centered on a joint.The supplied passage truncates the remaining target-heatmap definition.
- Reference architectures: Hourglass uses repeated bottom-up and top-down processing with skip-layer concatenation, while CPN uses skip-layer concatenation and online hard keypoint mining.Both are presented as reference architectures for understanding the baseline’s simplicity.
- Architectural comparison: Unlike Hourglass and CPN, the baseline combines upsampling and convolutional parameters in deconvolutional layers without skip-layer connections.All three methods use three upsampling steps and three levels of non-linearity from the deepest feature.
3 Pose Tracking Based on Optical Flow
The tracking pipeline assigns identities across video frames by matching pose instances, while optical-flow propagation supplements detections and supports flow-based pose similarity over multiple frames.
- Tracking formulation: Pose tracking first estimates human poses in each frame, then assigns unique identification numbers across frames; unmatched instances receive new ids.The tracked instance is represented as P = (J, id), where J contains body-joint coordinates.
- Flow-based similarity: The enhanced tracker replaces basic matching with a flow-based pose similarity metric and uses detector boxes together with optical-flow boxes.The design addresses fast motion, crowded scenes, pose changes, and missing detections caused by motion blur or occlusion.
- Flow-based detection: 15% expansion of the bounding box around propagated joints is used in the experiments to form candidate boxes for pose estimation.
- Flow-based similarity: Multi-frame flow-based similarity propagates poses from several previous frames, allowing instances to be relinked after disappearing in intermediate frames.
- Flow-based tracking algorithm: The complete inference algorithm detects and propagates boxes, estimates poses, computes similarities against a fixed-length history, greedily assigns ids, and updates that history.The history is stored in a deque whose length LQ determines how many previous frames participate in matching.
- Flow-based detection: The method combines detector boxes with boxes generated by propagating joints from previous frames using optical flow.Bounding-box NMS unifies the two sources before pose estimation, with propagated boxes complementing missing detections.
4 Experiments
Experiments evaluate the baseline on COCO and PoseTrack through ablations and comparisons with prior methods. The simple approach achieves strong pose-estimation and tracking results, while propagation and flow-based similarity improve tracking performance.
- Experimental Setup: COCO evaluation uses OKS-based AP, with training on COCO train2017 and final reporting on test-dev2017.The experiments also use val2017 for ablations and comparisons.
- COCO Ablations: Three deconvolutional layers producing 64 × 48 heatmaps outperform two layers producing 32 × 24 heatmaps by 2.5 AP.The three-layer configuration is the default.
- Comparison Caveat: Comparisons with Hourglass and CPN are based on cited results rather than implementations by the authors, so performance differences may reflect implementation differences.The authors therefore conclude that the baseline is comparable but simpler, without claiming algorithmic superiority.
- PoseTrack Ablations: Joint propagation improves both mAP and MOTA across detectors and backbones, including 4.3% mAP and 3.8% MOTA with R-FCN.The reported gains are 3.1% mAP and 2.3% MOTA with FPN-DCN, and 3.8% mAP and 2.8% MOTA with ResNet-152.
- PoseTrack Ablations: Flow-based pose similarity outperforms bounding-box similarity by 0.8% MOTA with multiple frames and 0.3% MOTA with a single frame.Using multiple frames further improves MOTA by 0.5% over using one previous frame, particularly when people reappear after occlusion.
- PoseTrack Comparisons: On PoseTrack, ResNet-152 improves pose-estimation mAP by 16.1% over the challenge winner and tracking MOTA by 10.2 and 5.8 on validation and test sets.Against the recent prior work, pose-estimation improvement is 10.2%, while tracking improvement is 7.1% and 6.6% on validation and test.
5 Conclusions
Simple and strong baselines for pose estimation and tracking achieve state-of-the-art results on challenging benchmarks and are validated through comprehensive ablation studies.
- Simple and strong baselines are presented for both human pose estimation and tracking.
- The baselines achieve state-of-the-art results on challenging benchmarks.
- Comprehensive ablation studies validate the proposed baselines.
- The authors hope the baselines will ease idea development and evaluation in the field.