SMAP: Single-Shot Multi-Person Absolute 3D Pose Estimation

1 Introduction

• Absolute multi-person pose estimation must recover each person’s camera-coordinate translation in addition to the root-relative 3D pose.Prior approaches that recover each detected person separately can ignore global image context; the paper argues that body size, occlusion, and scene layout provide depth cues.Source: §1 · pp. 1–2
• A fully convolutional network predicts root depth, 2D keypoint heatmaps, part affinity fields, and part relative-depth maps, which are used to associate joints and reconstruct absolute 3D poses.The approach predicts multiple people’s positions and poses in a single forward pass.Source: §1 · pp. 2–3
• Predicted body depths support depth-aware part association by helping reason about occlusion and avoid implausible keypoint links, with reported benefits for 2D pose estimation.The paper also identifies inter-person occlusion and bone-length constraints as part of its association approach.Source: §1 · p. 3
• SMAP reports state-of-the-art performance on public benchmarks, generalization to in-the-wild images, and flexibility for whole- and half-body pose estimation.The introduction lists these as the system’s reported results and scope.Source: §1 · p. 3

Multi-person 2D pose

• Bottom-up multi-person 2D methods localize keypoints before grouping them; Open-Pose uses PAF connection confidence and greedy association, while Newell et al. predict pixel-wise tags.Source: §2 · p. 3

Single-person 3D pose

• Single-person monocular 3D methods either directly regress 3D keypoints or estimate 2D pose and lift it to 3D [18,30,43,44,38,4].The latter group includes learning-, optimization-, and exemplar-based approaches.Source: §2 · pp. 3–4

Multi-person 3D pose

• Multi-person 3D top-down methods crop each person and estimate translation separately, while bottom-up methods represent poses as maps but may leave translation to post-processing or omit root localization.The paper notes that cropping can lose cues about interactions between people, and that ground-plane methods may require visible feet or camera calibration.Source: §2 · p. 4

Monocular depth estimation

• Monocular depth methods face single-view ambiguity; human depth maps derived from Mannequin Challenge data lack scale consistency and do not reflect real depth.Source: §2 · p. 4

3.1 Intermediate representations

• SMAP predicts 2D keypoint heatmaps, part affinity fields, a root depth map, and part relative-depth maps for reconstructing absolute 3D poses.Relative-depth maps encode depth between joints in a body part and can provide information for reconstructing a skeleton when some keypoints are invisible.Source: Figure 2 · p. 5, §3.1 · p. 6
• A root depth map stores each person's absolute depth at the root-joint location, representing multiple people without a predefined person count.The map is supervised only at root locations and requires 3D poses rather than full depth maps as training supervision.Source: §3.1 · pp. 5–6

3.2 Depth-aware part association

• Depth-aware association sorts people near-to-far and uses a depth-dependent adaptive bone-length threshold to resolve occlusion ambiguities and filter implausible links.The threshold depends on predicted depth and statistical bone length, not camera intrinsics or image resizing.Source: §3.2 · pp. 7–8

3.3 3D pose reconstruction

• The system reconstructs joint depths from root and relative depths, back-projects 2D joints through a camera model, and optionally refines poses without changing root depths.RefineNet addresses cumulative localization and back-projection errors and completes invisible keypoints; internet images use a default focal length equal to image width when the focal length is unknown.Source: §3.3 · p. 8

4.1 Datasets

• The evaluation uses CMU Panoptic indoor social-activity images and MuPoTS-3D indoor and outdoor scenes with varied camera poses.The Panoptic test set contains 9,600 images from four activities, while 160,000 images from different sequences form its training set.Source: §4.1 · p. 9

4.2 Implementation details

• Models are trained separately on CMU Panoptic and MuCo-3DHP for 20 epochs, with COCO data mixed into training batches.The input size is 832×512; 3D loss weights are zero for COCO data because it lacks 3D pose annotations.Source: §4.2 · p. 9

4.3 Evaluation metrics

• Metrics assess root-relative pose with MPJPE, root localization with RtError and PCKroot, absolute and relative pose with PCKabs and PCKrel, and inter-person ordinal depth with PCOD.PCOD classifies each pair as closer, farther, or roughly the same in depth, with the same category defined as within 30 cm.Source: §4.3 · pp. 9–10

4.4 Comparison with state-of-the-art methods

• SMAP outperforms previous methods by a large margin across all reported metrics on CMU Panoptic, and leads on MuPoTS-3D in PCKabs, PCKroot, and PCOD.On MuPoTS-3D, SMAP also has higher PCKrel than all bottom-up methods and most top-down methods, but has higher PCKrel at the 15 cm threshold; SMAP has higher AUCrel.Source: Table 1 · p. 10, §4.4 · pp. 10–11, Table 2 · p. 10
• SMAP has better PCKroot across thresholds than and reprojection-fit translation, with roughly 100% accuracy at a 1 m threshold; leads PCKrel at 15 cm, while SMAP wins at smaller thresholds and in AUCrel.Qualitative examples show can fail in varied poses, occlusion, and truncation; SMAP uses whole-image features. Its runtime and memory remain almost unchanged as the number of people grows, unlike.Source: §4.4 · p. 12
• Depth estimates from SMAP are more consistent than alternatives on MuPoTS-3D, including read-out depths from a full depth map and the top-down approach.The comparison uses predicted versus ground-truth root-depth scatter plots; the authors note that methods were trained on different datasets, so the comparison is not rigorous.Source: §4.4 · p. 12, Figure 6 · p. 13

4.5 Ablation analysis

• Ablations show that depth normalization and multi-scale supervision benefit performance, while a one-stage hourglass still exceeds in PCKroot and PCKabs; RefineNet improves relative and absolute pose estimation.RefineNet completes invisible and refines visible keypoints with a learned 3D pose prior; its gains are more significant on Panoptic, where training and test camera views are similar.Source: §4.5 · pp. 12–13
• Depth-aware part association outperforms 2D part association across the evaluated metrics on Panoptic and MuPoTS-3D.The comparison uses the same heatmaps and part affinity fields for both association methods.Source: §4.5 · p. 13, Table 4 · p. 14

• A fully convolutional network regresses 2.5D representations for multiple people, from which their absolute 3D poses are reconstructed.The framework is designed for a single RGB image.Source: §5 · p. 14
• The depth-aware part association algorithm is reported to benefit 2D pose estimation in crowd scenes.Source: §5 · p. 14
• Experiments demonstrate state-of-the-art performance and generalization ability for the proposed approach.Source: §5 · p. 14

1.1 Loss function

• The root-depth branch uses L1 loss, while L2 loss supervises the keypoint heatmap, PAF, and part relative-depth branches.The root-depth prediction is evaluated at the detected root location against normalized ground-truth depth.Source: §1.1 · p. 15
• The total objective is a weighted sum of the 2D, relative-depth, and root-depth losses.Default weights are w2D=0.1, w∆Z=5, and wRZ=10.Source: §1.1 · p. 15

1.2 Running time and memory

• The method's runtime is reported as almost not affected by the number of people in the image.Table 5 compares running time and memory with top-down method.Source: §1.2 · p. 15, Table 5 · p. 16

• Sequence-wise PCKabs is higher than the state-of-the-art top-down method, especially for outdoor sequences TS6–TS20.Source: §2 · p. 16
• PCKrel is higher than for all bottom-up methods and most top-down methods except, while AUCrel is higher than.Source: §2 · p. 16

3.1 Effect of the multi-task structure

• Adding keypoint heatmap and PAF branches to root-position-and-depth prediction significantly improves root localization, though it remains inferior to the full model.The authors report that this improvement indicates 2D cues, including pose and body size, are beneficial to root-depth estimation.Source: §3.1 · p. 16, Table 11 · p. 18

3.2 Influence of camera intrinsics

• RtError is 23.3cm with the full model, 67cm without provided intrinsics, and 120cm without normalization.The ordinal depth relation remains unchanged when intrinsics are not provided.Source: §3.2 · p. 16
• The Human3.6M results report MPJPE without ground-truth bounding-box information at inference.Source: Table 10 · p. 18