BlazePose: On-device Real-time Body Pose tracking

• Single-person pose estimation is difficult because poses vary widely, bodies have many degrees of freedom, and joints may be occluded.Source: §1 · p. 1
• Heatmap-based methods handle multiple people efficiently but create models too large for real-time single-person inference on mobile phones.Source: §1 · p. 1
• Regression approaches are cheaper and more scalable but often predict mean coordinates without resolving pose ambiguity.Source: §1 · p. 1
• BlazePose predicts joint heatmaps with an encoder-decoder and then regresses directly to all joint coordinates with another encoder.Source: §1 · p. 1
• The heatmap branch is discarded during inference, making the resulting model lightweight enough for mobile deployment.Source: §1 · p. 1

Inference pipeline

• The detector-tracker pipeline uses a lightweight body detector followed by a tracker that predicts keypoints, person presence, and the next region of interest.When no person is detected, the detector runs again on the next frame.Source: §2.1 · p. 1

Person detector

• NMS can fail for articulated human poses because multiple ambiguous bounding boxes may satisfy its IoU threshold.Source: §2.2 · p. 1
• The person detector uses the face as a proxy for the torso and assumes the head is always visible in the single-person use case.The face provides a strong position signal because it has high-contrast features and fewer appearance variations.Source: §2.2 · p. 2

Topology

• The topology contains 33 body points, combining the keypoints used by BlazeFace, BlazePalm, and COCO for dataset and network consistency.Source: §2.3 · p. 2
• The topology retains only minimally sufficient face, hand, and foot keypoints to estimate region-of-interest rotation, scale, and position.Source: §2.3 · p. 2

Neural network architecture

• The model uses heatmap and offset losses only during training, then removes those output layers before inference while a regression encoder predicts coordinates.Gradient-stopping connections separate regression features from heatmap-trained features and improve coordinate regression accuracy.Source: §2.5 · p. 2

Alignment and occlusions augmentation

• Limiting augmentation ranges for angle, scale, and translation supplies a pose prior that reduces network capacity and computational and energy requirements.Source: §2.6 · pp. 2–3
• The system aligns the hips at the image center, rotates the mid-hip-to-mid-shoulder line vertically, and scales the body to fit a square input.It also applies 10% scale and shift augmentations for inter-frame movement.Source: §2.6 · p. 3

Evaluation setup

• PCK@0.2 measured whether 2D keypoint error was below 20% of the person’s torso size on two manually annotated 1000-image datasets.The AR dataset contained varied poses in the wild, while the second contained yoga/fitness poses; both used the 17-point MS Coco topology.Source: §3 · p. 3
• 97.2 PCK@0.2 was the average human re-annotation baseline on the AR dataset.Two annotators independently re-annotated the AR dataset to verify the human baseline.Source: §3 · p. 3

Results

• 25–75 times faster was BlazePose than OpenPose, comparing a single mid-tier phone CPU with a 20-core desktop CPU.The speed difference depended on the requested quality.Source: §3 · p. 3
• BlazePose Full slightly underperformed OpenPose on the AR dataset but outperformed it on yoga/fitness use cases.The compared BlazePose models included Full at 6.9 MFlop and 3.5M parameters and Lite at 2.7 MFlop and 1.3M parameters.Source: §3 · p. 3

Applications

• BlazePose targets performance-demanding applications including Sign Language, Yoga/Fitness tracking, and augmented reality.The model operates near real time on a mobile CPU and can reach superreal-time latency on a mobile GPU.Source: §4 · p. 3
• The model’s 33-keypoint topology is consistent with BlazeFace and BlazePalm, supporting use as a backbone for subsequent hand-pose and facial-geometry models.This topology consistency links the pose model with related inference networks.Source: §4 · p. 3

Extensions

• The approach scales natively to more keypoints, 3D support, and additional keypoint attributes without an additional full-resolution layer for each feature type.This follows from not relying on heatmaps or offset maps at inference.Source: §4 · p. 3

Face

• The keypoint list begins with facial landmarks including the nose, inner/outer eyes, ears, and mouth corners.These landmarks are assigned indices 0 through 10.Source: Appendix A · p. 4

Body and limbs

• The topology includes paired shoulders, elbows, wrists, hand knuckles, hips, knees, and ankles.The passage lists left and right landmarks with consecutive indices across these body regions.Source: Appendix A · p. 4