Source-linked AI summary
Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUs
Timo von Marcard, Bodo Rosenhahn, Michael J. Black, Gerard Pons-Moll
TL;DR
Human pose capture in natural settings is constrained by the intrusiveness of multi-sensor systems and the limitations of camera-based tracking. SIP addresses this by fitting a statistical body model jointly to orientation and acceleration measurements across frames, achieving accurate full-body tracking from six IMUs. The method is demonstrated on TNT15 and challenging outdoor motions, while global position drift and wrist and ankle parameter accuracy remain limitations.
Problem
Natural motion capture needs a less intrusive alternative because camera-based systems restrict applicability and conventional inertial systems can require 17 sensors.
Method
SIP uses a statistical SMPL body model and jointly optimizes poses over multiple frames to fit orientation and acceleration data from six IMUs.
Results
SIP achieves 13.32 degrees orientation error and 3.9 cm positional error, and is demonstrated on TNT15 and challenging outdoor motions.
Takeaways & Limitations
Six-IMU tracking makes full-body pose estimation feasible without video, MoCap databases, or strong motion assumptions.
Takeaways & Limitations
Global position estimates suffer drift over time, and the current solution cannot accurately capture wrist and ankle joint parameters.
Abstract
from arXiv · showhide
We address the problem of making human motion capture in the wild more practical by using a small set of inertial sensors attached to the body. Since the problem is heavily under-constrained, previous methods either use a large number of sensors, which is intrusive, or they require additional video input. We take a different approach and constrain the problem by: (i) making use of a realistic statistical body model that includes anthropometric constraints and (ii) using a joint optimization framework to fit the model to orientation and acceleration measurements over multiple frames. The resulting tracker Sparse Inertial Poser (SIP) enables 3D human pose estimation using only 6 sensors (attached to the wrists, lower legs, back and head) and works for arbitrary human motions. Experiments on the recently released TNT15 dataset show that, using the same number of sensors, SIP achieves higher accuracy than the dataset baseline without using any video data. We further demonstrate the effectiveness of SIP on newly recorded challenging motions in outdoor scenarios such as climbing or jumping over a wall.
1. Introduction
SIP targets practical motion capture in natural settings by estimating full 3D pose from six IMUs, avoiding the intrusiveness of larger inertial systems and limitations of camera-based capture. It combines a realistic statistical body model with joint multi-frame optimization and outperforms the stated baselines.
- Motivation: Marker-based and vision-based systems restrict motion capture through laboratory requirements, calibrated cameras, intrusiveness, or subject-specific tracking.These limitations hinder recording everyday activities and natural outdoor motion.
- Motivation: IMU systems support camera-free tracking in outdoor and occluded settings, but conventional systems can require 17 sensors and remain intrusive.The paper motivates a less intrusive alternative that can capture motion through occlusions.
- Contribution: SIP recovers full 3D human pose from six orientation- and acceleration-measuring IMUs attached to the wrists, lower legs, waist, and head.The sparse setup is intended as a minimally intrusive solution, despite weak orientation constraints and acceleration drift.
- Method: The method uses the statistical SMPL body model to encode human kinematic constraints and jointly optimizes all sequence poses against orientation and acceleration measurements.Joint sequence optimization enforces coherence across frames and addresses drift more effectively than frame-by-frame integration.
- Evaluation: SIP is evaluated with six tracking IMUs and four validation IMUs against orientation-only and alternate-body-model baselines, including outdoor recordings.The reported qualitative and quantitative results describe SIP as significantly more accurate than the baselines.
- Contribution: SIP estimates 3D pose from six IMUs without MoCap databases or learning methods that make strong assumptions about recorded motion.The paper presents this as the first method with this capability.
2. Related Work
Prior work reconstructs pose from sparse signals using databases, markers, or inertial systems, but existing approaches either face difficult learning problems or require many intrusive sensors. These limitations motivate sparse inertial pose estimation without such extensive instrumentation.
- Sparse-signal methods: Sparse accelerometer methods retrieve similar prerecorded poses, but noisy acceleration and a large acceleration space make learning difficult.The cited approach reconstructs pose from five accelerometers using database retrieval.
- Sparse-signal methods: Marker-based methods reconstruct full 3D pose from sparse body markers using online local PCA models and human-motion databases.The passage presents this as a related approach for sparse control signals.
- Inertial systems: Commercial inertial systems can provide stable pose tracking with 17 IMUs, but they are intrusive and require long setup times.Earlier systems also used 18 sensor boards with acoustic distance sensors to compensate for orientation drift.
3. Background
The paper represents rigid motion and articulated bodies with Lie-group geometry and the SMPL statistical body model, then relates these model coordinates to IMU measurements. SMPL supplies learned shape, skeletal, and joint-location structure for fitting sparse inertial data.
- Exponential Map on SO(3) and SE(3): Exponential mapping represents rotations and rigid motions on SO(3) and SE(3), offering an optimization-oriented alternative to Euler angles.SO(3) models three-dimensional rotations, while SE(3) models rotations with translations.
- Exponential Map on SO(3) and SE(3): SO(3) consists of valid 3D rotation matrices, and exponential coordinates specify a rotation axis and angle.The coordinates are obtained from the skew-symmetric Lie-algebra representation.
- Exponential Map on SO(3) and SE(3): SE(3) represents rigid body motions composed of a rotation and translation, with six exponential coordinates combining rotational parameters and translation-related parameters.The associated twist action belongs to the Lie algebra se(3).
- SMPL Body Model: SMPL uses a learned template and rigged skeleton with 24 ball joints, three rotational degrees of freedom per joint, and a 75-parameter pose vector including translation.Its forward kinematic map computes bone motions from joint states in the articulated chain.
- SMPL Body Model: SMPL models body-shape variation with linearly combined blend shapes and predicts joint locations from the resulting surface mesh.Surface-based joint prediction provides joint locations needed to predict acceleration measurements and avoids manual skeleton editing.
- IMUs: IMUs measure acceleration, rate of turn, and magnetic field strength in sensor coordinates, while tracking requires mappings among sensor, bone, model, and inertial frames.The sensor-to-bone offset is assumed constant because the sensors are tightly attached to the limbs and is computed at the first frame.
4. Sparse Inertial Poser
SIP addresses the ambiguity of pose recovery from six IMUs by combining a statistical body model with multi-frame optimization of orientation and acceleration consistency. The resulting formulation uses anthropometric constraints and joint optimization to recover plausible motion from sparse measurements.
- Problem: Six-IMU orientation measurements leave full-pose recovery highly ambiguous, while naive acceleration integration produces unbounded drift.The problem becomes more constrained when the full sequence and anthropometric body-model constraints are considered.
- Objective: SIP optimizes all poses across a sequence against orientation, acceleration, and anthropometric consistency energies.The objective stacks model poses over T frames and combines the three energies with fixed weights.
- Orientation Term: Orientation consistency compares estimated sensor orientations from forward kinematics with measured orientations across sensors and frames.The orientation error uses exponential coordinates, and its squared norm corresponds to geodesic rotational distance.
- Acceleration Term: Acceleration consistency transforms sensor accelerations into global coordinates, subtracts gravity, and matches them to numerical vertex accelerations from the body model.The acceleration error is accumulated across all frames and sensors.
- Anthropometric Term: The anthropometric term combines a statistical pose prior with explicit joint-limit penalties to constrain skeletal states to human-like poses.The pose prior uses a multivariate Gaussian learned from SMPL registrations, while joint-limit errors repel violations.
- IMU Placement: The tracker uses six IMUs on the lower legs, lower arms, waist, and head; this placement outperforms moving sensors to feet and hands because end-effectors add kinematic-chain uncertainty.The proposed placement constrains many pose parameters while remaining minimally intrusive.
5. Experiments
Experiments evaluate SIP against orientation-only and alternative-body-model baselines on TNT15, using quantitative errors and challenging outdoor recordings. SIP achieves more accurate tracking with six IMUs and remains robust across body-model variations.
- Baselines: SIP is evaluated against SOP, which ignores acceleration, and SIP-M, which uses a manually rigged body model.The tracking procedure initializes SIP with the sequentially optimized SOP trajectory before optimizing the full cost function.
- Evaluation protocol: The quantitative evaluation uses four validation IMUs for mean orientation error and thirteen virtual markers for mean position error.Marker ground truth comes from tracking with all ten IMUs; global translation is set to zero for position-error calculation.
- Quantitative results: SIP clearly outperforms SOP and SIP-M on a TNT15 jumping-jack sequence with extended limb motions and foot stamps.Acceleration resolves ambiguities that orientation measurements alone cannot, while SIP-M’s body model is insufficiently expressive for some motions.
- Quantitative results: Tracking errors remain comparably small when the SMPL body model is estimated from linguistic ratings or its body size is increased by 10% or 20%.The results indicate robustness to moderate body-shape variation and remove the need for laser-scan accuracy.
- Quantitative results: 19.64° to 13.32°: SIP reduces TNT15 mean orientation error relative to SOP, while mean position error falls from 7.2cm to 3.9cm.These results support incorporating acceleration data into sparse-IMU full-body tracking.
- Qualitative results: SIP successfully tracks outdoor motions including jumping over a wall, biking, climbing, and warming exercises using only six IMUs.Most recovered poses are visually accurate, and a separate handwriting experiment reconstructs recognizable wrist motion.
6. Conclusions and Future Work
SIP reconstructs human motion from six inertial sensors by combining a statistical body model with joint multi-frame optimization. The paper reports accurate pose recovery while identifying remaining limitations in global position drift and wrist and ankle estimation.
- Conclusions: SIP reconstructs human pose from sparse inertial sensors by jointly optimizing pose over multiple frames against orientation and acceleration data.The approach exploits a statistical body model and can also use approximate body models.
- Conclusions: 13.32 degrees orientation error and 3.9 cm positional error demonstrate accurate quantitative pose reconstruction with SIP.
- Conclusions: SIP achieves the best results among SIP, the orientation-only SOP baseline, and the manually rigged SIP-M variant in the TNT comparisons.The comparison attributes ambiguity without acceleration to SOP and describes SIP as producing the best results.
- Future Work: Global position estimates still suffer from drift over time even though SIP tracks the full-body pose without drift.The authors propose adding physical constraints, GPS, or body-mounted visual data to compensate for this drift.
- Future Work: The current sensor placement cannot accurately capture wrist and ankle joint parameters, leaving those parameters to anthropometric-prior optimization.The authors propose incorporating constraints from 3D world geometry and pose-conditioned joint limits.
- Future Work: SIP enables minimally intrusive motion capture with as few as six IMUs and is intended for applications including virtual reality, sports analysis, and health assessment.The authors also plan to extend the method from articulated motion to soft-tissue motion.