Source-linked AI summary
ArmPoser: Real-Time, Calibration-Free Arm Pose Estimation from Smartwatch IMU
Bishnu Dev, Vasco Xu, Xi-Aan Loh, Chenfeng Gao, Henry Hoffmann, Karan Ahuja
TL;DR
Prior smartwatch-based arm-pose methods require per-user calibration and transform device measurements into a body-model frame. ArmPoser learns in the smartwatch’s native gravity-aligned frame, and achieves the lowest positional error on all four datasets and the lowest average angular error, without user calibration.
Problem
Prior smartwatch-based methods require users to hold a known reference pose to align device-local IMU measurements with a body-model frame.
Source: §1 · pp. 1–2Method
ArmPoser learns directly from a smartwatch’s native gravity-aligned IMU frame and includes a detector for watch placement and crown orientation.
Source: §1 · p. 2, §4.1 · p. 4Results
8.1 cm median positional error across the wrist and elbow; ArmPoser had the lowest positional error on all four datasets and the lowest average angular error.
Source: §1 · p. 2Takeaways & Limitations
The authors conclude that calibration-free arm pose tracking becomes practical for everyday use across users and consumer devices.
Source: §1 · p. 2A 170 ms delay and 9–12 cm wrist error may suit rehabilitation feedback, activity logging, and coarse gestures, but not direct XR manipulation or small-target pointing.
Source: §8 · p. 10
Abstract
from arXiv · showhide
Arm pose estimation enables applications in fitness, extended reality input, rehabilitation, and life logging. Prior smartwatch-based approaches rely on calibration poses and preprocessing pipelines that transform raw IMU measurements into standardized training formats. These steps hinder deployment in everyday settings and introduce errors due to imperfect calibration and sensor drift. We present ArmPoser, a calibration-free arm pose estimation system using a single smartwatch IMU. Our central contribution is training models directly in the reference frame native to consumer smartwatches, aligning learning with how IMU data is produced by deployed devices. By operating on device-native axes, ArmPoser removes the need for coordinate transformations, explicit alignment, and bone-offset calibration used in prior work. We further augment training with physically grounded variations in watch placement and arm morphology to account for user-specific variability. ArmPoser also includes a wear-configuration module that infers anterior or posterior forearm placement and crown orientation. We evaluate pose estimation on public benchmarks and on a 10-participant, 30-activity study using watchOS and Android smartwatches, where ArmPoser matches or exceeds calibrated baselines without any user calibration.
1 Introduction
Prior smartwatch-based arm-pose methods require per-user calibration to align sensor readings with a body model, limiting casual use. ArmPoser learns in the smartwatch’s native frame and reports lower errors without calibration.
1 Introduction
- Per-user reference-pose calibration aligns smartwatch readings with the body, but is inconvenient, time-consuming, and incompatible with casual daily use.Prior methods also transform local measurements into a body-model frame and estimate sensor-to-bone offsets.Source: §1 · pp. 1–2
- ArmPoser trains directly in the gravity-aligned local frame native to consumer smartwatches, matching training to how deployed devices report IMU data.Training simulates sensor-to-bone offsets and arbitrary yaw to model user-specific variation and unknown heading.Source: §1 · p. 2
- A lightweight wear-configuration module recognizes anterior or posterior placement and smartwatch crown orientation.The system is designed to infer these wearing configurations without requiring the prior T-pose step.Source: §1 · p. 2, Figure 2 · p. 2
- 8.1 cm median positional error across wrist and elbow; ArmPoser has the lowest positional error of any method on all four datasets and the lowest average angular error.It was evaluated on public benchmarks and a new dataset of 10 participants performing 30 everyday activities.Source: §1 · p. 2
- The paper contributes a benchmark dataset pairing raw and calibrated IMU data with ground-truth SMPL poses.Its contributions also include calibration-free evaluation on public and collected datasets.Source: §1 · p. 2
2 Related Work
Related work spans full-body estimation from worn IMUs and arm-specific wearable tracking. These systems use multiple sensors, calibration, or known sensor locations; ArmPoser instead targets one smartwatch without calibration.
2.1 Full-body Pose Estimation using Worn IMUs
- Full-body IMU systems such as SIP, DIP, TransPose, PIP, PNP, and GlobalPose use multiple sensors, while TIC estimates calibration parameters at runtime but still depends on six Xsens IMUs for ego-yaw.TIC uses that shared heading reference for drift correction.Source: §2.1 · pp. 2–3
- IMUPoser and MobilePoser use consumer devices but require three-step session calibration; IMUCoCo requires known sensor coordinates and global alignment.ArmPoser focuses on one smartwatch and requires no calibration, sensor localization, or global reference alignment, while achieving lower positional error than calibrated prior methods.Source: §2.1 · p. 3
2.2 Arm Pose Estimation from Wearables
- ArmTrak uses one wrist IMU but is computationally heavy, has high real-time error, and requires users to remain stationary after calibration; SmartPoser combines phone and watch data and also requires calibration.SmartPoser uses UWB distance measurements with smartwatch IMU data.Source: §2.2 · p. 3
- ArmPoser achieves similar or lower error than prior arm-tracking methods on public benchmarks and its 10-participant, 30-activity dataset, while detecting watch placement and crown orientation.Prior works require recalibration when watch position changes on the wrist.Source: §2.2 · p. 3
3 Background
Single-wrist IMU arm-pose recovery is underconstrained, and conventional methods calibrate smartwatch readings into a body-model frame. TIC's runtime alternative assumes a pelvis-derived heading reference unavailable to a lone-watch setup.
3 Background
- A single wrist-mounted IMU leaves arm-pose recovery underconstrained because different shoulder and elbow angles can produce identical wrist measurements.Temporal context has enabled accurate, stable recovery in prior data-driven methods.Source: §3 · p. 3
- Conventional pipelines calibrate smartwatch readings into the body-model frame, including alignment of the watch and body-model frames and a fixed sensor-to-bone offset.A user may hold a T-pose to measure the transformations.Source: §3 · pp. 3–4
- TIC estimates sensor-to-bone offset at runtime but assumes a pelvis-mounted IMU supplies ego-yaw, which a single-watch setup lacks.With a generic body-model frame, dynamic calibration performs worse than static calibration.Source: §3 · p. 4
- ArmPoser reformulates training to remove the need for calibrated wrist signals and the associated global-frame transformation and bone offset.The reformulation is introduced as an alternative to training on calibrated wrist signals.Source: §3 · p. 4
4 Method
ArmPoser synthesizes smartwatch-like IMU inputs by modeling watch placement, attachment variability, and gravity-aligned frame differences, then estimates arm positions, rotations, and velocities with sequential LSTMs. Training combines component MSE with auxiliary losses, while streaming estimates use five future frames and become available about 170 ms after each sample.
4.1 Data Synthesis
- Training synthesis separates four placement-and-crown configurations from bounded attachment rotations reflecting strap, anatomy, and surface variability.The configurations combine anterior or posterior placement with crown direction; attachment perturbations model fine-grained differences.Source: §4.1 · p. 4, Figure 3 · p. 4
- Rather than sample arbitrary 3D rotations, synthesis models unknown heading as gravity-axis yaw and maps the SMPL y-up frame to the device z-up frame.Unconstrained rotation sampling would create an excessively large search space and encourage reliance mainly on temporal dynamics.Source: §4.1 · pp. 4–5
4.1.3 Acceleration.
- Acceleration synthesis samples mesh vertices from the anterior or posterior wrist region, including locations proximal and distal to the nominal wrist, then uses second-order finite differences.Vertex sampling represents differences in arm length and watch position.Source: §4.1.3 · p. 5
4.2 Model
- The model predicts joint positions from smartwatch IMU sequences, then combines those positions with the IMU data to estimate joint rotations and velocities.The velocity estimator regularizes training only; inference uses predicted positions and rotations.Source: §4.2 · p. 5, Figure 4 · p. 5
4.3 Training
- Training applies mean squared error to each component, with temporal-consistency and jerk losses additionally weighted by 1×10−5 for position and rotation estimation.The model uses a 130-frame temporal window; training lasts 60 epochs, followed by 15 fine-tuning epochs.Source: §4.3 · p. 5
4.4 Real-time Inference
- Using five future frames introduces the delay: each pose is available about 170 ms after its corresponding IMU sample, while inference runs faster than the 30 Hz sensor rate.The prototype streams Apple Watch SE data to a MacBook Air M3 for inference at 30 Hz.Source: §4.4 · p. 5
5 Evaluation
The evaluation combines three public datasets with a new smartwatch dataset, and specifies device configurations, collection procedures, calibration comparisons, metrics, and dataset-specific fine-tuning and testing splits.
5.1 Proof of Concept Setup
- Evaluation uses posterior-crown-away for the left wrist and posterior-crown-toward for the right, while a wear-configuration module is assessed for supporting other placements without user input.These configurations are common in the public datasets.Source: §5.1 · p. 5
5.2 Public Datasets
- AMASS supplies synthesized training IMU data, while DIP-IMU, TotalCapture, and IMUPoser provide public evaluation data with differing sensor and calibration characteristics.For datasets with only calibrated or bone-aligned signals, perturbations are limited to 5° because converting them to smartwatch-style frames does not recreate attachment-induced bone offsets.Source: §5.2 · p. 6
5.3 ArmPoser Dataset
- The ArmPoser dataset pairs raw and calibrated consumer-watch IMU measurements with ground-truth SMPL poses to evaluate raw-signal performance and prior methods.The authors state that the dataset and code will be made available for reproducibility.Source: §5.3 · p. 6
5.3.1 Data Collection Apparatus.
- Apple Watch SE 2nd Generation and Google Pixel Watch 3 were worn side by side on the left wrist, recording gravity-aligned orientation and linear acceleration at 30 Hz.Ground-truth SMPL poses came from six calibrated RGB cameras operating at 60 Hz and were paired with the nearest IMU sample in time.Source: §5.3.1 · p. 6
5.3.2 Data Collection Procedure.
- Ten participants performed 30 everyday activities, with each activity repeated roughly five times to capture motion variation.Each session lasted about 16 minutes, yielding 160 minutes of paired data per platform.Source: §5.3.2 · p. 6
5.3.3 Calibration.
- Calibration for prior methods used chest placement to align device measurements to the SMPL frame, followed by a three-second T-pose to estimate and correct orientation offsets.Calibrated data was collected solely to evaluate prior approaches and enable comparison.Source: §5.3.3 · p. 6
5.4 Evaluation Protocol
- Evaluation reports angular error and median wrist, elbow, and average wrist–elbow positional errors, averaging left- and right-arm errors except for the left-arm-only ArmPoser dataset.The positional metrics align the shoulder before measuring the wrist, elbow, or their average.Source: §5.4 · p. 6
- Models fine-tuned on DIP-IMU are tested on DIP-IMU and TotalCapture, while models fine-tuned on IMUPoser are tested on IMUPoser and ArmPoser.The first pair uses high-grade commercial sensors; the second pair uses consumer devices.Source: §5.4 · pp. 6–7
6 Results
ArmPoser reports low pose errors across four datasets and activity types, with performance varying by dataset, activity, and device OS. Its ablations and comparisons with prior methods assess the contribution and performance of calibration-free estimation.
6.1 Arm Pose Estimation Accuracy
- Across four datasets, ArmPoser’s angular error averages 25.6 degrees and its median wrist–elbow positional error is 8.1 cm.Median elbow error is 5.81 cm and wrist error is 10.27 cm; the table evaluates TotalCapture, DIP-IMU, IMUPoser, and ArmPoser.Source: §6.1 · p. 7, Table 2 · p. 8
6.1.1 Across Activity Types.
- Across ArmPoser activities, median wrist–elbow error ranges from approximately 6.0 cm for miscellaneous activities to 8.9 cm for exercises.The heatmap shows front and top views; blue marks lower error and red higher error, while the prose describes errors as fairly uniform across the arm and activities.Source: §6.1.1 · p. 7, Figure 5 · p. 8
6.1.2 Across Device OS.
- On the ArmPoser dataset, median positional error is 8.05 cm for Android and 7.25 cm for watchOS, with watchOS performing better.The authors suggest this may reflect evaluation using a model fine-tuned on IMUPoser data collected with Apple devices.Source: §6.1.2 · p. 7
- TotalCapture has the highest cross-joint median positional error, 9.21 cm, and highest wrist error, 11.67 cm; precise object manipulation may require lower wrist error.The paper attributes the smaller advantage on TotalCapture to its research-grade sensors differing from the targeted consumer-watch setting.Source: §6.1.3 · p. 7
6.2 Ablation Studies
- Removing yaw augmentation increases wrist–elbow positional error by 7.1%, removing attachment perturbation and wrist-region sampling increases it by 14.9%, and removing both increases it by 26.8%.The ablations test augmentation for heading ambiguity and sensor-to-bone attachment variability.Source: §6.2 · p. 7
6.3 Comparison With Prior Works
- Across four datasets, ArmPoser improves median positional error by 13.2% over IMUPoser and 9.1% over MobilePoser, and angular error by 11.3% and 7.2%, respectively.It achieves the lowest positional error on all four datasets and the lowest average angular error, despite using raw smartwatch IMU rather than calibrated inputs.Source: §6.3 · p. 7
7 Wear Configuration Tracking
The Wear Configuration Tracking module infers forearm placement and crown orientation from smartwatch IMU sequences. Its measured accuracy rises with longer observation windows.
7 Wear Configuration Tracking
- The module predicts whether the watch is on the anterior or posterior forearm and whether its crown faces toward or away from the user.These are the two wear preferences the system estimates to minimize user setup.Source: §7 · p. 8
7.1 Data Collection
- The evaluation dataset covers all eight watch configurations with 10 participants, collecting two minutes of motion data per participant and configuration.Configurations combine left or right wrist, anterior or posterior placement, and crown orientation toward or away from the user.Source: §7.1 · p. 8
7.2 Model and Training
- Two bidirectional LSTMs infer forearm side and crown orientation from sequences of three-axis acceleration and 3 × 3 orientation signals.Each model average-pools outputs across the sequence to make one prediction; training uses synthesized data for four pairings of forearm side and crown orientation.Source: §7.2 · p. 8
7.3 Evaluation
- Mean configuration accuracy increases from approximately 91.8% with a 10-second window to approximately 98.7% with the maximum approximately two-minute window.The model need not observe continuously; the paper says reclassification is needed if the watch moves to a different configuration.Source: §7.3 · p. 9
8 Limitations and Future Work
The paper identifies limits in ArmPoser’s sensor scope, dataset coverage, evaluation design, global-heading recovery, and application suitability. It outlines data collection and modeling directions to address some of these boundaries.
Scope of pose estimation
- ArmPoser estimates arm pose from one smartwatch; calibration-free full-body estimation would require multiple devices and a way to maintain inter-device consistency without explicit alignment.The paper proposes learning from synthetic data in each device’s local frame with kinematic constraints as a possible direction.Source: §8 · p. 9
Dataset coverage
- The dataset uses mostly young participants, short scripted sessions, and only left-wrist recordings, leaving transfer to broader real-world conditions unassessed.The paper calls for longer studies with more diverse participants and varied body shape, watch size, clothing, sweat, strap tightness, and repeated donning.Source: §8 · p. 9
Evaluation design
- Pose estimation and wear-configuration recognition were evaluated separately, so their combined effect on pose accuracy remains unmeasured.An end-to-end evaluation requires synchronized smartwatch and camera recordings across configurations, including natural donning and transitions.Source: §8 · p. 9
Heading
- ArmPoser estimates arm motion relative to the body but does not recover absolute global heading, limiting applications such as navigation.The paper suggests magnetometer measurements when magnetic interference is low, with inertial priors when they are unreliable.Source: §8 · p. 10
Application boundaries
- A 170 ms delay and 9–12 cm wrist error may suit coarse feedback and gesture input but not direct XR manipulation or small-target pointing.The authors call for task-level evaluations to establish these boundaries.Source: §8 · p. 10
9 Conclusion
ArmPoser estimates arm pose in real time from a single smartwatch IMU without user calibration, learning in the watch’s native gravity-aligned frame. On public benchmarks and a newly collected dataset, it matches or exceeds calibrated baselines.
9 Conclusion
- ArmPoser is a real-time arm-pose estimation system whose evaluations on public benchmarks and a newly collected dataset show it matches or exceeds calibrated baselines using one smartwatch and no user calibration.Source: §9 · p. 10
- Learning in the smartwatch’s native gravity-aligned frame eliminates global-frame alignment, sensor-to-bone offset estimation, and per-session calibration.Source: §9 · p. 10
- A lightweight wear-configuration module estimates anterior or posterior placement and crown orientation.Source: §9 · p. 10
A Deriving Bone Orientation and Vertex Acceleration
The appendix summarizes how standard procedures derive joint orientations and vertex positions from body-model parameters, and derive linear acceleration from vertex motion.
A.1 Bone Orientation
- Forward kinematics uses pose parameters, shape parameters, and global translation to compute global joint orientations and mesh-vertex positions.Source: §A.1 · p. 11
- For smartwatch data synthesis, the wrist joint’s global orientation is used.Source: §A.1 · p. 11
A.2 Vertex Acceleration
- Linear acceleration is synthesized from mesh-vertex motion using second-order finite differences, with vertices sampled from a predefined wrist-region set.Δt is the frame interval.Source: §A.2 · p. 11
A.3 Use in Data Synthesis
Canonical orientation and acceleration signals in the body-model reference frame are used to synthesize smartwatch IMU measurements. The manuscript also discloses generative AI assistance with wording.
A.3 Use in Data Synthesis
- Canonical orientation and acceleration signals are expressed in the body-model reference frame.Source: §A.3 · p. 11
- Mounting and frame transformations convert the canonical signals into synthetic smartwatch IMU measurements.Source: §A.3 · p. 11
B Usage of Generative AI
- Generative AI tools assisted with manuscript clarity, grammar, and phrasing; the authors reviewed and edited the content and take responsibility for the final text.Source: B Usage of Generative AI · p. 11