PaperlayerView exact revision on arXiv

Source-linked AI summary

Human movement reconstruction through latent structural representations during assisted transitions and multidirectional hopping

Guilherme Manna Cesar, Ligia Yumi Mochida, Chady Ghnatios

arXiv:2610.10055v1cs.CE

TL;DR

Atypical posture, physical assistance, and rapid movement challenge 3D kinematic quantification, while video-derived trajectories must retain biomechanical information beyond landmark detection. The paper uses rank-constrained latent representations with feedforward marker-coordinate regression; across test repetitions, reconstruction recovered dominant displacement profiles and sagittal-plane angular trajectories, while cyclograms showed both preserved hip-knee relationships and limitations in reproducing individual patterns.

  • Problem

    Atypical posture, physical assistance, and rapid movement make 3D kinematic quantification difficult, and video-derived trajectories must retain biomechanical information beyond landmark detection.

    Source: Discussion · p. 18

  • Method

    The framework combines rank-constrained latent representations with feedforward marker-coordinate regression to reconstruct movement from synchronized video.

    Source: Discussion · p. 18

  • Results

    Across test repetitions excluded from model training, reconstruction recovered dominant displacement profiles and sagittal-plane angular trajectories, while cyclograms showed preserved hip-knee relationships but limitations in reproducing individual patterns.

    Source: Discussion · p. 18

  • Takeaways & Limitations

    Movement organization can be an explicit reconstruction outcome, judged through both recovered trajectories and the biomechanical information they retain.

    Source: Discussion · p. 22

    The authors frame this as a proof of concept, with relatively small errors needing to be considered alongside preservation of coupled movement.

    Source: Discussion · pp. 18, 21

Abstract

from arXiv · show

The quantification of three-dimensional human kinematics is fundamental to neurorehabilitation and musculoskeletal research, yet atypical posture, physical assistance, and rapid movement create conditions of partial observability. Here, we evaluate a latent-structure-informed framework combining convolutional rank-reduction autoencoding with feedforward regression to reconstruct movement from synchronized video. Three case-based demonstrations span therapist-assisted sit-to-stand in a child with cerebral palsy, unassisted sit-to-stand in a typically developing child, and multidirectional single-leg hopping in a young adult. Reconstruction was examined through positional trajectories, projected angular descriptors, and hip-knee coupling. Across test repetitions excluded from model training, mean absolute errors in a predicted marker position during assisted sit-to-stand ranged from 12.02 to 57.11 mm. Sagittal knee and trunk descriptor errors were 5.32{\deg} and 5.38{\deg} in the typically developing case. Hip-knee cyclograms captured aspects of coupled-motion patterns in both cases, with closer trajectory correspondence in the typically developing case. Sixteen test hops yielded a vertical knee-marker mean absolute error of 15.58 mm. These findings connect latent-structure-informed reconstruction with biomechanically interpretable movement features, providing a foundation for video-derived assessment spanning clinically atypical movement and high-dynamic athletic movement.

Introduction

This study evaluates a rank-constrained video-to-marker framework for reconstructing positional and biomechanical movement features across case demonstrations of assisted and unassisted sit-to-stand and multidirectional hopping. The case-specific, held-out evaluations use task-relevant error and trajectory measures, with the demonstrations’ scope and limitations considered alongside recovered movement patterns.

Introduction

  • 3D kinematics support musculoskeletal research, neurorehabilitation, and clinical decisions, but marker visibility and protocol demands challenge measurement in children with disabilities and during rapid, self-occluding movement.These constraints motivate reconstruction that relates available visual information to articulated-body configuration rather than evaluating landmark visibility alone.Source: Introduction · pp. 2–3
  • Markerless methods can miss individual movement deviations, including pelvic tilt and transverse hip rotation in cerebral palsy gait [13] and out-of-plane pediatric sit-to-stand patterns [6].The problem extends beyond detecting landmarks to retaining relationships among body segments that distinguish movement strategies.Source: Introduction · p. 4
  • The framework combines view-specific, rank-constrained latent representations with feedforward marker-coordinate regression, then assesses positional, projected-angular, and coupled-motion features.Effective latent rank is defined by truncated singular value decomposition; “structure” denotes low-rank organization, not an explicitly encoded anatomical graph.Source: Introduction · pp. 4–5
  • The case demonstrations span therapist-assisted sit-to-stand in a child with cerebral palsy, unassisted sit-to-stand in a typically developing child, and multidirectional single-leg hopping in a young adult.Synchronized marker-based measurements provided held-out reference data; training used five participants, while reported test outcomes came from three participants with repetitions analyzed within each case.Source: Introduction · p. 6, Results · p. 6

Results

  • The rank-reduction component defines effective latent dimensionality by retained matrix rank and projects newly encoded observations onto a training-derived basis at inference.View-specific latent representations support image reconstruction and feed a multilayer perceptron for marker-coordinate prediction; joint training uses image-reconstruction, perceptual, and marker-coordinate losses.Source: Results · p. 6

Assisted sit-to-stand reconstruction differs across spatial components

  • 27.00, 59.47, and 17.76 mm were the right-back marker’s X-, Y-, and Z-axis MAEs across four assisted sit-to-stand tests.The Z trajectory had the strongest agreement (R² = 0.835), while the Y prediction retained the temporal progression and trajectory pattern despite lower positional agreement.Source: Assisted sit-to-stand reconstruction… · p. 7, Figure 3 · p. 8

Unassisted sit-to-stand reconstruction is more consistent for sagittal descriptors

  • Absolute errors for sagittal-, frontal-, and transverse-plane projected thigh-shank descriptors were 5.65°, 4.46°, and 14.55°, respectively, with corresponding R² values 0.944, 0.425, and 0.872.Source: Unassisted sit-to-stand reconstruction… · p. 9

Hip-knee cyclograms reveal differences not captured by pointwise error alone

  • 6.16° and 3.38° were the hip and knee flexion-proxy MAEs in the cerebral-palsy case, with a 7.60° mean paired-angle discrepancy.Hip and knee R² values were 0.422 and 0.423; recognizable coupled-motion features remained, but excursion and local trajectory geometry differed, and paired data covered 80.4% of required samples.Source: Hip-knee cyclograms reveal differences… · p. 13
  • 7.17° and 4.67° were the hip and knee flexion-proxy MAEs in the typically developing case, with a 9.41° mean paired-angle discrepancy.Hip and knee R² values were 0.879 and 0.960, with 100% paired-data availability; the cyclograms more closely reproduced the broad reference trajectory across three test repetitions.Source: Hip-knee cyclograms reveal differences… · p. 16

profile

  • The hopping evaluation assessed agreement between reference and predicted toe-marker-derived flight times; at 50 Hz, one frame difference can produce a 20 ms bias.The supplied passages identify the flight-time measures and their table, but do not provide its numerical results.Source: profile · pp. 16–17, Table 1 · p. 17
  • Across 16 test hops, the left-knee marker Z-coordinate had a mean absolute error of 20.96 mm, R² of 0.760, and Pearson correlation of 0.878.The predicted trajectory reproduced the dominant rise-and-fall pattern, although individual repetitions differed in amplitude.Source: Figure 6 · p. 17, profile · p. 17

Discussion

  • Across held-out repetitions, the framework recovered dominant displacement profiles and sagittal angular trajectories, while cyclograms showed both preserved hip-knee relationships and limits in individual coordination.The case-based evaluation spans clinically atypical therapist-assisted pediatric movement and high-speed multidirectional hopping.Source: Discussion · p. 18
  • Assisted sit-to-stand reconstruction provides an initial basis for studying movement with physical support retained, but detecting rehabilitation-related change requires longitudinal evaluation.The discussion also cautions that case differences in assistance, posture, movement excursion, and training samples prevent attributing cyclogram discrepancies to cerebral palsy alone.Source: Discussion · pp. 19–20
  • The typically developing case recovered hip-knee coupling, whereas the cerebral palsy case had smaller paired-angle discrepancy but weaker trajectory fit and more pronounced shape differences.Overall transition reconstruction does not by itself establish preservation of an individual coordination strategy.Source: Discussion · p. 19
  • In an assisted sit-to-stand repetition, the predicted hip-knee cyclogram remained continuous during an interval when the marker-derived trajectory was discontinuous.This extends reconstruction beyond individual coordinate prediction to a continuous relationship between angular quantities during assisted movement.Source: Discussion · p. 20
  • The framework did not reproduce marker-derived frontal-plane excursions to the same extent, and accuracy within reference-unavailable intervals remains independently unestablished.Temporal smoothing was part of the demonstrated pipeline.Source: Discussion · p. 20
  • Each test condition involved one participant; the separately trained task datasets demonstrate distinct conditions, not transfer, and component contributions remain unresolved.The demonstrations examine repeated movements within cases rather than between-participant generalizability.Source: Discussion · p. 21
  • The study’s case-based contribution is to recover task-relevant movement features while treating the preservation of their relationships as an explicit part of reconstruction assessment.Computational performance is judged both by recovered trajectories and by the biomechanical information they retain.Source: Discussion · p. 22

Ethical oversight and participant selection

  • Five participants were included—two children with cerebral palsy, two typically developing children, and one young adult—to examine cases rather than estimate population-level performance.The cases represented assisted pediatric movement, an unassisted pediatric comparator, and a high-dynamic task.Source: Ethical oversight and participant… · p. 22

Task-specific data acquisition

  • Sit-to-stand included hand-held assistance and additional hip support for the child with cerebral palsy, while hopping used single-leg multidirectional sequences on a 3 by 3 grid.Assistance was part of the task condition, not assumed mechanically neutral; each complete hopping sequence contained 16 individual hops.Source: Task-specific data acquisition · pp. 22–23

Motion capture and synchronization

  • Marker trajectories were recorded at 100 Hz and synchronized video at 50 Hz from two complementary views using a trigger and common time base.The views accommodated examiner assistance.Source: Motion capture and synchronization · p. 23

Reference trajectories and kinematic descriptors

  • Outcomes included recorded marker coordinates and projected inter-segment angle descriptors; these were geometric measures, not anatomical joint-rotation estimates.Hip and knee flexion proxies were also used to construct cyclograms.Source: Reference trajectories and kinematic… · pp. 23–24

Video preprocessing and computational framework

  • Video streams were aligned and cropped to task-specific windows, with participant and examiner regions included when applicable; motion-sensitive background-subtraction channels supplemented RGB inputs.Windows covered initiation, ascent, and stabilization for sit-to-stand, and take-off, flight, and landing for hopping.Source: Video preprocessing and computational… · p. 24
  • Each video view used a pretrained convolutional RRAE to produce rank-constrained latent features, which were concatenated and passed to an MLP to predict 3D marker coordinates.Truncated singular value decomposition constrained effective latent rank, and inference projected new encoded observations onto a basis derived from training data. The reported pipeline included post hoc temporal smoothing.Source: Video preprocessing and computational… · p. 25

Data partitioning and scope of evaluation

  • The computational datasets contained 8 cerebral palsy, 4 typically developing, and 8 hopping recordings; one recording per condition was held out for testing, and separate condition-specific models were constructed.Source: Data partitioning and scope of… · p. 25

Error metrics and descriptive analysis

  • Mean absolute error measured coordinate- or descriptor-specific discrepancy, while R² and Pearson correlation described complementary aspects of trajectory fit and association.Errors were kept separate by outcome and unit; the paired-angle cyclogram discrepancy is a distance in a two-angle representation, not positional error in millimeters.Source: Error metrics and descriptive analysis · p. 26
  • Analyses were descriptive, with test outcomes from three individual cases among five participants; no population-level hypothesis testing was performed.Paired-data availability was 80.4% for the cerebral palsy test cyclogram and 100% for the typically developing test cyclogram; these percentages describe samples entering paired comparisons, not global body or view coverage.Source: Error metrics and descriptive analysis · p. 26

Data availability

  • De-identified data are not publicly available but may be requested from the corresponding author, subject to institutional review, ethical requirements, and appropriate agreement.Informed consent and institutional approvals did not include unrestricted public sharing.Source: Data availability · p. 26

Code availability

  • Custom computational-analysis code will be made publicly available in a permanent repository upon publication and is available from the corresponding author during peer review upon reasonable request.Source: Code availability · p. 26

Supplementary Information

The supplementary information documents the video-to-coordinate reconstruction workflow, recording and acquisition scope, and outcome-specific training and test errors. It also details evaluation and paired-data procedures, including descriptive statistical framing and task-specific handling of missing data and trajectory gaps.

Supplementary Note 1 | Organization of recordings and analyzed repetitions

  • Displayed pediatric cases involved two participants (one training-only and one reported with train/test data), while adult hopping involved one; repetition counts are not biological-sample counts.For the CUBE sequence, each outward hop and return to center was treated as a separate movement.Source: Supplementary Note 1 | Organization of… · p. 31

Supplementary Note 2 | Training reconstruction

  • Training MAEs were 2.67, 6.40, and 3.13 mm for CP back-marker X, Y, and Z, and 5.98 mm for adult LKNE Z versus 20.96 mm in testing.These training-to-test differences do not establish participant-level generalization.Source: Supplementary Note 2 | Training… · p. 32

Supplementary Note 3 | Outcome-specific test performance

  • Supplementary Table 3 reports test-set errors of 5.65° for TD right-knee sagittal and 5.38° for TD trunk sagittal, plus adult LKNE Z 20.96 across 16 individual hops.Source: Table 3 · p. 33, Supplementary Note 3 |… · p. 34
  • Cyclogram proxy test MAEs were 6.16° (CP right hip), 3.38° (CP right knee), 7.17° (TD left hip), and 4.67° (TD left knee).Pair discrepancies were 7.60° (CP; 80.4% availability) and 9.41° (TD; 100% availability); the analyses included n=4 and n=3 repetitions, respectively.Source: Table 3 · p. 33, Supplementary Note 3 |… · p. 34

Acquisition, calibration, and temporal alignment

Reference processing and geometric definitions

  • Knee descriptors were unsigned projected thigh–shank angles rather than anatomical joint-coordinate rotations; missing-marker time steps were retained as prediction-only observations.Positional outcomes, projected knee descriptors, and hip- and knee-flexion proxies were evaluated as separate quantities.Source: Reference processing and geometric… · p. 35

Video representation and model implementation

  • Each RGB view was paired with a background-removed RGB view, concatenated into a six-channel input tensor for its corresponding rank-reduction autoencoder.Source: Video representation and model… · p. 35
  • The RRAE used truncated SVD to retain a rank-constrained latent representation during training; at inference, test observations were projected onto the training-derived basis before coordinate regression.Source: Video representation and model… · pp. 35–36
  • One recording per condition was held out for testing; remaining sequences were split 80% for training and 20% for validation, separate models were fitted per dataset, and predictions were temporally filtered before biomechanical evaluation.The reported implementation used Python 3.11.9 and TensorFlow 2.16.2.Source: Video representation and model… · p. 37

Error aggregation and paired-data availability

  • MAE, R², Pearson’s r and Spearman’s ρ were calculated on paired finite samples pooled across repetitions, so repetitions contributed according to their valid-sample counts rather than mean curves or repetition-level metrics.Source: Error aggregation and paired-data… · p. 37
  • No temporal-autocorrelation correction was applied, and repeated movements within a participant were not independent biological replicates, so results were treated as descriptive.Source: Error aggregation and paired-data… · p. 37
  • Cyclogram discrepancy was the mean Euclidean distance between paired hip-knee points with finite reference and predicted flexion-proxy values.Source: Error aggregation and paired-data… · p. 37
  • Paired-data availability measured included normalized samples against all normalized samples, was specific to the task’s considered markers rather than global occlusion, and used no prediction-confidence threshold.Source: Error aggregation and paired-data… · p. 37
  • Sit-to-stand trajectories were normalized over 0–100% and compared only across contiguous finite segments, without interpolation across missing-data gaps.Missing marker measurements propagated to nonfinite reference descriptors.Source: Error aggregation and paired-data… · p. 37

Supplementary figures

The supplementary figures show training-set reference and predicted trajectories across marker positions, angular descriptors, hip-knee cyclograms, and multidirectional hops. They also document flight-status processing and the encoder-decoder-regressor architecture.

Supplementary figures

  • Five training sit-to-stand repetitions show reference and predicted RBAK coordinates across the X, Y, and Z components.Training fit is reported separately from the test reconstruction in main Fig. 3.Source: Supplementary figures · p. 39, Figure 1 · p. 39

Supplementary Figure 1a

  • Four training repetitions show aggregate left-knee thigh-shank descriptors in sagittal, frontal, and transverse planes, not a same-limb comparison with the right-sided test panels.Solid curves are references and dashed curves are predictions.Source: Supplementary Figure 1a · p. 41, Figure 2 · p. 41

Supplementary Figure 2a

  • Four training repetitions show sagittal, frontal, and transverse trunk descriptors, distinct from back-marker position and knee descriptors.Solid curves indicate marker-based references and dashed curves predictions.Source: Supplementary Figure 2a · p. 43, Figure 3 · p. 43

Supplementary Figure 3a

  • Right-sided hip-knee flexion-proxy cyclograms across five training repetitions report hip MAE 0.53°, knee MAE 0.73°, paired-angle discrepancy 0.86°, and 62.4% paired-data availability.The summary applies only to samples entering the analysis, not all frames or occlusion intervals.Source: Supplementary Figure 3a · p. 45, Figure 4 · p. 45

Supplementary Figure 4

  • Left-sided hip-knee flexion-proxy cyclograms across four training repetitions report hip MAE 1.96°, knee MAE 1.59°, paired-angle discrepancy 2.67°, and 100% paired-data availability.Solid curves show references and dashed curves predictions.Source: Supplementary Figure 4 · p. 46, Figure 5 · p. 46

Supplementary Fig. 6 | Training knee-marker reconstruction during multidirectional

  • Training knee-marker reconstruction during multidirectional hopping had an LKNE Z-coordinate MAE of 5.984 mm and R² of 0.9823 across 16 training hops.The curves compare marker-based references with predictions; hops are components of one participant’s recorded sequence.Source: Supplementary Fig. 6 | Training… · p. 47

Supplementary Figure 6

  • Flight status is post-processed from reference and prediction data using the same algorithm, with ground level identified from the toe marker’s minimum Z coordinate.The in-flight criterion is a toe-marker Z coordinate 1 mm above the identified ground level.Source: Supplementary Figure 6 · p. 47, Figure 7 · p. 47
  • The model diagram shows a view-specific convolutional encoder, a decoder reconstructing a six-channel image representation, and a feedforward multilayer perceptron mapping concatenated representations to coordinate outputs.Conv2D denotes two-dimensional convolution; Conv2DTranspose denotes transposed two-dimensional convolution.Source: Supplementary Figure 6 · p. 50, Figure 8 · p. 51

Related papers

Loading 2610.10055v1…