PaperlayerView exact revision on arXiv

Source-linked AI summary

Automatic Posture and Movement Tracking of Infants with Wearable Movement Sensors

Manu Airaksinen, Okko Räsänen, Elina Ilén, Taru Häyrinen, Anna Kivi, Viviana Marchi, Anastasia Gallen, Sonja Blom, Anni Varhe, Nico Kaartinen, Leena Haataja, Sampsa Vanhatalo

arXiv:1909.09823v2cs.CVcs.LGstat.ML

TL;DR

Early recognition of infants at risk for neurodevelopmental disorders is difficult to scale with existing screening approaches. The study uses a sensor-equipped jumpsuit, human annotations, and machine learning to track infant posture and movement; its classifier reaches human-like consistency, while movement annotations remain more ambiguous than posture annotations.

Abstract

from arXiv · show

Infants' spontaneous and voluntary movements mirror developmental integrity of brain networks since they require coordinated activation of multiple sites in the central nervous system. Accordingly, early detection of infants with atypical motor development holds promise for recognizing those infants who are at risk for a wide range of neurodevelopmental disorders (e.g., cerebral palsy, autism spectrum disorders). Previously, novel wearable technology has shown promise for offering efficient, scalable and automated methods for movement assessment in adults. Here, we describe the development of an infant wearable, a multi-sensor smart jumpsuit that allows mobile accelerometer and gyroscope data collection during movements. Using this suit, we first recorded play sessions of 22 typically developing infants of approximately 7 months of age. These data were manually annotated for infant posture and movement based on video recordings of the sessions, and using a novel annotation scheme specifically designed to assess the overall movement pattern of infants in the given age group. A machine learning algorithm, based on deep convolutional neural networks (CNNs) was then trained for automatic detection of posture and movement classes using the data and annotations. Our experiments show that the setup can be used for quantitative tracking of infant movement activities with a human equivalent accuracy, i.e., it meets the human inter-rater agreement levels in infant posture and movement classification. We also quantify the ambiguity of human observers in analyzing infant movements, and propose a method for utilizing this uncertainty for performance improvements in training of the automated classifier. Comparison of different sensor configurations also shows that four-limb recording leads to the best performance in posture and movement classification.

Introduction

Early recognition of infants at risk for neurodevelopmental difficulties is important, but scalable screening remains difficult. This work motivates spontaneous movement assessment as a route toward more objective, quantitative tracking.

Introduction

  • More than one in ten infants is considered at neurodevelopmental risk, while extensive screening is impractical in many settings and unjustifiable even in wealthy nations.The challenge is targeting early intervention to infants who would benefit from it.Source: Introduction · pp. 1–2
  • Screening methods need to be scalable, objective, effective, and robust to variation in infants, professionals, environments, recording methods, and typical development.Source: Introduction · p. 2
  • Existing infant observation protocols are neither genuinely quantitative nor suitable for longitudinal tracking beyond four months of age.Spontaneous movement may provide a window into brain function and could support early screening.Source: Introduction · p. 2
  • The paper presents an infant smart jumpsuit, a visual annotation protocol, and an automated classifier, reporting posture and movement recognition comparable to human raters.Source: Introduction · p. 2

Data collection methods

The study combines a four-sensor infant garment and mobile data logging with video-based annotation and machine-learning analysis. It collected recordings from typically developing infants and developed a scheme for classifying posture and gross movement.

Data collection methods

  • The workflow comprised wearable development and mobile data collection, visual analysis for a human benchmark, and machine-learning development, training, and testing.Source: Data collection methods · p. 2

Design of the smart jumpsuit

  • The smart jumpsuit is a full-body garment with four wireless sensors placed proximally on the upper arms and legs to capture infant movement.Sensors record accelerometer and gyroscope signals at 52 Hz; the logging software streams and synchronizes data and can record video concurrently.Source: Design of the smart jumpsuit · pp. 2–3, Figure 1 · p. 3
  • The jumpsuit is a technical proof of concept rather than a ready commercial product.Its current retail-component construction costs approximately 200 USD; larger-scale production could reduce garment cost to 30–50 USD.Source: Design of the smart jumpsuit · p. 3

Data collection

  • After two infants with substantially advanced motor skills were excluded, the study analyzed 22 recordings totaling 12.1 hours, of which 10.6 hours were used for classifier training and testing.Infants were typically developing and recorded in clinic-like sessions; physical contact was limited to soothing or lifting when needed.Source: Data collection · pp. 3–4

Movement annotation protocol

  • The annotation scheme separates posture from gross movement, covering five posture categories and seven analyzed movement categories for infants aged three to seven months before upright posture.Crawl 4 limbs was omitted because it appeared in only one recording; distal fine-motor movements were excluded because they could not be identified consistently at the desired temporal resolution.Source: Movement annotation protocol · pp. 4–5
  • Three authors independently annotated each recording using synchronized video and multichannel waveforms, with annotator consistency assessed through inter-rater agreement.The scheme was developed iteratively by clinical and machine-learning experts; presenting synchronized waveforms was found to improve annotation timing accuracy.Source: Movement annotation protocol · pp. 4–5

Annotation consistency analysis

Human agreement was high for posture annotations but moderate for movement, with substantial class-specific confusion. Signal-feature visualization likewise showed clearer posture boundaries than movement-category boundaries.

Annotation consistency analysis

  • Posture annotations had very high overall agreement (Fleiss’ κ = 0.923), while movement annotations had moderate agreement (κ = 0.580), with substantial variability between classes.Confusions included prone with crawl or side postures, and macro still or crawl proto with other movement classes.Source: Annotation consistency analysis · p. 5, Figure 2 · p. 6
  • The t-SNE visualization showed clearly separated posture categories but ambiguous movement boundaries, with movement-category mini-clusters forming a continuum.This pattern illustrates that posture classification is relatively easy compared with accurate movement categorization.Source: Annotation consistency analysis · p. 5, Figure 2 · p. 6
  • Annotators reported uncertainty about movement boundaries and transient actions, and suggested that their category criteria could vary with an infant’s overall activity level.The authors relate these disagreements to temporal contiguity and infants’ ongoing continuous motor development.Source: Annotation consistency analysis · pp. 5–6

Automatic detection of infant posture and movement

The study compares an engineered-feature SVM with an end-to-end CNN for infant posture and movement classification, using iterative annotation refinement to address inconsistent training labels. Evaluation uses leave-one-subject-out cross-validation and compares classifier performance with human annotations and agreement.

Training of automatic classifiers for posture and movement

  • The CNN extracts sensor-level features, fuses them into frame-level features, and models temporal dependencies; movement classification also receives posture predictions as context.The SVM provides a baseline using established signal-level features, while the CNN learns task-relevant representations from the data.Source: Training of automatic classifiers for… · p. 6, CNN classifier · pp. 7–8
  • The SVM uses 336 features per frame, calculated from 14 signal features per channel after windowing the signal into 120-sample frames with 50% overlap.At 52 Hz, each frame spans 2.3 seconds.Source: SVM classifier · p. 7
  • About one third of movement-track frames lacked full agreement among three experts, motivating iterative annotation refinement (IAR) to combine human labels with classifier likelihoods.IAR updates class probabilities while retaining the original human annotations as priors; unambiguous frames remain unchanged while ambiguous labels can change.Source: Iterative annotation refinement for… · p. 8

Experimental setup and evaluation

  • Performance was evaluated with leave-one-subject-out cross-validation, applying five IAR iterations only to training annotations and testing against original human annotations.The held-out infant’s samples were not refined.Source: Experimental setup and evaluation · p. 8
  • CNN performed significantly better than SVM for prone-position movement patterns including crawl proto, turn, and pivot.Figure 4 presents class-specific F-score box plots for individual recordings.Source: Figure 4 · p. 9
  • The evaluation reports accuracy, unweighted average recall, precision, and F-score, and calculates pairwise κ to assess machine consistency with human annotators.The primary benchmark distinguishes full-agreement frames from all frames, the latter labeled by class majority vote.Source: Experimental setup and evaluation · p. 9

Results for automatic classification

The CNN classifier reached human-equivalent agreement on posture and movement, with IAR label refinement improving performance. The results also compare CNN and SVM classifiers, sensor configurations, and human and classifier movement distributions.

Assessment of classifier performance

  • CNN movement median F-score was approximately 80%, while SVM performance was consistently 5 to 10 percentage points lower.Posture classification was generally comparable between the classifiers; SVM was notably worse for prone movements including crawl commando, crawl proto, and pivoting.Source: Assessment of classifier performance · p. 9
  • CNN confusion matrices were highly similar across full-agreement and all-frame sets, and comparable to human annotation matrices.Frequent posture confusions involved prone, side, and supine; movement decisions were challenging around macro still, crawl proto, and other categories.Source: Table 1 · p. 10, Figure 5 · p. 10

Assessment of IAR effectiveness on annotation consistency and classifier performance

Added value from multi-sensor setup

  • Individual sensors performed below the four-sensor setup, while one-arm-plus-one-leg configurations may achieve comparable UAR.The two-sensor setup may be more susceptible to placement errors or hardware failures and cannot measure laterality differences in limb synchronies.Source: Figure 7 · p. 11, Added value from multi-sensor setup · p. 12

Automatic recognition of infants with different levels of motor performance

  • Movement distributions from human annotations and classifier outputs did not differ significantly in the 22-infant comparison.The figure compares category distributions and individual-level agreement between human and classifier outputs.Source: Figure 8 · p. 12

Discussion

The study reports human-comparable infant posture and movement classification with a wearable, alongside an annotation scheme and an approach to use human annotation uncertainty. Practical use requires balancing synchronized multisensor recording and setup complexity, while future work could apply tracking to posture-specific analysis and neurodevelopmental monitoring.

Discussion

  • The study introduced categories for infant postures and movements, an approach to exploit inter-rater inconsistencies, and found that multisensor recording supports maximal movement-classification performance.The work describes a wearable intended for non-intrusive infant posture and independent-movement measurement, potentially in out-of-hospital clinical research recordings.Source: Discussion · p. 13
  • The CNN achieved classification performance comparable to human annotator consistency, while outperforming an SVM that struggled with crawling posture and pivoting.The SVM was sufficient for posture tracking and some movement categories; the CNN yielded statistically significant improvements in movement tracking.Source: Discussion · p. 13

Practical aspects of jumpsuit analysis

  • The swim-suit-based design placed sensors proximally to support comfort and undisturbed movement; piloted distal placements added little benefit and drew infants' attention.Distal sensors were also mechanically less stable.Source: Practical aspects of jumpsuit analysis · p. 13
  • Reliable synchronized data collection, operator usability, and reduced setup complexity remain practical requirements for the multisensor system.The setup required high-rate wireless data collection over Bluetooth; additional sensors increased movement-pattern recognition accuracy, although posture detection alone could be relatively reliable from one sensor.Source: Practical aspects of jumpsuit analysis · p. 13

Future prospects

  • Posture and movement tracking could support context-specific analyses, including assessing hand or leg movements within posture contexts and measuring symmetry or limb synchrony.The passage notes that infant intentional hand or leg movements are heavily modulated by posture, while no method was available to analyze them separately by posture context.Source: Future prospects · p. 14
  • The authors report proof-of-concept differentiation of higher- and lower-performing infants by movement-category incidence, and identify atypical motor-pattern tracing and intervention-efficacy evidence as potential applications.They describe functional growth charts and tummy time as trends that could benefit from automated posture and movement tracking.Source: Future prospects · p. 14

Related papers

Loading 1909.09823v2…