Source-linked AI summary
The Blind Spot in 2D Infants' Pose Estimation:Robust Learning from Noisy Annotations
Emanuele Cardinale, Marco Proietti, Alessandro Cacciatore, Maria Francesca Spadea, Lucia Migliorelli, Sara Moccia
TL;DR
Noisy annotations are a largely underexplored problem in pose estimation, especially for preterm-infant images where visual ambiguities make labeling difficult. The paper introduces REMIND, which clusters keypoint-wise training-dynamics features to identify unreliable annotations, and reports strong noisy-keypoint identification and preservation of pose-estimation performance. The authors position the approach as a step toward reliable infant monitoring when annotation quality is uncertain.
Problem
Noisy annotations can impair pose-estimation learning, yet noisy-label robustness has received limited attention in pose estimation despite difficult preterm-infant imaging conditions.
Method
REMIND uses keypoint-wise loss trajectories and clustering to identify noisy annotations, then excludes them from loss computation without discarding whole samples.
Results
REMIND consistently outperforms the Small-Loss trick across tested noise configurations and pose-estimation models, with AUC values ranging from 93.3% to 97.7%.
Takeaways & Limitations
REMIND limits the average mAP drop under noisy training to 1.8% ± 1.2%, compared with 22.5% ± 10.7% when training with noisy samples.
Takeaways & Limitations
The NeoPose dataset is limited in size and collected from a single clinical center, motivating planned multi-center validation.
Abstract
from arXiv · showhide
Noisy annotations pose a significant challenge for supervised deep learning, as neural networks rely on large-scale, high-quality labeled data whose corruption can severely impair model performance. Although robustness to label noise has been extensively studied for classification tasks, it remains relatively underexplored in Pose Estimation (PE). This limitation becomes critical in clinical contexts, including neonatology, where PE of preterm infants is used to support the assessment of spontaneous motility, a key indicator of neurodevelopmental trajectories. In such settings, infants' images labeling is further hindered by visual challenges (e.g., keypoint self-occlusions, caregiver interference), making the annotation process inherently susceptible to errors. To tackle noisy annotations in PE, we introduce REliable keypoint selection via Memory of traINing Dynamics (REMIND), a clustering-based keypoint-selection strategy that exploits keypoint-wise training dynamics to identify noisy labels without assuming any prior knowledge of the noise distribution, thus enabling noise-free model training. When evaluated on the proprietary NeoPose dataset, comprising 46 videos of 46 preterm infants recorded in real clinical settings, REMIND correctly identifies noisy annotations across multiple corruption scenarios, achieving up to 93\% Area Under the Curve (AUC) with three different PE architectures used in the relevant literature. To our knowledge, this is the first study to explicitly address label noise in preterm infants' PE, paving the way for the design of trustworthy learning-based algorithms for infants'monitoring support when data quality cannot be guaranteed.
1. Introduction
Infant pose estimation could support clinical assessment of spontaneous movements, but neonatal images are difficult to annotate reliably and noisy-label methods remain underexplored for pose estimation. The paper introduces REMIND and validates it on NeoPose as a keypoint-level strategy for identifying unreliable annotations.
- Motivation: Vision-based infant pose estimation is being explored as an unobtrusive aid for assessing spontaneous movements and supporting clinicians.Spontaneous-movement quality provides insights into nervous-system development, while current expert observation is subjective, training-intensive, and time-consuming.
- Motivation: Annotation quality can degrade pose-estimation performance because labeling is repetitive, error-prone, and difficult under neonatal visual ambiguities.Self-occlusions, medical devices, casts, and caregivers’ or practitioners’ hands can hinder accurate keypoint localization.
- REMIND: REMIND uses the temporal evolution of keypoint-wise loss values to identify noisy annotations without predefined filtering thresholds.The strategy is unsupervised and operates at the keypoint level rather than discarding entire samples.
- Evaluation: The study validates REMIND on NeoPose, a dataset of preterm-infant videos acquired during routine clinical practice.The authors describe NeoPose as currently among the largest datasets focused specifically on hospitalized preterm infants.
2. Related work
Related work spans infant-specific adaptation of adult pose-estimation models and broader noisy-label learning based on training dynamics. REMIND is situated at their intersection by using keypoint-wise training information for infant pose-estimation data.
- Infant pose estimation: Research has benchmarked adult-trained pose-estimation models on infant images, including OpenPose, HRNet, AlphaPose, and ViTPose.Performance typically degrades on infants because of domain differences, motivating fine-tuning and domain-adaptation strategies.
- Infant pose estimation: Infant pose-estimation studies have used specialized datasets, fine-tuned adult backbones, and infant-specific keypoint representations.Examples include datasets with approximately 4,500 annotated frames or 20,748 labeled images and a clinician-defined 21-keypoint representation.
- Learning with noisy labels: Noisy-label research exploits the tendency of deep networks to learn clean patterns before eventually fitting incorrect annotations.This memorization effect motivates sample-selection methods based on early losses or broader training dynamics.
- Learning with noisy labels: The Small-Loss hypothesis treats consistently low-loss samples as more likely to be correctly labeled.REMIND is presented against this background as a strategy for noisy annotation handling in pose estimation.
3. Methods
REMIND identifies potentially noisy keypoint annotations from their training-loss dynamics, clusters them without predefined filtering thresholds, and excludes the noisy cluster from subsequent loss computation. The method is evaluated under synthetic Gaussian perturbations on NeoPose and across multiple pose-estimation models and datasets.
- REMIND score: REMIND records each keypoint’s loss at every training epoch and smooths the resulting trajectory to reduce stochastic fluctuations.Clean keypoints typically show decreasing loss, whereas noisy keypoints show limited reduction and more irregular trajectories.
- REMIND score: The REMIND score combines normalized loss reduction with the training time at which the minimum keypoint-wise loss is reached.The two components capture how much the model learns from a keypoint and when its minimum loss occurs.
- Keypoint selection: K-means clusters keypoints in (∆t, ∆l) space, assigning the cluster with the lower centroid to noisy annotations.The method exploits the empirical tendency of both quantities to decrease as annotation noise increases.
- Keypoint selection: Keypoints assigned to the noisy cluster are excluded from loss computation by setting their COCO visibility flag to zero.The pose model is then trained on the resulting de-noised dataset.
- Noise taxonomy and evaluation: Synthetic noise is injected by adding Gaussian perturbations to randomly selected ground-truth keypoint coordinates under four corruption configurations.The configurations combine 20% or 50% corrupted training samples with 1–4 or 5–9 perturbed keypoints per image.
- Evaluation: Evaluation uses AUC, sensitivity, specificity, precision, agreement across three models, and mAP for clean, noisy, and REMIND-filtered training sets.The experiments use NeoPose, HRNet, ResNet 101, and ViTPose; cross-domain validation additionally uses SurgPose and cephalometric radiographs with HRNet.
4. Results and discussion
REMIND identifies noisy keypoints through training-dynamics scores and clustering, outperforming the SL trick across tested noise configurations and pose-estimation models. Filtering noisy keypoints substantially preserves training behavior and infant pose-estimation performance, while results also reveal lower separability and precision-related limitations in the infant setting.
- AUC values range from 93.3% to 97.7% across noise configurations and tested pose-estimation models, consistently outperforming the SL trick.
- REM scores show high linear correspondence with noise severity, with R² ranging from 81% to 87.9%.The analysis groups samples by their number of corrupted keypoints and regresses mean REM scores against that count.
- K-means separates noisy keypoints into a region with high values of both Δl and Δt in the HRNet KP-Noise20% setting.Figure 5 compares ground-truth labels with cluster assignments in the (Δt, Δl) space.
- Precision remains lower than sensitivity and specificity, partly because hard binary clustering places some clean keypoints in the noisy cluster.The reported limitation concerns borderline, hard, or ambiguous samples that are not represented by the clean-versus-noisy binary partition.
- Training on noisy labels produces higher losses than the clean baseline, whereas REMIND filtering yields a trajectory closely aligned with the clean-label baseline.The comparison includes the clean baseline, KP-Noise50% training without selection, and REMIND-filtered training.
- REMIND filtering limits the average mAP drop to 1.8% (±1.2%), compared with 22.5% (±10.7%) for training with noisy samples.The mAP values are measured on an infant-level hold-out test set designed to prevent data leakage.
- Performance is higher on SurgPose and cephalometric landmarks than on infant pose estimation, reflecting the infant task’s greater visual complexity.Infant-specific challenges include self-occlusions, inter-part overlap, pose variability, blankets, medical devices, and caregiver interactions.
5. Conclusion
The work introduces REMIND for identifying unreliable keypoint annotations, while acknowledging limits in dataset size, clinical-site coverage, validation conditions, and noise modeling.
- REMIND uses training-dynamics evolution to identify unreliable and noisy keypoint annotations.
- The curated dataset is limited in size and currently collected from a single clinical center.
- Future evaluations will consider noisy validation sets and broader standard pose-estimation benchmarks.
- The current Gaussian-shift noise assumption does not fully capture unequal corruption probabilities and ambiguous neonatal keypoints.
Conflict of Interest
The authors report no known competitive interests.
- The authors report no known competitive interests.
- The reported disclosure concerns the authors.
- The disclosed interest category is competitive interests.
Ethics Approval and Consent to Participate
The study received ethics approval, and written informed consent was obtained from each infant’s legal guardian.
- The study received approval from the Ethics Committee of Ospedali Riuniti di Ancona, Italy.
- The ethics approval identifier was Prot. 2019-399.
- Written informed consent was obtained from each infant’s legal guardian.