Source-linked AI summary
Toward Reliable Infant Pose Estimation: A Training-Dynamics Approach to Noisy Annotation Detection
Emanuele Cardinale, Sara Moccia, Alessandro Cacciatore, Lucia Migliorelli
TL;DR
Noisy keypoint annotations can distort the joint trajectories used for clinical assessment of preterm infants. The paper detects them using cross-entropy training dynamics, reporting improved detection over an MSE-based approach and measurable downstream gains after filtering on COCO.
Problem
Manually annotated keypoints can be mislocalized, and noisy keypoints can distort clinical joint trajectories and undermine assessment reliability.
Source: §6 · p. 21Method
The framework predicts keypoint anatomy and body side, then uses cross-entropy training-dynamics features and unsupervised clustering to separate noisy from clean annotations.
Source: §1 · p. 5, §3.2 · p. 11Results
The CE-based approach achieves higher noisy-keypoint detection F1 than the MSE-based method across both NeoPose noise scenarios and yields measurable downstream pose-estimation gains on COCO after filtering.
Source: §1 · p. 5, §6 · p. 22Takeaways & Limitations
Filtering detected noisy annotations improves downstream localization performance on a clean test set in the COCO evaluation.
Source: §1 · p. 5The earlier MSE-based approach has low precision because correctly annotated but hard-to-localize keypoints can be mistaken for noisy ones.
Source: §1 · p. 4
Abstract
from arXiv · showhide
Spontaneous movement analysis in preterm infants relies increasingly on markerless pose estimation (PE) to derive clinically relevant motion biomarkers directly from video recordings. Training accurate infant PE models requires large sets of manually annotated keypoints, and human annotation is inherently prone to error. Noisy keypoints (i.e., keypoints mislocalized with respect to their true anatomical position) are especially problematic in this clinical setting, since they can propagate as artificial artifacts into the reconstructed joint trajectories. Building on the small-loss hypothesis and training-dynamics-based sample selection established in the noisy-label learning literature, we propose a novel framework for detecting noisy keypoint annotations. A hybrid convolutional-attention model is trained to predict the anatomical category of each keypoint from its spatial coordinates and local visual features; the resulting cross-entropy training dynamics are then used to derive per-keypoint descriptors, which are partitioned into clean and noisy subsets via unsupervised clustering. We validate the approach on NeoPose, a newly collected dataset of 65 hospitalized preterm infants, under two realistic noise scenarios (random positional perturbation and left-right swapping) across multiple noise levels. Results show that the proposed approach achieves an F1-score of up to 91.9% in noisy-keypoint detection. The framework further generalizes to the heterogeneous COCO benchmark, where filtering CE-detected noisy keypoints from the training set also yields measurable improvements (up to 7.4 AP points) in downstream pose estimation accuracy at moderate-to-high noise levels.
1. Introduction
Infant pose estimation can support quantitative movement assessment, but noisy annotations may distort the joint trajectories used to compute clinical biomarkers. This work uses cross-entropy training dynamics and clustering to detect noisy keypoints, testing the approach on NeoPose and heterogeneous COCO data.
1. Introduction
- Mislocalized annotations can introduce artificial displacements into joint trajectories, which may affect the motion biomarkers computed from those trajectories.This makes annotation quality especially consequential for infant movement assessment.Source: §1 · p. 3
- The earlier MSE-based approach had relatively low precision because noisy keypoints and correctly annotated but difficult-to-localize keypoints can produce similar unstable losses.Occlusion, motion blur, and ambiguous limb configurations are examples of localization difficulty.Source: §1 · pp. 3–4
- A hybrid convolutional-attention classifier predicts keypoint anatomy from coordinates and local visual features; per-keypoint CE training dynamics are clustered into clean and noisy subsets.The approach reframes noisy-keypoint detection as classification, using the mismatch between coordinates and local visual evidence.Source: §1 · pp. 4–5
- The study asks whether CE-based dynamics reflect keypoint-label reliability and whether the detector generalizes from NeoPose to heterogeneous PE datasets such as COCO.The motivating clinical setting is preterm infants monitored during hospitalization.Source: §1 · pp. 4–5
- CE-based training dynamics outperform MSE-based dynamics across positional-perturbation and left-right-swap noise scenarios, and filtering detected annotations improves downstream pose-estimation performance on COCO.The work evaluates detection across increasing noise levels and tests filtering on a clean COCO test set.Source: §1 · p. 5
2. Related work
Prior noisy-label research uses training losses and their temporal dynamics to identify likely clean examples, but comparatively little work addresses regression or pose-estimation settings. The cited infant pose-estimation study used MSE trajectories and clustering, while Figure 1 depicts this paper’s two-stage classifier-and-clustering framework.
2.1. Learning with noisy labels
- The small-loss hypothesis motivates sample selection from training losses, with prior methods using co-teaching, learning-time metrics, trajectory analysis, or sequence models.These methods are developed almost exclusively for classification, while noisy-label learning under regression objectives has received comparatively little attention.Source: §2.1 · p. 6
- Figure 1 shows a two-stage framework: a hybrid classifier predicts keypoint anatomy and lateral position, then unsupervised clustering separates clean and noisy annotations.The inputs are RGB images and their corresponding keypoint annotations.Source: Figure 1 · p. 7
- Prior infant pose-estimation work extracted MSE regression-loss features for each keypoint and clustered annotations into clean and noisy subsets [21].The related work identifies this as the closest prior approach.Source: §2.1 · p. 7
3. Method
The method detects mislocalized keypoints by classifying their anatomical category and side, then clustering features derived from classification-loss training dynamics. It combines spatial and visual evidence in the classifier and uses the resulting trajectories to separate clean from noisy annotations.
3. Method
- The framework treats annotations whose coordinates do not match true joint positions as noisy and replaces earlier MSE-based selection with a classification approach to address its precision limitation.The classifier exploits the mismatch between a keypoint’s spatial location and local visual evidence.Source: §3 · p. 8
3.1. First stage: Predicting anatomical class
- The first stage predicts each keypoint’s anatomical category and left/right side from its coordinates and sampled image features, using a hybrid convolutional-attention classifier trained with cross-entropy losses.Spatial and visual representations are combined before cross-attention, and separate heads predict category and side.Source: §3.1 · pp. 9–10
3.2. Second stage: Noisy and clean keypoints clustering
- The second stage smooths joint and side loss trajectories, extracts four variation-and-timing features per keypoint, and applies K-means, assigning the lower-centroid cluster as noisy.The features capture normalized peak-to-trough variation and when the minimum loss occurs relative to the maximum.Source: §3.2 · pp. 10–11
4. Experimental Protocol
The protocol tests two injected keypoint-noise types on NeoPose and compares CE-based detection with an MSE-based method across NeoPose and COCO.
4.1. Noise injection
- Noise was injected as Gaussian positional perturbations or left–right swaps, at levels from 10% to 40%.Random displacement variance ranged from 0.15% to 0.25% of infant bounding-box size.Source: §4.1 · p. 11
4.2. Dataset
- NeoPose comprises 7,563 annotated frames from 65 preterm infants recorded at one Italian neonatology department.Approximately 116 frames were sampled per infant from five-minute recordings; expert neonatologists annotated them in COCO format.Source: §4.2 · p. 12
4.3. Training settings
- The model used an ImageNet-pretrained EfficientNet-B3 backbone and was trained for 40 epochs with a 10-epoch warm-up.Training used Adam and a 128-dimensional embedding; experiments ran on one NVIDIA Ampere A100 GPU.Source: §4.3 · pp. 12–13
4.4. Comparisons
- Comparisons first selected CE-derived keypoint descriptors, then evaluated the chosen configuration against the earlier MSE-based method on NeoPose and COCO.The COCO comparison used random noise and assessed both detection and downstream pose-estimation accuracy.Source: §4.4 · p. 13
4.4.1. Feature selection for noisy-keypoint clustering
- Ablations tested subsets of four CE-trajectory descriptors to identify noise-specific and noise-agnostic configurations across NeoPose noise levels.The noise-agnostic configuration was used unchanged in later experiments.Source: §4.4.1 · p. 13
4.4.2. Comparison with the MSE-based method and cross-dataset evaluation on COCO
- The MSE baseline used HRNet loss trajectories and the same K-means procedure, isolating loss type in comparisons with CE-based detection.On COCO, downstream HRNet accuracy was evaluated on a clean test set after removing detected noisy keypoints.Source: §4.4.2 · p. 13
4.5. Performance metrics
- F1-score measured detection, SeM quantified clean–noisy trajectory overlap, and COCO AP assessed downstream pose estimation.Lower SeM indicates better separation; COCO AP averages over OKS thresholds from 0.50 to 0.95 in 0.05 increments.Source: §4.5 · p. 14
5. Results and discussion
The results compare feature sets and CE- versus MSE-based training dynamics for noisy-keypoint detection, then test whether filtering detected annotations helps downstream pose estimation. CE dynamics outperform the MSE baseline on NeoPose and COCO, while filtering helps at moderate-to-high noise but not at the lowest tested level.
5.1. Feature selection for noisy-keypoint clustering
- The full four-feature set was best or joint-best across both noise types (random 0.908 ± 0.003; swapping 0.812 ± 0.081), with joint dynamics informative for positional noise and side dynamics for swaps.The authors adopted the complete feature set for subsequent experiments without requiring prior knowledge of the corruption type.Source: §5.1 · pp. 15–16, Table 1 · p. 15
5.2. RQ1: To what extent do per-keypoint CE-based training dynamics reflect label reliability in human PE?
- On NeoPose, CE-based training dynamics consistently outperformed the MSE baseline across random and swapping noise, with wider clean/noisy trajectory separation in the random-noise comparison.The paper also reports lower, better SeM values for CE dynamics across random-noise levels.Source: Table 2 · p. 15, §5.2 · pp. 16–17, Table 3 · p. 16
5.3. RQ2: Can the proposed noisy-label detection method generalize to heterogeneous PE datasets?
- On heterogeneous COCO, CE still outperformed MSE at every tested noise level despite lower absolute detection performance and reduced clean/noisy trajectory separation than on NeoPose.Detection performance improved as noise increased; the authors hypothesize that K-means performs better as clean and noisy classes become more balanced. Detection difficulty also differed by landmark, with both ankles harder than the others.Source: §5.3 · pp. 18–20, Figure 4 · p. 19, Figure 5 · p. 19
5.3.1. Downstream impact of noisy-label filtering on PE performance
- COCO AP rose from 0.544 to 0.618 at 20%, 0.542 to 0.599 at 30%, and 0.522 to 0.573 at 40% after filtering, but fell at 10%.At 10%, COCO AP was 0.633 on unfiltered data versus 0.601 after filtering.Source: §5.3.1 · pp. 20–21
6. Conclusion
Noisy keypoints can distort reconstructed joint trajectories and undermine the reliability of clinical assessment. The proposed framework improves noisy-keypoint detection across two noise scenarios and transfers to downstream pose estimation, while retaining limitations in clustering and noise coverage.
6. Conclusion
- Noisy annotations are especially consequential because they can distort joint trajectories used to compute motion biomarkers, undermining clinical-assessment reliability.The authors propose integrating detection as an online data-curation step within the pose-estimation training loop.Source: §6 · pp. 21–22
- The CE-based classification framework improves noisy-keypoint detection over the MSE-based method across random-positional and left–right swap noise, and filtering detected keypoints yields downstream pose-estimation gains on COCO.The conclusion reports measurable COCO gains but gives no numeric values here.Source: §6 · p. 22
- K-means clustering has a class-balance bias that penalizes detection precision at low noise levels.The authors identify clustering strategies robust to class imbalance as future work.Source: §6 · p. 22
- The tested noise scenarios do not cover all practical annotation errors, including those arising from occlusions or ambiguous poses.The authors propose extending the noise taxonomy to additional real-world error types.Source: §6 · p. 22