Source-linked AI summary
Self-Supervised Pretext Tasks for Infant Cry Analysis: A Controlled Comparison and a Cautionary Result on Donateacry
Luigi Simeone
TL;DR
Infant cry research needs reliable comparisons of self-supervised objectives and trustworthy reason-classification evaluation. This paper compares six objectives under a fixed budget and leakage-free protocol, finding strong detection but chance-level donateacry reason classification and protocol-sensitive inflated accuracy.
Problem
The paper addresses uncertainty about which self-supervised pretext task works at compact scale and what donateacry labels support under subject-wise evaluation.
Method
The study compares six pretext tasks using one compact encoder, fixed public pretraining audio, and subject-wise evaluation with leakage-free per-fold adaptation.
Results
Reconstructive objectives lead cry detection, while donateacry reason classification remains at chance despite adaptation and capacity control, and leaky protocols reproduce high reported accuracies.
Takeaways & Limitations
Subject-wise splitting, majority-class baselines, and post-split augmentation are necessary for interpreting donateacry results, whose effective sample size is infants.
Takeaways & Limitations
The reason-task conclusion is bounded by donateacry labels, and the human labeling pass used a single listener.
Abstract
from arXiv · showhide
We compare six self-supervised pretext tasks for infant cry analysis under a fixed budget, meaning the same compact encoder of 1.17M parameters, the same 115 hours of license-verified public pretraining audio, and the same evaluation protocol for every candidate. On cry detection the reconstructive objectives dominate, and a linear probe over a masked-spectrogram encoder reaches 0.988 AUC with subject-wise splits even though the encoder never observed a cry during pretraining. On cry-reason classification over donateacry, the de facto public benchmark for cry reasons, every encoder performs at chance (0.38 to 0.54 macro AUC over 5 classes), and neither domain adaptation on 1.8 hours of real cries nor end-to-end fine-tuning moves the result. Since a frozen HuBERT-base with 80 times more parameters shows the same pattern, the bottleneck must sit in the labels and not in model capacity. We then reproduce the 90\%+ accuracies of the donateacry literature on our own system by changing nothing but the evaluation protocol: clip-wise splits raise accuracy to 85.2% (barely above the 83.8% majority-class baseline), and applying augmentation before splitting raises it to 97.9%, matching the reported state of the art, from the same model that measures 0.49 macro AUC under subject-wise splits. Under leakage-free splits, a twentyfold augmentation of the labeled set (vocoder speaker perturbation and noise mixing, 21 hours) leaves cross-subject AUC unchanged: for this task the effective sample size is the number of infants. We release code, seeds and per-clip license manifests.
1 Introduction
The paper tests six self-supervised objectives under identical conditions and questions whether public cry-reason benchmarks support their reported performance. It contrasts controlled detection results with suspiciously high donateacry results and evaluates leakage directly.
- Benchmark concern: 90%+ donateacry accuracy reports are treated with suspicion because they concern a small parent-labeled corpus and differ from the 74.5 macro AUC clinical result.The cited motivation also notes that the published figures use different metrics.
- Research questions: Six pretext tasks are compared under a fixed budget to determine which objective performs best with a compact encoder.The comparison uses identical conditions and includes controlled variants designed to expose anticipated failure modes.
- Benchmark concern: The study asks what donateacry labels support when subject identity is respected throughout evaluation, including self-supervised stages.This question is paired with a protocol designed to test evaluation leakage.
- Research questions: 0.988 AUC is achieved for cry detection by a reconstructive objective, while the encoder pretrained without infant cries still succeeds.The contribution passage identifies reconstructive objectives as clearly ahead for cry detection.
- Contributions: The contributions include a model-capacity control, leakage-free adaptation, and augmentation evidence that labeled subjects—not clips—are the binding resource.The paper also contributes an ear-verified hard-negative set containing adult cries mislabeled as infant cries.
2 Related work
Related work spans infant-cry analysis, donateacry reason classification, leakage in clinical machine learning, and compact self-supervised audio representations. The paper positions its contribution as a controlled ranking at roughly one-million-parameter scale with subject-wise evaluation.
- Reason classification on donateacry: Donateacry studies report mid-90s accuracy, but the strongest cited result reaches 96.4% with an 80/20 split that does not mention subject identity.The paper states that no donateacry result known to the authors reports subject-wise evaluation.
- Evaluation leakage in clinical machine learning: Record-wise clinical cross-validation can inflate accuracy relative to subject-wise evaluation because deployment encounters new patients rather than new records from known patients.The paper connects this established leakage concern to a controlled audio-specific comparison.
- Self-supervised audio representations: The compared self-supervised families include contrastive or distillation-style embedding, masked reconstruction, and masked latent-target prediction.HuBERT enters this study as a frozen capacity control rather than as one of the compact candidates.
- Study design: Figure 1 organizes the study into six pretraining runs followed by a shared subject-wise evaluation harness, with invalid protocols reserved for leakage measurement.The encoder is pretrained on public audio without infant cries and its task heads are discarded before evaluation.
- Novelty: The paper’s novelty is a controlled ranking of pretext objectives at about one-million-parameter scale, a regime rarely examined by benchmark papers.This ranking is intended to distinguish individual design ingredients rather than merely produce a leaderboard.
3 Data
The study constructs a public, license-filtered audio pool and applies human verification before supervised experiments. Cleaning removes non-cries and supplies hard negatives while preserving a subject-linked donateacry dataset.
- Label verification: Human review followed PANNs triage and separated infant cries from adult cries and animal sounds.A second listening pass was applied after 127 borderline cases were selected by stratified value of information.
- Label verification: About half of surviving FSD50K clips tagged as baby or infant cry were adults crying, so they enter evaluation as hard negatives.This cleaning step directly affects the detection evaluation pool.
- Label verification: 14 spurious donateacry clips were excluded, leaving 433 clips from 204 infants for supervised experiments.The excluded material included 11 of the 15 lowest-scoring donateacry “cries” identified as non-cries.
4 Method
The method compares six interchangeable self-supervised pretext heads under a fixed encoder, data, training budget, and subject-wise evaluation protocol. The candidates test whether explicit frequency structure, temporal sensitivity, reconstruction, and related design choices transfer to infant-cry tasks.
- Design: Figure 1 uses one shared encoder with six interchangeable pretext heads during pretraining and one downstream evaluation harness.The encoder is retained while each pretext head is discarded after pretraining.
- Encoder: The shared CNN encoder has 1.17M parameters and processes normalized 3-second patches with 300 frames and 64 mel bands.Every candidate differs only in its pretraining head.
- Pretext tasks: Candidates A and B test filter identification, while the remaining objectives vary augmentations, temporal direction, and reconstruction within controlled comparisons.A and B use synthetic or real filtered inputs; C demotes filters to augmentations, E adds time direction, and D and F use reconstruction losses.
- Design rationale: The filter-ID hypothesis assumes cry information follows frequency structure, but the task risks band-energy shortcuts, static representations, and synthetic-data mismatch.The proposed cry structure includes fundamentals at 250 to 700 Hz and higher harmonics.
- Novelty: Band-filter identification is introduced as a new audio self-supervised pretext task, whereas band manipulation had previously appeared as augmentation.Candidates A and B specify the new filter-ID formulation; candidate C uses the filters as distortions inside SimSiam.
- Budget: Each candidate trains for 8,000 steps at batch size 256 on FSD50K and VocalSound, with donateacry and Freesound cries excluded from pretraining.The training uses AdamW, a cosine schedule, and mixed precision.
- Evaluation: Evaluation uses subject-wise five-fold splits with macro one-vs-rest AUC, balanced accuracy, and ECE, rerunning cry-audio adaptation within each training fold.Synthetic augmentation variants inherit their source infant identity, and evaluation uses real audio only.
5 Results
Reconstructive objectives lead cry detection, while reason classification remains at chance under subject-wise evaluation and is highly sensitive to leakage. Augmentation increases labeled audio without improving cross-subject reason classification.
- 0.988 AUC: the masked-spectrogram encoder leads cry detection with a linear probe, despite no cries during pretraining.Reconstructive objectives outperform contrastive and filter-identification objectives at this scale.
- 0.49 macro AUC: the same model under subject-wise evaluation contrasts with the inflated leaky-split results.The leakage ladder shows that split protocol, rather than only model choice, determines the apparent performance.
- 80 times larger: frozen HuBERT shows the same floor and leaky-split jump, shifting the bottleneck away from model capacity.The authors instead identify donateacry’s parent-provided labels and severe class imbalance as the remaining concern.
- 85.2% accuracy: clip-wise fine-tuning rises above the 83.8% majority-class baseline, while augmentation before splitting reaches 97.9% accuracy.The latter protocol places synthetic variants of test clips in training and matches reported state-of-the-art accuracy.
- 0.47, 0.46, 0.43 and 0.49 macro AUC: four fine-tuning augmentation arms are statistically indistinguishable across folds.Twentyfold augmentation from the same 204 infants adds no cross-subject signal.
6 Reproducibility
The study is designed for reproducibility on modest hardware, with fixed seeds, recorded outputs, and explicit data inventories.
- Pretraining takes under an hour and per-fold adaptation takes minutes on one consumer laptop.The complete study fits in a few evenings using an NVIDIA RTX 4060 Laptop, 8 GB VRAM, and 16 GB system RAM.
- Fixed configuration seeds and CSV-written outputs make every reported number traceable to its producing script.Feature caches use memory mapping, and the per-clip license manifest serves as the exact data inventory.
7 Limitations
The negative reason-classification result is bounded to donateacry’s labels and this study’s modeling choices, not to the general learnability of cry reasons.
- Clinical labels remain a demonstrated route to learnable cry-reason classification, unlike donateacry’s labels in this study.The paper points to labels tied to observed outcomes or retrospective caregiver resolution as a data-side need.
- Mean pooling over 3-second patches may erase sequence structure, motivating latent-predictive temporal objectives as future candidates.The single above-chance result points toward temporal modeling, but this is a proposed next direction rather than a demonstrated solution.
- Detection is reported as ranking quality, so false alarms per night and systematic multi-SNR evaluation remain untested operating considerations.The human labeling pass was also performed by a single listener.
8 Conclusion
Under a fixed budget and leakage-free evaluation, reconstructive pretraining leads cry detection, while cry-reason classification remains at chance and is highly sensitive to evaluation protocol. The results support subject-wise splitting, post-split augmentation, and infant-level data planning.
- 8 Conclusion: 0.988 AUC: masked-spectrogram reconstruction leads cry detection under the fixed, leakage-free comparison.The result comes from a linear probe, with no cries in pretraining.
- 8 Conclusion: Every cry-reason encoder collapses to chance, including after a 94M-parameter capacity control and 1.8 hours of domain adaptation.Using the literature’s clip-wise, augment-first protocol instead resurrects reported 96%+ accuracy from the same model.
- 8 Conclusion: Subject-wise splitting, majority-class baselines, and augmentation only after splitting are the recommended evaluation practices.The donateacry majority-class baseline is 83.8%.
- 8 Conclusion: Under these rules, donateacry supports cry detection but is misleading for cry reason, while data plans should prioritize the number of infants reached.The conclusion identifies infants, rather than recorded hours, as the relevant planning unit.
- 8 Conclusion: Progress on cry reason requires outcome-linked labels across enough infants for subject-wise evaluation, alongside objectives that model temporal structure.The paper identifies latent-trajectory prediction as a natural next modeling direction.