Pseudo-Label Augmentation for Affect Sensing in Small Collaborative Groups

1 Introduction

• Sparse self-reports leave thousands of physiological windows with few VAD labels, and person-specific affect patterns vary with personality, task role, and group context.The study uses GroupAffect-4, which combines multimodal interaction signals with sparse post-task affect reports.Source: §1 · p. 1
• The comparison progresses from labelled-only training to GP pseudo-labelling, personality-aware trust weighting, and a joint personality-plus-confidence variant.The main comparison holds the RBF-SVM model constant to isolate pseudo-label construction.Source: §1 · pp. 1–2
• The study tests whether Big Five cosine similarity can serve as a stable trust signal, but reports that personality constrains the useful pseudo-label pool rather than finely ordering individuals.The proposed role is a same-team membership filter, not a calibrated trust dial.Source: §1 · p. 2
• The practical target is later-task prediction for a known team using continuous physiology and only occasional self-reports.Earlier-task observations support prediction on later, behaviorally distinct tasks rather than within-task interpolation.Source: §1 · p. 2
• Personality-aware augmentation helps mainly as a within-team filter, while known-team augmentation is the main success case and LOGO is a conservative lower bound for unseen-group transfer.The contribution taxonomy also includes LOSO cross-subject and strict LOGO stress tests.Source: §1 · p. 2

Dimensional affect and target construction

• The target pipeline is motivated by ordinal, tied, task-dependent SAM ratings and by the limitations of fixed threshold binning for imbalanced classes.The paper motivates balanced midpoint-tertile targets while preserving ordered uncertainty.Source: §2 · p. 2

Physiological and oculomotor affect recognition

• Affect benchmarks and reviews support multimodal physiological sensing while emphasizing sensitivity to preprocessing and split protocol.The related work names wearable and multimodal datasets including DEAP, DREAMER, MAHNOB-HCI, WESAD, and ASCERTAIN.Source: §2 · p. 2

Social and group affect datasets

• Existing group-affect resources cover dyadic interaction, shared-media viewing, and social signals, while GroupAffect-4 combines four-person collaboration with physiology, eye tracking, VAD self-report, and pre-session personality.The cited comparison distinguishes collaborative problem solving from shared-media group viewing.Source: §2 · p. 2

Sparse labels, augmentation, and generalisation

• Sparse-label methods include time-series augmentation, contrastive learning, confidence-threshold pseudo-labelling, and Gaussian-process interpolation, with subject- and group-aware splits needed to interpret generalisation.The passage notes that pseudo-label usefulness depends on reliable confidence.Source: §2 · p. 2

Personality and affect dynamics

• Personality studies link Big Five traits to physiological affect responses and report trait-specific moderation and benefits from combining personality with temporal gaze and stimulus context.The cited work includes felt-emotion prediction and VAD dynamics during real interaction.Source: §2 · p. 2

Why personality and not only physiological synchrony?

• Personality similarity offers a slower, more stable prior than physiological synchrony for weighting cross-person pseudo-labels when self-reports are sparse and uncertain.Synchrony can reflect shared events or movement as well as affective convergence.Source: §2 · p. 3

3 Dataset and Targets

• GroupAffect-4 comprises 10 four-person sessions with five tasks, post-task 9-point SAM VAD reports, pre-session BFI-44, and 49-dimensional features summarized in 15-second windows.T4 Dominance was not administered, and overlapping windows are task-level supervisory proxies rather than fine-grained trajectories.Source: §3 · p. 3
• The canonical known-team split trains on T0+T1, validates on T2, and tests on T3, with 103, 78, and 65 windows respectively.The forward cross-task ordering is intended to mirror later-task prediction within the same team; T3 has complete VAD labels for 32 participants.Source: §3 · p. 3

3.1 Why Target Construction Matters

• Balanced midpoint-tertile thresholds fitted on training labels keep tied SAM responses together, correct residual imbalance, and place hard and GP pseudo-labels in the same class space.The shared target pipeline addresses fixed-threshold class collapse that could inflate Valence F1 when a test fold lacks Low-valence examples.Source: §3 · p. 3, §3.1 · p. 3

3.2 Deployment Regimes

• The three deployment regimes distinguish later-task prediction within a known team, transfer to an unseen participant, and transfer to an entirely unseen group.The known-team temporal holdout is the primary setting for A2/A3; strict LOGO excludes the held-out session from GP fitting, pool windows, BFI maps, and parameter estimation.Source: §3.2 · pp. 3–4

4 Augmentation Methods

• GP pseudo-labels use participant- and dimension-specific Ornstein–Uhlenbeck GP posteriors from task-level SAM anchors, converting posterior mass across training-fitted thresholds into a class and confidence weight.In the canonical split, only T0 and T1 anchors are available; the GP is treated as a weak interpolation prior, not a recovered continuous affect trajectory.Source: §4 · p. 4
• A2 weights pseudo-labels by the source participant’s BFI similarity to groupmates; A3 multiplies that weight by GP posterior confidence, while all variants retain one SVM.The BFI weight is participant-level, not pairwise to a recipient, and is applied equally to that source’s pseudo-labels.Source: §4 · p. 4
• Scenario 1 fits the GP on T0+T1 only; strict Scenario 2 excludes the held-out session from GP fitting, pool windows, BFI maps, and parameter estimation.Only Scenario 2 tests unseen-group deployment.Source: §4 · p. 4

4.1 Why We Tested Personality Weighting

• A2 tests whether group-centrality in Agreeableness, Extraversion, and Neuroticism makes pseudo-label sources more transferable, while limiting personality to sample weights rather than high-capacity conditioning.The paper cites N=103 as too small to support trait-conditioned projection matrices without increased overfitting risk.Source: §4.1 · p. 5
• BFI cosine similarities span only 0.91–0.99; shuffled A2 weights matched real weights (permutation p=0.39), so personality acted as a team filter, not a calibrated ranking.Without a constant-weight A1 control near the observed BFI range, the results cannot separate near-uniform weighting from admitting the same-team pseudo-label pool.Source: §4.1 · p. 5

5.1 Known-Team Augmentation

• Valence 0.608 and Dominance 0.435 under A2; Arousal peaks under A1 at 0.546 in the known-team comparison.The clearest gains come from personality-aware within-team filtering rather than GP interpolation alone; these are informative point estimates, not statistically settled effect sizes.Source: §5.1 · p. 5

5.2 LOSO Cross-Subject Stress Test

• LOSO remains harder than the known-team setting, especially on Valence, but extracts some participant-independent affect structure under strict participant isolation.The stress test holds out T3 windows for each of 32 participants and uses no pseudo-label augmentation.Source: §5.2 · pp. 5–6

5.4 Uncertainty and Significance

• The canonical T0+T1→T3 comparison reports point estimates, while exploratory task-CV supports the broad pattern but shows substantial between-task variance.A1 was omitted from task-CV, so it is not a complete four-variant comparison.Source: §5.4 · p. 6

5.3 Strict Session Isolation

• In strict session-isolated LOGO, every augmentation variant falls within A0 ± 0.03 per dimension, with no significant improvement over A0 (all p>0.2).Across only 10 groups, LOGO is a conservative lower bound on unseen-group transfer, not evidence that cross-group label mining is intrinsically ineffective.Source: Figure 2 · p. 6, §5.4 · p. 6, §5.3 · p. 6

5.4 Uncertainty and Significance

• Shuffling BFI similarity scores leaves the null distribution indistinguishable from real weights (permutation p=0.39), so weight ordering is not what drives the gain.The permutation test shuffled scores 500 times while keeping pseudo-labels unchanged.Source: §5.4 · p. 6

5.5 Preprocessing and Window-Size Ablation

• Smoothing improves every retained variant, especially on Arousal; A1–A3 tie to rounding on Valence and Arousal, while A3's Dominance margin over A1 is 0.015 on 65 test windows.The 30-second windows improve Arousal and partly Dominance under A0 but weaken Valence, halve temporal resolution, and reduce labelled training examples; 15 seconds remains the all-around compromise.Source: §5.5 · pp. 6–7, Table 7 · p. 7

• Valence and Dominance are more appraisal-heavy and relational, and are the dimensions where within-team label transfer helps most.Source: §6 · p. 7
• Arousal is more phasic and benefits more from smoothing or longer windows than from personality cues alone.Source: §6 · p. 7
• Dominance remains difficult because physiology omits speaking, directed gaze, and influence structure.Source: §6 · p. 7
• A3 can prune too aggressively without smoothing, but becomes competitive when EDA/PPG smoothing improves GP confidence.Source: §6 · p. 7
• The narrow BFI range supports a same-team filter, not calibrated personality ranking.Source: §6 · p. 7

7 Discussion

• The narrow BFI similarity range makes personality weighting a keep-or-discard pool filter rather than a graded confidence signal.Direct BFI conditioning overfits at the current label density (N=103); validating graded weighting requires groups with more diverse personality profiles.Source: §7 · p. 7
• A2 is a within-team calibration method for monitoring the same team across tasks, not a result establishing generalization to unfamiliar groups.It uses pre-session personality and earlier-task unlabelled windows to smooth the team’s affect trajectory.Source: §7 · p. 7
• A1–A3 tie to rounding on Valence and Arousal, while A3 has the highest augmented Dominance score in the fixed 15-second smoothed SVM comparison.Some higher scores come from ablations that change architecture or windowing, so they are not controlled augmentation comparisons.Source: §7 · p. 7
• LOSO transfer is encouraging, especially for Arousal, but LOGO is a lower bound on unseen-group transfer given only 10 groups.Brief self-report calibration for new groups is suggested but remains untested.Source: §7 · p. 7
• The Dominance setup uses globally fitted tertiles and separate member samples, leaving within-group rank, reciprocity, and influence structure unmodeled.The discussion proposes modeling these relations explicitly using speaker turns, person-directed gaze, and group-relative Dominance targets.Source: §7 · p. 7

8 Limitations

• With only 10 groups, strict LOGO estimates have high fold variance and cannot establish how unseen-group performance might approach known-team results.The canonical A2 comparison is a temporal known-team result, and its improvement is small in absolute terms.Source: §8 · p. 7
• The canonical positive result relies on one T0+T1→T3 split, with T3 comparatively favourable; auxiliary task-CV omits A1.The cross-task findings are exploratory until all variants are compared across repeated task and group splits.Source: §8 · p. 7
• Pseudo-labels lack validation against dense human annotation or an independent affect measure, and retained pseudo-label counts are unreported.Overlapping windows also call for participant- or session-level resampling rather than treating windows as independent.Source: §8 · p. 7
• Direct trait-conditioned modeling added too much capacity for N=103, while A3’s extra GP-confidence gate could prune too aggressively without smoothing.Thirty-second windows improved Arousal and partly Dominance but weakened Valence and temporal resolution.Source: §8 · p. 7

9 Conclusion

• GroupAffect-4 experiments frame sparse affect annotation as label mining and show that additional supervision can be mined usefully in the known-team regime.The framing identifies scarcity of trustworthy supervision, rather than sensor volume, as the bottleneck in group-interaction datasets.Source: §9 · p. 8
• The BFI cosine-similarity range of 0.91–0.99 renders weight ordering negligible (permutation p=0.39), so personality acts as a same-team filter, not a calibrated trust dial.A2 nevertheless delivers point-estimate V/D gains of +0.038 / +0.050.Source: §9 · p. 8
• With centered EDA/PPG smoothing, A3 scores V=0.642 / A=0.624 / D=0.442 in the fixed 15-second SVM comparison, but the appropriate configuration depends on dimension and deployment.LOSO is encouraging for this small dataset; LOGO with only 10 groups is a conservative lower bound on unseen-group transfer.Source: §9 · p. 8