Source-linked AI summary
UNWIND: Any-Length Facial Video for Stress Detection without Temporal Windowing
Stefanos Gkikas, Christian Arzate Cruz, Eric Nichols, Giorgos Giannakakis, Randy Gomez
TL;DR
Most video-based stress-recognition methods segment recordings, adding temporal partitioning choices and potentially losing context across boundaries. UNWIND folds time into spatial-channel representations and processes complete recordings with asymmetric attention; its best test accuracy is 70.02% at τ = 15, with 69.73% at τ = 1.
Problem
Many video-based stress-recognition methods divide recordings into temporal segments, adding design choices and potentially losing context across segment boundaries.
Source: §1 · p. 3Method
UNWIND folds a recording’s temporal dimension into the channel dimension and processes the complete input with a unified asymmetric-attention architecture.
Source: §1 · p. 3, §3.2 · p. 5, §4.4 · p. 12Results
70.02% test accuracy is achieved at τ = 15, while τ = 1 reaches 69.73% test accuracy and requires 348.78 GFLOPs.
Source: §4.4 · p. 13Takeaways & Limitations
UNWIND processes a 120-second recording of 3,600 frames at τ = 1 in one forward pass without temporal windowing or external segmentation.
Source: §4.4 · p. 12The evaluation uses one dataset and binary neutral-versus-stress classification, without a direct comparison against video window-based approaches under the same validation.
Source: §4.4 · p. 13
Abstract
from arXiv · showhide
Automatic stress recognition from facial video provides a non-contact approach for affective monitoring. However, most existing video-based methods divide complete recordings into shorter temporal segments before performing classification. Such segmentation requires additional decisions concerning segment duration, overlap, and prediction aggregation, and may restrict the model from exploiting information distributed across the entire recording. We introduce UNWIND, a facial-video framework for stress detection that analyzes a complete recording as a single model input, eliminating the need for temporal windowing or external segmentation. UNWIND reorganizes the video by folding its temporal dimension into the channel dimension of a two-dimensional spatial representation, which is subsequently processed through a unified asymmetric-attention architecture. With a temporal stride of $τ=1$, the framework processes the entire $120$-second sequence, corresponding to $3{,}600$ frames sampled at $30$~fps, in a single input. We evaluate seven temporal-stride settings on a stress dataset comprising $58$ subjects, using a stratified subject-level protocol that covers configurations from dense frame retention to sparse temporal sampling. The highest test accuracy, $70.02\%$, is obtained at $τ=15$, while processing all frames at $τ=1$ achieves a comparable accuracy of $69.73\%$. Computational requirements range from $12.48$ to $348.78$ GFLOPs across the evaluated stride settings, illustrating the balance between temporal sampling density and computational efficiency. The findings show that effective facial-video stress recognition can be achieved without dividing recordings into temporal windows and that complete-recording inference can be performed within a single unified model.
1 Introduction
Conventional stress measures can be delayed, retrospective, intrusive, or difficult to scale, motivating passive alternatives. UNWIND addresses temporal windowing by processing a complete facial recording as one input.
1 Introduction
- Questionnaires can be affected by recall bias and capture short-term stress fluctuations poorly, while cortisol sampling is intrusive and delayed.Stress responses can vary in magnitude and duration, from short-lived reactions to persistent chronic stress.Source: §1 · p. 2
- Stress is a public-health concern linked in the cited studies to rising perceived stress and adverse health outcomes.Surveys across 146 countries reported an approximately twofold increase in perceived stress over 18 years; occupational factors were associated with cardiovascular disease and depression cases.Source: §1 · p. 2
- Self-report and wearable approaches face practical constraints, including incomplete or delayed reports, social desirability, sensor artifacts, adherence, and scaling.These challenges motivate passive, non-intrusive automated approaches using signals captured naturally in everyday settings.Source: §1 · p. 2
- Facial video offers a non-contact source of stress cues, including eye behavior, mouth motion, and head movement.Facial Action Units provide a structured representation of these behavioral patterns.Source: §1 · p. 2
- Complete recordings are rarely analyzed directly: temporal segmentation adds partitioning choices and can exclude context spanning segment boundaries.UNWIND instead uses axis folding and asymmetric attention to process each recording as one input, evaluated across stride settings on 58 subjects.Source: §1 · p. 3
2 Related Work
Video-based stress recognition has progressed from handcrafted facial cues to learned spatiotemporal and AU-based representations. Many systems still segment recordings, while direct complete-recording analysis remains comparatively underexplored.
2 Related Work
- Earlier handcrafted facial descriptors gave way to deep learning that learns facial and action-related features from video, improving performance over feature engineering.Spatiotemporal models learn changes across space and time, typically from short, fixed-duration clips.Source: §2 · p. 3
- Facial Action Units offer an interpretable representation of muscle activations, with AU-based methods distinguishing stressed and neutral states across individuals and induction conditions.Later work combines automatic AU estimation with facial geometry and deep appearance features, and uses explainability and graph-based formulations.Source: §2 · pp. 3–4
- Video stress-recognition systems commonly classify predefined segments, which reduces long-recording computation but adds duration, overlap, and aggregation choices.Segment boundaries can limit access to temporal information beyond each segment, and direct complete-recording analysis remains comparatively underexplored.Source: §2 · p. 4
3 Methodology
UNWIND preprocesses face-centered video, folds time into the channel dimension, and uses asymmetric attention over spatial segments. Its four-layer design combines segment-local retrieval with global interaction among compact segment states.
3.1 Video preprocessing
- Face crops are anchored by BlazeFace detections, updated every 150 frames, and kept at fixed spatial dimensions throughout the recording.Each frame is cropped and resized to 224 × 224; preprocessing preserves the complete 30 fps sequence without temporal subsampling or windowing.Source: §3.1 · pp. 4–5
3.2 Video Tokenization
- Temporal stride τ yields L retained frames, which are folded into the channel dimension of a 224 × 224 spatial representation.At τ = 1, a 120-second recording retains all 3,600 frames; spatial tokens combine folded video channels with positional features and are grouped into four spatial segments, not temporal windows.Source: §3.2 · pp. 5–6
3.3 Asymmetric Attention
- Four successive layers use asymmetric cross-attention for segment-local token retrieval, then self-attention for interaction among the four segment states.The states are initialized from shared learnable global latent parameters, compressed across layers, averaged, and passed to a linear classification head.Source: §3.3 · pp. 6–7
4 Experimental Evaluation & Results
UNWIND was evaluated on subject-separated stress data across temporal strides, with test accuracy peaking at τ = 15 and remaining comparable when all frames are retained. The analysis also compares inference costs and protocols, while limiting its conclusions to one dataset and binary classification.
4.1 Dataset and Protocol
- The 58-subject study uses facial video for binary neutral-versus-stress classification and stratifies participant-level training, validation, and test splits as 38, 8, and 12 subjects.The protocol combines four stress-induction phases and includes four neutral, six stress-inducing, and one relaxation task; relaxation is grouped with neutral.Source: §4.1 · pp. 7–9
4.2 Video
- At τ = 30, UNWIND requires 12.48 GFLOPs versus 348.78 GFLOPs at τ = 1, an approximately 28-fold computation decrease as retained frames fall from 3,600 to 120.Latency is 16.47 ms at τ = 30 and 133.02 ms at τ = 1.Source: §4.2 · pp. 9–10
4.4 Overall Analysis & Discussion
- 70.02% test accuracy at τ = 15 requires 24.07 GFLOPs, while τ = 1 achieves 69.73% at 348.78 GFLOPs.Test accuracy varies non-monotonically with stride; higher temporal density does not inherently improve generalization.Source: Figure 2 · p. 11, §4.4 · p. 13
4.3 Comparison with a Related Approach
- Under leave-one-subject-out evaluation, UNWIND achieves 80.04% mean accuracy across subjects, with an accuracy standard deviation of 10.86%.The precision and F1-score standard deviations are 10.03% and 11.52%, respectively.Source: §4.3 · p. 12
- Facial video reaches 70.02% accuracy at τ = 15, close to electrodermal activity at 70.97% on the same dataset and stratified hold-out protocol.The video model uses 5.93 million parameters and 24.07 GFLOPs, compared with 2.06 million parameters and 1.58 GFLOPs for electrodermal activity.Source: §4.3 · pp. 10, 12, Table 5 · p. 12
4.4 Overall Analysis & Discussion
- At τ = 1, UNWIND processes all 3,600 frames of a 120-second recording in one forward pass, without temporal windowing or external segmentation.The same architecture supports stride settings from τ = 30 to τ = 1 without structural changes.Source: §4.4 · pp. 12–13
- The study is limited to one dataset and binary neutral-versus-stress classification and does not directly compare UNWIND with video window-based approaches under the same validation.The authors caution against interpreting the results as evidence that full-recording inference universally outperforms or should replace window-based strategies.Source: §4.4 · p. 13
5 Conclusion
UNWIND processes complete facial-video recordings without temporal windowing, and its full-frame configuration remains competitive with the best reported stride. The conclusion identifies broader durations, modalities, and prediction explanations as directions for future work.
5 Conclusion
- 70.02% test accuracy at τ = 15 compared with 69.73% at τ = 1 on a 58-subject stress dataset.At τ = 1, the model processes all 3,600 frames of a 120-second recording sampled at 30 fps.Source: §5 · p. 13
- UNWIND folds the temporal dimension into a 2D spatial representation and applies asymmetric attention to process complete recordings without temporal windows or external segmentation.The same architecture handles sparse subsampling and complete-frame sequences without structural changes.Source: §5 · p. 13
- Future work should test longer, more variable-duration recordings, compare directly with window-based baselines, and explore additional modalities, multiclass recognition, and identifying influential recording portions.The passage highlights identifying contributing portions as particularly relevant to real-world monitoring.Source: §5 · p. 14