Intensity-Aware Loss for Dynamic Facial Expression Recognition in the Wild

• DFER classifies video sequences into discrete expressions and provides richer facial information than single-image SFER.Source: §1 · p. 1
• In-the-wild sequences vary in expression intensity, and low-intensity sequences are more likely to be misclassified when discrete labels lack intensity-related priors.Source: §1 · p. 1
• GCA rescales feature-map channels to enhance target-related channels in low-intensity samples while suppressing less useful channels.Source: §1 · p. 2
• IAL gives extra attention to the most confusing class of low-intensity expression sequences.Source: §1 · p. 2
• The method outperforms the baseline significantly and achieves state-of-the-art results on two popular in-the-wild DFER benchmarks.Source: §1 · p. 2

2.1 DFER in the Wild

• Earlier DFER methods include hand-crafted features and, later, deep-learning approaches that surpassed them on in-the-wild datasets.Source: §2.1 · p. 2
• CNN-RNN, 3D-CNN, and transformer approaches model spatial and temporal features, but the reviewed methods ignore expression-intensity effects.Source: §2.1 · p. 2

2.2 Attention Mechanisms

• Common channel-attention methods aggregate channel information using global spatial pooling before learning channel dependencies.Source: §2.2 · p. 2
• Global pooling can discard spatial location information, while sigmoid attention weights only suppress channels; GCA is proposed to model dependencies and enhance critical channels.Source: §2.2 · pp. 2–3

3.1 Overview

• The network rescales frame-level features with GCA, fuses static and dynamic features, and uses a temporal transformer to learn long-distance frame dependencies before classification.The mean of the token sequence is fed to a fully connected layer for recognition.Source: §3.1 · p. 3

3.2 Input Clips

• Training clips are formed by dividing raw sequences into U segments and randomly selecting V frames per segment; testing selects the middle V frames from each segment.Both sampled clips have length T = U × V.Source: §3.2 · p. 3

3.3 Global Convolution-attention Block

• GCA rescales feature channels with weights in (0, 2), enabling both suppression and enhancement while retaining feature-location information.Temporal global average pooling also offers a larger inter-frame field of view.Source: §3.3 · p. 4

3.4 Intensity-aware Loss

• IAL addresses low-intensity samples in discrete-label DFER, whose continuous-label alternatives are costly to annotate and difficult to standardize.Its design assumes low-intensity samples are likely to be confused with low-intensity samples from other classes.Source: §3.4 · p. 4
• The overall objective combines intensity-aware loss with cross-entropy, with λ controlling the loss coefficients.IAL gives extra attention to the class most likely to cause confusion; when cross-entropy converges to zero, IAL also converges to zero.Source: §3.4 · p. 4

4.1 Datasets

• DFEW evaluation uses 12,059 single-label clips in five-fold cross-validation; FERV39k has 38,935 clips across 22 scenes and an 80/20 train/test split.Both datasets use UAR and WAR; FERV39k clips are annotated by 30 professional annotators.Source: §4.1 · pp. 4–5

4.2 Implementation Details

• Models train from scratch with 16-frame sampling and SGD; schedules run 80 epochs for DFEW and 100 epochs for FERV39k.The learning rate starts at 0.001 for both datasets.Source: §4.2 · p. 5

4.3 Ablation Studies

• 2.15%/1.82% UAR/WAR on DFEW and 3.03%/3.30% on FERV39k: GCA plus IAL (setting d) exceeds the baseline (setting a).Replacing the ResNet18 building block with GCA and adding IAL each boost performance on both benchmarks.Source: §4.3 · p. 5, Table 1 · p. 5
• GCA exceeds SE by 0.65%/0.71% UAR/WAR on DFEW and 1.51%/1.00% on FERV39k; auxiliary classifiers improve all tested attention blocks.The authors attribute GCA’s advantage to preserving feature-location information by replacing global pooling with global convolution.Source: §4.3 · p. 5, Table 2 · p. 5

4.5 Visualization

• t-SNE shows clearer class boundaries on DFEW and more discriminative FERV39k features, especially for neutral and surprise, though FERV39k remains messy.On DFEW, neutral features learned by the method no longer sit at the center of non-neutral features.Source: Figure 4 · p. 6, §4.5 · p. 6

4.3 Ablation Studies

• Smaller IAL loss coefficients and larger GCA dimensionality-reduction ratios tend to perform better on both datasets.The authors say a large loss coefficient can overemphasize the most confusing category and weaken distinctions between other categories; larger ratios correspond to smaller parameter sizes.Source: Table 3 · p. 5, §4.3 · p. 6

4.4 Comparison with State-of-the-Arts

• 1.5%/1.05% UAR/WAR over NR-DFERNet on DFEW; on FERV39k, the method exceeds Former-DFER by 1.69% WAR with comparable UAR.The method achieves the best results in both DFEW metrics; on FERV39k, it achieves comparable UAR to Former-DFER.Source: §4.4 · p. 6

4.6 Discussions

• The proposed sequence is IAL finds low-intensity samples, then GCA enhances their features; accurate discovery and enhancement without extra supervision remain open challenges.The authors also note there is room to improve both steps without additional supervision information.Source: §4.6 · p. 6

• Experiments and visualization results demonstrate the method’s effectiveness and superiority.Source: §5 · p. 6
• GCA rescales feature-map channels to enhance target-related features in low-intensity sequences and suppress less important features, while IAL focuses on each sample’s most likely confusing class.Source: §5 · p. 6
• The work focuses on low-intensity samples in in-the-wild DFER and identifies expression intensity as an issue in discrete-label DFER.Source: §5 · p. 6
• The paper includes state-of-the-art comparison tables for DFEW and FERV39k.Source: Table 4 · p. 7, Table 5 · p. 7