LEARNet Dynamic Imaging Network for Micro Expression Recognition

I. INTRODUCTION

• Micro-expressions last only 1/3 to 1/25 seconds and have low intensity, making them difficult to detect even for trained observers.The introduction reports 47% recognition accuracy for professionally trained observers.Source: §I · p. 1
• MER systems typically detect and preprocess faces, extract spatiotemporal features, then classify expressions; robust feature descriptors are essential to performance.The introduction notes that substandard features can increase false positives and false negatives, even with strong classifiers.Source: §I · p. 1
• The paper combines dynamic imaging, which represents video-sequence appearance and motion in one frame, with LEARNet for micro-expression classification.LEARNet uses laterally connected layers and accretion layers that combine prior lateral-layer responses to refine salient features.Source: §I · p. 2
• The proposed network targets involuntary micro-level changes and classifies expressions into basic emotion classes.The introduction describes dynamic imaging as transforming video sequences into a single frame while conserving spatiotemporal information.Source: §I · p. 2
• The method is evaluated on CASME-I, CASME-II, CAS(ME)^2, and SMIC benchmark datasets.The introduction says the paper compares its results with existing state-of-the-art methods.Source: §I · p. 2

II. LITERATURE SURVEY

• Prior work uses spatiotemporal descriptors including 3D-gradient histograms, LBP-TOP, TIM, DTSA, TICS, CBP, optical flow, and MDMD.The cited approaches variously describe temporal and spatial patterns, normalize video duration, model tensor information, or extract motion and appearance features.Source: §II · p. 2
• CNN-based approaches have been applied to micro-expression feature learning, including a model for spatiotemporal features and VGGfacenet features paired with evolutionary search.The survey cautions that a single-instance network would restrict its ability to learn micro-level features.Source: §II · p. 2

III. PROPOSED LEARNET

• Dynamic imaging turns a video into one image that retains spatial and temporal information, which LEARNet processes to classify the expression.The image is constructed from frame features ranked over time.Source: §III · pp. 2–3
• LEARNet processes inputs through four lateral pathways, combining sequential CNN paths with hybrid paths that accrete responses from earlier layers.The paths use 1×1, 3×3, and 5×5 filters; their responses are concatenated and passed to a 256-filter convolution layer and then a fully connected classifier.Source: Figure 2 · p. 3, §III · p. 4
• LEARNet uses stride-2 convolution rather than max pooling for downsampling because pooling can discard micro-level edges important for classification.The paper reports that stride-2 convolution outperforms max pooling in improving network learning for micro-expression recognition.Source: §III · pp. 4–5, Figure 5 · p. 6
• Accretion layers combine pertinent feature responses from earlier layers, enriching features that individual convolution layers may fail to capture.The accretion response joins preferable feature maps from two layers.Source: §III · pp. 4–5, Figure 4 · p. 5
• LEARNet normalizes the previous layer's output by subtracting the batch mean and dividing by the standard deviation.The batch mean and variation are calculated from the mini-batch.Source: §III · p. 4
• Independent lateral paths use decoupled feature maps to preserve minute facial muscle changes and distinguish emotion-class variations.The first convolution layer's disgust and happy responses are shown as an illustration.Source: Figure 3 · p. 4, §III · p. 5

Experimental setup

• Experiments used N-fold cross-validation with an 80:20 training/testing split, then divided training data 70:30 for training and validation.The reported dataset partitions included 183 CASME-I sequences, 251 CASME-II videos, 341 CAS(ME)^2 sequences, and 35- and 71-video SMIC subsets.Source: §IV · pp. 6–7
• Dynamic images were rotated from -45° to 45° in 15° increments, histogram-equalized, and horizontally flipped.The paper says data augmentation enlarged the data pool to avoid overfitting.Source: §IV · p. 6

Experimental results

• LEARNet exceeded ResNet by 4.23% on CASME-I, 2.08% on CASME-II, 1.85% on CAS(ME)^2, and 2.82% on SMIC-II.The authors also report gains over other CNN and handcrafted approaches; the comparison tables cover CASME-I, CASME-II, CAS(ME)^2, and SMIC.Source: §IV · pp. 6–7, Table III · p. 7, Table IV · p. 7

Qualitative analysis

• LEARNet responses highlighted expressive regions, including eyes, nose, and mouth, and the authors report preserved minute variations across facial regions.Reported examples include the glabella and eyes for tension, eyes and mouth for happiness, eyes and eyebrows for anger, and forehead, eyes, and lips for surprise.Source: §IV · p. 8

Computational complexity

• LEARNet has 1.4 million learnable parameters and 14 layers, compared with 11 million parameters and 34 layers for ResNet.The paper also reports parameter counts of 138M for VGG-16, 144M for VGG-19, and 4M for GoogleNet.Source: §IV · p. 8, Table V · p. 8

• Dynamic images compress sequence facial movements into one frame, which LEARNet processes through four independent paths, two using hybrid response maps.Source: §V · p. 8
• Figure 11 visually compares LEARNet with an existing model for Tension, Happy, Anger, and surprise across the four datasets.Source: Figure 11 · p. 9
• LEARNet gains 4.03%, 1.90%, 1.79%, and 2.82% accuracy over ResNet on CASME-I, CASME-II, CAS(ME)^2, and SMIC, respectively.Source: §V · p. 9
• LEARNet uses 1x1, 3x3, and 5x5 filters and accretion layers to merge response-map features; its small parameter count reduces training and testing time.Source: §V · p. 9