PaperlayerView exact revision on arXiv

Source-linked AI summary

From Segments to Trajectories: Evolving Affective Graphs with Evidence Retrieval for Continuous EEG Emotion Recognition

Chi Yang, Jihong Wang, Chengxi Xie, Kai He, Huan Liu, Man Yao, Shile Qi, Yuzhe Zhang

arXiv:2609.30890v1cs.HC

TL;DR

Most EEG emotion-recognition methods assign trial labels to short segments, while emotional responses can change over time, motivating time-aligned whole-trial prediction. EAGER models evolving EEG spatial organization and retrieves multi-scale temporal evidence to predict affective trajectories. Across three datasets, it consistently improves trajectory-tracking metrics over representative baselines.

  • Problem

    Most existing EEG emotion-recognition methods assign each short segment its trial label, rather than predicting a time-aligned affective trajectory across the whole trial.

    Source: Introduction · pp. 1–2

  • Method

    EAGER combines evolving affective-graph modeling of EEG spatial organization with multi-scale temporal evidence retrieval to predict whole-trial trajectories.

    Source: Introduction · p. 2, Task Definition and Overall Framework · p. 3

  • Results

    Across MAHNOB-HCI, SEED-VII, and REFED, EAGER consistently improves trajectory-tracking metrics over representative baselines.

    Source: Conclusion · p. 7

  • Takeaways & Limitations

    Trajectory-level analysis indicates that EAGER tracks event-driven and subject-specific affective changes.

    Source: Conclusion · p. 8

    EAGER tracks the direction of strong idiosyncratic responses but can underestimate their magnitude.

    Source: Visualization Analysis · p. 7

Abstract

from arXiv · show

Electroencephalography (EEG)-based emotion recognition is important for affective computing and human-computer interaction, yet most existing methods divide a long trial into short segments and assign each segment the label of its source trial. Although this strategy increases the number of training samples, it reduces an evolving emotional response to a segment-level, coarse-grained, and static prediction problem. In reality, emotion may continuously emerge, intensify, weaken, and fluctuate as a stimulus unfolds, motivating the prediction of a time-aligned affective trajectory from the complete EEG trial. This task requires coordinated modeling of how spatial neural organization evolves throughout the trial and how local emotional fluctuations interact with longer-term trends. In this work, we formally define and systematically investigate continuous EEG emotion recognition as whole-trial affective trajectory prediction. We propose EAGER, an Evolving Affective Graph framework with Evidence Retrieval for continuous EEG emotion recognition. EAGER comprises two complementary modules: Affective State-guided Topology Evolution models the evolving spatial organization of EEG activity, while Multi-scale Temporal Evidence Retrieval integrates short-term fluctuations with longer-range temporal trends for time-aligned prediction. Experiments on MAHNOB-HCI, SEED-VII, and REFED show consistent gains in trajectory-tracking metrics over representative methods, with competitive pointwise errors.

Introduction

The paper reframes EEG emotion recognition as whole-trial, time-aligned affective trajectory prediction rather than static segment classification. It proposes EAGER, which combines evolving spatial graph modeling with multi-scale temporal evidence retrieval.

Introduction

  • Conventional methods split long trials into short segments that inherit a single trial label, increasing sample counts but reducing recognition to static, coarse-grained classification.Existing approaches often encode local windows and aggregate them over time, with limited coordination between transient fluctuations and slower trial-wide trends or evolving spatial organization.Source: Introduction · pp. 1–2
  • Continuous EEG emotion recognition predicts a time-aligned affective trajectory from a coherent whole trial, capturing when and how strongly emotion changes.The trajectory can also help locate emotionally salient moments and support content-preference analysis, highlight detection, and continuously responsive human-computer interaction.Source: Introduction · p. 2
  • EAGER jointly models evolving EEG spatial organization and multi-scale temporal evidence, treating each trial as a developing spatiotemporal process.Its two complementary stages are Affective State-guided Topology Evolution (ASTE) and Multi-scale Temporal Evidence Retrieval (MTER).Source: Introduction · p. 2
  • ASTE estimates short-term channel coupling, integrates mid- and long-range EEG contexts, and propagates a slowly varying graph state across windows to enhance each time step’s frequency-domain representation.Source: Introduction · p. 2
  • MTER retrieves evidence at multiple temporal resolutions so each prediction integrates local affective fluctuations with longer-term trial trends.Source: Introduction · p. 2
  • The study evaluates continuous trajectory recognition on MAHNOB-HCI, SEED-VII, and REFED, and reports representative-method comparisons, ablations, and qualitative trajectory analyses.The datasets provide continuous annotations of video-induced responses, time-varying basic-emotion intensities, and real-time valence and arousal trajectories, respectively.Source: Introduction · p. 2

Related Work

Prior EEG emotion-recognition work largely assigns static labels to fixed windows, while continuous approaches model temporal dependencies but do not maintain an evolving graph state across the full trial.

Conventional EEG Emotion Recognition

  • Conventional EEG emotion recognition mostly assigns a trial-level label to fixed windows, so its diverse methods still process segments independently.Approaches progressed from handcrafted time- or frequency-domain features to convolutional, recurrent, Transformer, graph, domain-adaptation, and contrastive-learning methods (Duan, Zhu, and Lu 2013; Song et al. 2020; Zhong, Wang, and Miao 2022; Ding et al. 2024a; Philip Chen, Chen, and Zhang 2025; Zhang et al. 2020; Song et al. 2023; Zhang et al. 2023; Xiao, Zheng, and Zhao 2025; Zhao, Yan, and Lu 2021; Wang, Zhang, and Tang 2024; Weng et al. 2026; Liu et al. 2025).Source: Conventional EEG Emotion Recognition · p. 2

Continuous EEG Emotion Recognition

  • Continuous EEG methods model dependencies across successive windows, but use fixed connectivity or separately reconstructed window graphs rather than an evolving whole-trial graph.Examples include temporal graph modeling, space-aware temporal layers, and temporal contextual Transformers; related work also incorporates visual, eye-movement, fNIRS, or facial cues (Ding and Guan 2023; Ding et al. 2024b; Ding et al. 2025; Zhang, Tang, and Guan 2022; Jiang et al. 2023; Si et al. 2025; Soleymani et al. 2016; Choi, Kim, and Song 2020).Source: Continuous EEG Emotion Recognition · p. 2

Method · Task Definition and Overall Framework

EAGER formulates continuous EEG emotion recognition as whole-trial, time-aligned affective trajectory prediction and uses evolving graph structure and multi-scale temporal evidence to produce the trajectory. The method maps complete-trial EEG features to predictions of the ground-truth emotional trajectory.

Task Definition and Overall Framework

  • EAGER replaces static segment-to-trial-label prediction with whole-trial, time-aligned affective trajectory prediction.The conventional paradigm maps an isolated short segment to a static trial-level label.Source: Task Definition and Overall Framework · pp. 2–3
  • The framework maps complete-trial raw EEG and its frequency-domain feature sequence to predictions approximating the ground-truth emotional trajectory.The raw signal has C channels and Traw sampling points; the feature sequence has T time steps, C channels, and F features per channel.Source: Task Definition and Overall Framework · p. 3

Method

  • ASTE evolves a trial-level channel graph and enhances frequency-domain features, while MTER retrieves multi-scale temporal evidence to produce the predicted trajectory.Each ASTE cell combines a Local Coupling Trace and Contextual Topology Gate to produce a dynamic graph and enhanced feature.Source: Task Definition and Overall Framework · p. 3, Figure 2 · p. 3
  • A separate scalar regression model is trained for each affective dimension provided by the dataset.Source: Task Definition and Overall Framework · p. 3
  • The method partitions T time steps into K long windows of length Ll, each divided into R short windows of length Ls, and indexes global, long-window, short-window, and local time.For the k-th long window, each local-step frequency feature, target, and prediction is denoted X(k,j), y(k,j), and ŷ(k,j).Source: Task Definition and Overall Framework · p. 3
  • For each short window, the raw EEG is denoted Xraw(k,r) and has C channels and Ss raw samples.The window spans local steps j = (r−1)Ls + 1 through rLs.Source: Task Definition and Overall Framework · p. 3

Affective State-guided Topology Evolution

ASTE addresses noisy short-term EEG connectivity and the limits of fixed graphs by evolving channel relationships over a trial and enhancing EEG features. Its components estimate local connectivity, use broader temporal context to guide graph fusion, and propagate graph information across long windows.

Affective State-guided Topology Evolution

  • ASTE combines short-window connectivity estimates, context-guided fusion, and graph-state propagation to model changing channel relationships and enhance features; its dynamic graphs are not causal connectivity estimates.The framework comprises Local Coupling Trace (LCT), Contextual Topology Gate (CTG), and Graph-State Evolution (GSE).Source: Affective State-guided Topology… · p. 3

Graph-State Initialization

  • At trial start, ASTE initializes its graph state from an identity matrix and a fixed electrode-distance prior, which is not added to later dynamic graphs.The prior is a nonlearnable Gaussian-kernel graph built from channel coordinates.Source: Graph-State Initialization · p. 3

Short-Window Local Coupling Trace

Contextual Topology Gate

  • CTG derives mid- and long-range contexts and uses them to generate time-step-specific fusion weights for historical graph state and current local connectivity.The long-range context change contributes to contextual change intensity; weights are independently generated at each time step even when local connectivity is fixed within a short window.Source: Contextual Topology Gate · p. 4

Graph-State Evolution and Feature Enhancement

  • GSE combines local connectivity with the preceding graph state, aggregates dynamic graphs to update state across long windows, and gates graph-propagated information into frequency-domain features.The retention parameter ρ controls how much graph state is retained across adjacent long windows.Source: Graph-State Evolution and Feature… · p. 4

Multi-scale Temporal Evidence Retrieval

MTER retrieves time-aligned evidence at multiple temporal scales from topology-enhanced representations, then refines the sequence for regression across the complete trial without updating the EEG graph.

Multi-scale Temporal Evidence Retrieval

  • MTER models complete-trial temporal dependencies through multi-scale contextual retrieval and temporal refinement while leaving the EEG graph unchanged.It operates on topology-enhanced representations.Source: Multi-scale Temporal Evidence Retrieval · p. 4
  • At scales 1, 2, and 4, multi-head attention uses pooled normalized inputs and learnable queries aligned to the original time axis to retrieve temporal evidence.The scale set is SMTER = {1, 2, 4}.Source: Multi-scale Temporal Evidence Retrieval · p. 4
  • Time-step-specific gates fuse evidence across scales, and a separate learnable scalar η injects the fused evidence into temporal refinement.η is a learnable scalar, not the output of a gating network.Source: Multi-scale Temporal Evidence Retrieval · p. 4
  • Temporal refinement uses normalization, temporal self-attention, feed-forward mapping, and residual connections; after Nenc layers, a learnable regression head predicts.The supplied passage does not specify the prediction output further.Source: Multi-scale Temporal Evidence Retrieval · p. 4

Training Objective and Inference

The primary objective combines concordance correlation coefficient (CCC) and mean squared error (MSE) losses. Auxiliary terms supervise contextual branches, contextual change intensity, and multi-scale query diversity.

Training Objective and Inference

  • The primary objective combines concordance correlation coefficient (CCC) and MSE losses.The CCC citation is given as Lin 1989.Source: Training Objective and Inference · p. 4
  • Auxiliary terms supervise the mid- and long-range context branches, contextual change intensity, and multi-scale query diversity.Their definitions and weights are provided in the supplementary material.Source: Training Objective and Inference · p. 5

Experiments

EAGER predicts whole-trial affective trajectories by combining evolving EEG topology with multi-scale temporal evidence. Across three datasets it leads in trajectory-tracking metrics, while analyses examine evaluation protocols, sensitivity, component contributions, and representative predictions.

Experiment Setup

  • MAHNOB-HCI and SEED-VII use LOSO validation, REFED uses subject-dependent 3-fold validation, and scoring concatenates trial sequences within each test partition.Results average 24, 20, and 96 runs, respectively; CCC/PCC measure trajectory agreement, while MAE/RMSE measure pointwise error. EAGER uses band-power features for MAHNOB-HCI and differential entropy for SEED-VII and REFED, with a Transformer backbone and nine baselines.Source: Experiment Setup · p. 5

Comparison Experiments

  • Across three datasets, GIGN and LSTM are less stable, while SVR and KNN perform worst, nearly failing on SEED-VII (CCC below 3); EAGER's coupled spatial-temporal method has consistent advantage.On REFED, all methods have CCC below 21, and EAGER achieves the highest CCC on both dimensions.Source: Comparison Experiments · pp. 5–6
  • EAGER achieves the best CCC/PCC across all three datasets and remains competitive on MAE/RMSE, with TSMMF the exception on pointwise errors.On REFED, CCC/PCC better reflect trajectory tracking because stepwise labels let near-mean predictions attain low MAE/RMSE; EAGER has the highest CCC on both dimensions. Most paired subject-level comparisons with deep-learning baselines on MAHNOB-HCI and SEED-VII yield p < 0.05.Source: Comparison Experiments · pp. 5–6, Table 2 · p. 6

Sensitivity Analysis

  • The selected settings are ρ = 0.7, temporal windows (4, 24, 64), and MTER scales {1, 2, 4}.Performance varies nonmonotonically with ρ; shorter or longer contexts degrade at least one metric, and adding scale 8 provides no additional benefit.Source: Figure 3 · p. 6, Sensitivity Analysis · p. 6

Ablation Study

  • ASTE removal causes the largest degradation, CTG and GSE each support all four metrics, and the full model performs best on every metric.Replacing multi-scale temporal retrieval with a plain LSTM yields the weakest overall performance; LCT is not ablated alone because it produces connectivity used by CTG and GSE.Source: Ablation Study · pp. 6–7

Visualization Analysis

  • Representative trials show EAGER tracking stimulus-linked changes and individualized responses, but attenuating the magnitude of a strong idiosyncratic deviation.In one trial it follows a subject’s positive response to a fear-inducing scene rather than the population median; in another, it tracks the direction of a strong deviation but remains closer to the population tendency.Source: Visualization Analysis · p. 7

Conclusion

EAGER frames continuous EEG emotion recognition as whole-trial, time-aligned affective trajectory prediction and combines evolving spatial modeling with multi-scale temporal evidence retrieval. It improves trajectory-tracking metrics over representative baselines, while trajectory-level analysis indicates it tracks event-driven and subject-specific affective changes.

Conclusion

  • EAGER consistently improves trajectory-tracking metrics over representative baselines on MAHNOB-HCI, SEED-VII, and REFED.Source: Conclusion · p. 7
  • EAGER formulates continuous EEG emotion recognition as whole-trial, time-aligned trajectory prediction, coupling evolving spatial organization with retrieval of local fluctuations and longer-term trends.Source: Conclusion · p. 7
  • Trajectory-level analysis indicates that EAGER tracks event-driven and subject-specific affective changes; future work targets causal streaming inference and better capture of extreme subject-specific responses.Source: Conclusion · p. 8

Related papers

Loading 2609.30890v1…