Source-linked AI summary
AOI-Net: Structural Face AOI-Guided Eye-Gaze Track Representation Learning for Autism Spectrum Disorder Detection
Zhanpei Huang, Binbin Sun, Jialiang Chen, Yiou Wang, Taochen Chen, Yuzhu Ji, Yiqun Zhang, Yiu-Ming Cheung
TL;DR
Existing ASD gaze-analysis methods incompletely capture the event-driven temporal and AOI-structural organization of eye movements, while clinical datasets are often imbalanced. AOI-Net jointly models local temporal dynamics and AOI structure, adaptively fuses their representations, and is reported to outperform state-of-the-art methods on a large clinical database; its current evaluation remains bounded by short, sparse sequences and hospital cohorts.
Problem
Existing methods often emphasize temporal sequences or aggregate AOI statistics while neglecting AOI relationships, and clinical eye-movement datasets commonly exhibit class imbalance.
Method
AOI-Net combines a temporal expert for short-range fixation dynamics with a structural expert for AOI-based relational modeling, then fuses them through a gating mechanism.
Results
AOI-Net consistently outperforms state-of-the-art methods on a large-scale clinical eye-tracking database with more than 1,300 participants.
Takeaways & Limitations
The framework provides interpretable gaze-behavior modeling for ASD-related assessment and a practical basis for scalable screening applications.
Takeaways & Limitations
The evaluation uses short, sparse gaze sequences from hospital cohorts, while multiscale temporal modeling remains future work.
Abstract
from arXiv · showhide
Eye-movement tracking has emerged as a promising non-invasive approach to Autism Spectrum Disorder (ASD) screening, with systematic differences in attentional allocation and revisit behaviors observed during socially interactive tasks. Existing computational methods typically characterize eye-movements using discrete gaze trajectories and fixation events, yielding representations dominated by short-range temporal dynamics and limiting models that primarily emphasize long-range dependencies. Meanwhile, gaze behavior is naturally organized across semantically meaningful Areas of Interest (AOIs), whose attention allocation and transitions provide important structural cues, yet their relationships are rarely modeled explicitly. To address these limitations, we propose a structural face AOI-guided Eye-Gaze Track Network (AOI-Net) that jointly models short-term temporal dynamics and AOI-level structural organization. A network gating mechanism adaptively integrates the complementary temporal and structural representations according to their contributions to gaze-behavior characterization. To mitigate the pronounced class imbalance commonly encountered between individuals with ASD and Typically Developing (TD) participants in clinical datasets, class-distribution-aware learning is further employed to facilitate discriminative embedding learning under skewed class distributions. Experiments on a unique and large-scale clinical eye-tracking database comprising eight stimulus subsets and more than 1,300 participants show that AOI-Net consistently outperforms state-of-the-art methods. The proposed framework also enables interpretable gaze-behavior modeling and provides a practical basis for scalable AI-driven ASD screening in real-world healthcare. The code is available at https://github.com/Zhanpei-ai/CIM-AOI-Net/tree/main/Code
I. INTRODUCTION
ASD screening is motivated by the limits of labor-intensive clinical assessment and by gaze differences during socially rich tasks. AOI-Net addresses gaps in existing gaze models by jointly representing local fixation dynamics and AOI structure.
- Clinical ASD assessment relies on trained experts and is time-consuming and difficult to scale to large populations.
- Eye movements provide a non-invasive signal because ASD and TD children differ in social attention allocation and visual-focus shifts during interactive tasks.
- Existing approaches capture static gaze statistics or temporal dependencies but often discard fixation order, local event dynamics, or AOI transition structure.
- AOI-Net jointly models short-term fixation dynamics and AOI-level organization through complementary temporal and structural representations.
- A graph-based strategy models relational connections among fixation events to capture attention allocation and gaze interactions associated with ASD.
- The framework is evaluated on an imbalanced clinical database containing more than 1,300 participants under dynamic video-based interaction paradigms.
II. RELATED WORK
Related work spans handcrafted gaze descriptors, general-purpose temporal models, and AOI summaries. These approaches often fail to represent the event-driven nature of eye movements together with relational AOI organization and clinical class imbalance.
- General-purpose time-series models emphasize long-range dependencies under assumptions of continuous evolution or stable temporal structure.
- Eye-movement recordings are event-driven, with discrete fixation states and rapid saccadic transitions that differ from conventional continuous time series.
- Handcrafted eye-movement descriptors summarize fixation and saccade statistics but discard temporal ordering.
- AOI-based studies provide interpretable attention summaries but commonly overlook relational and transitional organization across regions.
- Clinical eye-movement datasets can exhibit class imbalance because some conditions are underrepresented.
III. PROPOSED METHOD
The proposed method treats gaze as an event-driven fixation sequence and combines local temporal modeling with AOI-level structural modeling. Its temporal encoder emphasizes proximal fixation dependencies to preserve rapid attentional dynamics.
- A. Problem Formulation: Eye-movement recordings are represented as ordered fixation events, with each fixation encoded as a d-dimensional vector.
- A. Problem Formulation: The learning objective maps an event-driven fixation sequence to a clinical diagnostic label for binary ASD screening.
- B. Temporal Modeling of Eye-Movement: Adjacent fixation transitions are emphasized because they encode rapid shifts in attentional focus, whereas distant aggregation may obscure fine-grained dynamics.
- B. Temporal Modeling of Eye-Movement: A locality-constrained temporal strategy uses a Temporal Convolutional Network to capture short-range fixation dynamics rather than expansive global interactions.
- B. Temporal Modeling of Eye-Movement: The temporal encoding aggregates bounded neighborhood offsets with nonlinear activation in a residual convolutional module.
- B. Temporal Modeling of Eye-Movement: This localized encoder produces compact representations of stable, behaviorally meaningful gaze transitions without introducing spurious long-range dependencies.
C. Graph Construction from Eye-Movement Sequences
The graph construction represents eye movements as fixation events linked by both temporal adjacency and AOI-based structural relationships. This preserves local gaze dynamics while explicitly modeling recurrent, semantically organized attention patterns.
- AOIs represent semantically meaningful regions where fixation identity, sustained inspection, revisitation, and region-specific attention are organized.
- Eye-movement behavior includes recurrent fixations and cyclic transitions across AOIs that may span non-adjacent time steps.
- Directly concatenating categorical AOI identities with continuous gaze features creates a representational mismatch between symbolic region membership and measurable behavioral attributes.
- AOI-based edges connect fixation events sharing a region regardless of temporal distance, enabling structural information propagation among semantically consistent fixations.
- Temporal adjacency edges preserve fixation ordering, while GNN message passing jointly propagates information across temporal and AOI-based relationships.
- The graph topology is motivated by ASD-related gaze patterns including unstable attention maintenance, fragmented exploration, and recurrent shifts toward non-social or peripheral regions.
D. Dual-Expert Fusion Module
The dual-expert fusion module combines temporal and structural views of gaze behavior. A mixture-of-experts gate dynamically weights these complementary representations according to each input sequence.
- Temporal transitions and AOI-level coherence provide complementary information arising from oculomotor control and higher-level attentional allocation.
- A mixture-of-experts mechanism dynamically regulates specialized submodule contributions instead of statically concatenating temporal and structural embeddings.
- The Temporal Expert encodes short-range dependencies between adjacent fixations, whereas the Structural Expert uses AOI-augmented graph message passing to capture cross-temporal regional coherence.
- Learnable gating coefficients softly allocate representational capacity between dynamic transitions and region-level organization on a per-sample basis.
- The two experts encode complementary relational scales: local oculomotor transitions and AOI-semantic attentional organization.
E. Model Training and Inference
Model training combines class-distribution-aware representation learning with expert-load regularization. The resulting fused representation is used for ASD classification while addressing class skew and preventing routing dominance.
- Clinical ASD eye-tracking datasets can be highly imbalanced, causing majority-class bias and weakening minority-class embedding geometry.
- Class-Distribution-Aware Learning combines compact same-class embeddings with separated different-class embeddings through a class-aware representation objective.
- Inverse-frequency weighting increases the loss contribution of minority-class samples to counteract majority-class dominance.
- This imbalance mechanism improves sensitivity to underrepresented behavioral patterns.
- The unified imbalance objective balances representation-level regularization and classification-level reweighting through η ∈[0, 1].
- Expert-load regularization penalizes persistent routing asymmetry while allowing data-driven adaptation and preserving expert specialization.
- The fused representation is passed to task-specific classification layers after temporal modeling, AOI graph reasoning, and adaptive expert fusion.
IV. EXPERIMENT
The experiments evaluate AOI-Net across detection performance, interpretability, ablations, efficiency, temporal scale, and generalization. Supplementary analyses additionally examine training, participant statistics, sensitivities, and fusion choices.
- The evaluation covers comparative ASD detection, AOI importance, prototype behavior, ablation, GNN depth, efficiency, temporal modeling scale, and dataset-exclusion generalization.
- Supplementary materials provide the training algorithm, participant statistics, η and AOI-importance sensitivity analyses, and fusion-strategy and gate-activation ablations.
A. Experimental Setup
The experiments use eight clinically collected eye-movement datasets and evaluate AOI-Net against time-series and medical time-series baselines with multiple classification metrics.
- Model Configuration: AOI-Net analyzes eye-movement data through temporal and structural experts integrated with Class-Distribution-Aware Learning.η is set to 0.25, µ to 0.01, and Adam is used across all datasets.
- Counterparts and Metrics: Compared methods include general time-series models, a medical time-series classifier, and eye-movement or clinical-signal baselines.Evaluation uses F1-score, Accuracy, Sensitivity, and Specificity.
- Dataset: The study uses eight datasets—Speaking, Walking1, Walking2, Helicopter, Baby, Tablet, Attention, and Sad—from Shenzhen Maternity and Child Healthcare Hospital.ASD participants were clinically diagnosed, while TD participants were screened for known neurodevelopmental disorders.
B. Comparative Study of ASD Detection
AOI-Net is evaluated across eight datasets and representative ROC analyses, with results indicating strong overall detection, screening sensitivity, and discrimination relative to competing methods.
- Overall Performance: AOI-Net consistently achieves superior ACC and F1-score across all eight datasets, often exceeding the second-best methods by substantial margins.The reported result is described as robust across heterogeneous eye-movement patterns elicited by different paradigms.
- Model Comparison: General-purpose Transformer models underperform AOI-Net, while Detach Rocket performs relatively better but still lags behind it.The comparison attributes AOI-Net’s advantage to jointly modeling local temporal dynamics and AOI structural organization.
- Screening Reliability: AOI-Net achieves the best or near-best Sensitivity across most datasets while maintaining competitive Specificity.This trade-off is presented as appropriate for screening, where missed ASD cases can delay early intervention and diagnosis.
- Behavioral Graphs: Behavioral graphs show denser, more balanced transitions across socially salient facial regions for TD participants than for ASD participants.ASD graphs show weaker connectivity among socially relevant AOIs and stronger concentration toward peripheral or non-social regions.
- ROC Analysis: AOI-Net’s ROC curves dominate competing approaches over most operating regions and achieve the largest AUC values on Speaking and Walking1.These results indicate stronger discrimination across varying decision thresholds.
C. Analysis of AOI Importance and Prototype Behavior
AOI-Net’s prototype graphs distinguish denser, more balanced TD gaze transitions from restricted ASD facial exploration, while ablations show that AOI and temporal relational modeling both contribute to performance. Predefined AOI semantic edges provide stable structural guidance compared with alternative edge-construction strategies.
- Prototype behavior: TD participants show denser, more balanced transitions across socially salient facial regions, whereas ASD participants show more restricted facial exploration.The strongest contrast involves transitions between the two eyes and other central facial areas.
- AOI importance: Removing AOI-based edges causes discernible performance degradation, indicating that explicit AOI relationships support ASD detection.AOI edges encode scanpath strategies describing transitions between semantically significant regions.
- Temporal and structural modeling: Removing temporal edges causes a more pronounced degradation than retaining temporal connections without AOI edges, highlighting the structural expert’s role in fixation relations.Without temporal edges and graph message passing, the model becomes purely sequential and lacks explicit fixation-to-fixation interactions.
- Joint modeling: Simply providing AOI features to a temporal module is insufficient; explicit GNN relational modeling captures structured dependencies that sequential modeling alone cannot fully recover.Integrating these components captures joint spatial-semantic and relational dynamics in autistic gaze behavior.
- AOI edge construction: Predefined AOI edges outperform KNN, dynamic, and weighted alternatives by preserving stable semantic organization among attentional regions.KNN connections may include less relevant fixations, while flexible learned edges can weaken consistent AOI-level semantic relationships.
E. Impact of GNN Propagation Depth
GNN propagation depth has a non-monotonic effect: shallow propagation improves detection, whereas excessive depth degrades performance by diluting local attentional patterns. AOI-Net also maintains competitive training efficiency and low inference latency across datasets.
- Propagation depth: Detection performance first improves and then declines as the number of GNN layers increases.Moderate structural aggregation strengthens AOI-level attentional representations, while excessive propagation expands feature mixing.
- Propagation depth: Shallow graph propagation is sufficient to aggregate behaviorally meaningful information without diluting discriminative local attentional patterns.The findings support controlled graph propagation for localized ASD-related gaze behaviors.
- Efficiency: AOI-Net demonstrates competitive training efficiency across datasets while TimesNet and MPTSNet incur significantly higher training costs.Most transformer-based baselines maintain relatively low and stable training times.
- Efficiency: AOI-Net achieves stable, low inference latency comparable to or better than several existing methods.This efficiency is maintained while modeling both temporal dynamics and AOI-level structure.
G. Impact Analysis of Temporal Modeling Scales
Temporal-scale analysis supports localized modeling: expanding the localized temporal kernel from 3 to 9 continuously reduces performance. The framework’s current scope is short, sparse hospital-cohort gaze data, with broader temporal dependencies and community-screening shifts left for future evaluation.
- Temporal modeling scale: Performance continuously degrades as the localized temporal kernel size expands from 3 to 9.The analysis uses the temporal branch alone to assess the utility of longer-range modeling.
- Generalization: The generalization analysis evaluates missing-stimulus conditions to assess whether learned ASD-related gaze representations are shared across paradigms.A designated paradigm is removed from the test set while retained during training.
- Temporal modeling scale: AOI-Net’s temporal strategy focuses on local fixation transitions and recurrent gaze patterns because current task recordings are generally short and sparse.The paper does not exclude potential value from long-range dependencies and plans multiscale temporal modeling as future work.
- Generalization: The evaluation is based on hospital cohorts, so community screening may introduce differences in participant distributions, ASD prevalence, and recording conditions.These variations could create distribution shifts affecting model calibration and generalization.
- Framework scope: AOI-Net couples short-term gaze dynamics with semantically meaningful AOI topology and adaptively fuses the resulting behavioral signals.The framework is validated on a clinical database containing more than 1,300 participants across eight visual paradigms.