PaperlayerView exact revision on arXiv

Source-linked AI summary

AOI-Net: Structural Face AOI-Guided Eye-Gaze Track Representation Learning for Autism Spectrum Disorder Detection

Zhanpei Huang, Binbin Sun, Jialiang Chen, Yiou Wang, Taochen Chen, Yuzhu Ji, Yiqun Zhang, Yiu-Ming Cheung

arXiv:2608.29289v1cs.CVcs.AIcs.ETcs.MM

TL;DR

Existing ASD gaze-analysis methods incompletely capture the event-driven temporal and AOI-structural organization of eye movements, while clinical datasets are often imbalanced. AOI-Net jointly models local temporal dynamics and AOI structure, adaptively fuses their representations, and is reported to outperform state-of-the-art methods on a large clinical database; its current evaluation remains bounded by short, sparse sequences and hospital cohorts.

  • Problem

    Existing methods often emphasize temporal sequences or aggregate AOI statistics while neglecting AOI relationships, and clinical eye-movement datasets commonly exhibit class imbalance.

    Source: §I · p. 1, C. Eye-Movement Data Analysis for… · p. 4, C. Graph Construction from… · p. 5

  • Method

    AOI-Net combines a temporal expert for short-range fixation dynamics with a structural expert for AOI-based relational modeling, then fuses them through a gating mechanism.

    Source: §I · p. 2, D. Dual-Expert Fusion Module · p. 6

  • Results

    AOI-Net consistently outperforms state-of-the-art methods on a large-scale clinical eye-tracking database with more than 1,300 participants.

    Source: §V · p. 13

  • Takeaways & Limitations

    The framework provides interpretable gaze-behavior modeling for ASD-related assessment and a practical basis for scalable screening applications.

    Source: §V · p. 13

    The evaluation uses short, sparse gaze sequences from hospital cohorts, while multiscale temporal modeling remains future work.

    Source: §I · p. 12

Abstract

from arXiv · show

Eye-movement tracking has emerged as a promising non-invasive approach to Autism Spectrum Disorder (ASD) screening, with systematic differences in attentional allocation and revisit behaviors observed during socially interactive tasks. Existing computational methods typically characterize eye-movements using discrete gaze trajectories and fixation events, yielding representations dominated by short-range temporal dynamics and limiting models that primarily emphasize long-range dependencies. Meanwhile, gaze behavior is naturally organized across semantically meaningful Areas of Interest (AOIs), whose attention allocation and transitions provide important structural cues, yet their relationships are rarely modeled explicitly. To address these limitations, we propose a structural face AOI-guided Eye-Gaze Track Network (AOI-Net) that jointly models short-term temporal dynamics and AOI-level structural organization. A network gating mechanism adaptively integrates the complementary temporal and structural representations according to their contributions to gaze-behavior characterization. To mitigate the pronounced class imbalance commonly encountered between individuals with ASD and Typically Developing (TD) participants in clinical datasets, class-distribution-aware learning is further employed to facilitate discriminative embedding learning under skewed class distributions. Experiments on a unique and large-scale clinical eye-tracking database comprising eight stimulus subsets and more than 1,300 participants show that AOI-Net consistently outperforms state-of-the-art methods. The proposed framework also enables interpretable gaze-behavior modeling and provides a practical basis for scalable AI-driven ASD screening in real-world healthcare. The code is available at https://github.com/Zhanpei-ai/CIM-AOI-Net/tree/main/Code

I. INTRODUCTION

ASD screening is motivated by the limits of labor-intensive clinical assessment and by gaze differences during socially rich tasks. AOI-Net addresses gaps in existing gaze models by jointly representing local fixation dynamics and AOI structure.

  • Clinical ASD assessment relies on trained experts and is time-consuming and difficult to scale to large populations.Source: §I · p. 1
  • Eye movements provide a non-invasive signal because ASD and TD children differ in social attention allocation and visual-focus shifts during interactive tasks.Source: §I · p. 1
  • Existing approaches capture static gaze statistics or temporal dependencies but often discard fixation order, local event dynamics, or AOI transition structure.Source: §I · p. 2
  • AOI-Net jointly models short-term fixation dynamics and AOI-level organization through complementary temporal and structural representations.Source: §I · p. 2
  • A graph-based strategy models relational connections among fixation events to capture attention allocation and gaze interactions associated with ASD.Source: §I · p. 2
  • The framework is evaluated on an imbalanced clinical database containing more than 1,300 participants under dynamic video-based interaction paradigms.Source: §I · p. 3

II. RELATED WORK

Related work spans handcrafted gaze descriptors, general-purpose temporal models, and AOI summaries. These approaches often fail to represent the event-driven nature of eye movements together with relational AOI organization and clinical class imbalance.

III. PROPOSED METHOD

The proposed method treats gaze as an event-driven fixation sequence and combines local temporal modeling with AOI-level structural modeling. Its temporal encoder emphasizes proximal fixation dependencies to preserve rapid attentional dynamics.

A. Problem Formulation

  • Eye-movement recordings are represented as ordered fixation events, with each fixation encoded as a d-dimensional vector.Source: A. Problem Formulation · p. 4
  • The learning objective maps an event-driven fixation sequence to a clinical diagnostic label for binary ASD screening.Source: A. Problem Formulation · p. 4

B. Temporal Modeling of Eye-Movement

C. Graph Construction from Eye-Movement Sequences

The graph construction represents eye movements as fixation events linked by both temporal adjacency and AOI-based structural relationships. This preserves local gaze dynamics while explicitly modeling recurrent, semantically organized attention patterns.

  • AOIs represent semantically meaningful regions where fixation identity, sustained inspection, revisitation, and region-specific attention are organized.Source: C. Graph Construction from… · p. 5
  • Eye-movement behavior includes recurrent fixations and cyclic transitions across AOIs that may span non-adjacent time steps.Source: C. Graph Construction from… · p. 5
  • Directly concatenating categorical AOI identities with continuous gaze features creates a representational mismatch between symbolic region membership and measurable behavioral attributes.Source: C. Graph Construction from… · p. 5
  • AOI-based edges connect fixation events sharing a region regardless of temporal distance, enabling structural information propagation among semantically consistent fixations.Source: C. Graph Construction from… · p. 5
  • Temporal adjacency edges preserve fixation ordering, while GNN message passing jointly propagates information across temporal and AOI-based relationships.Source: C. Graph Construction from… · p. 6
  • The graph topology is motivated by ASD-related gaze patterns including unstable attention maintenance, fragmented exploration, and recurrent shifts toward non-social or peripheral regions.Source: C. Graph Construction from… · p. 6

D. Dual-Expert Fusion Module

The dual-expert fusion module combines temporal and structural views of gaze behavior. A mixture-of-experts gate dynamically weights these complementary representations according to each input sequence.

  • Temporal transitions and AOI-level coherence provide complementary information arising from oculomotor control and higher-level attentional allocation.Source: D. Dual-Expert Fusion Module · p. 6
  • A mixture-of-experts mechanism dynamically regulates specialized submodule contributions instead of statically concatenating temporal and structural embeddings.Source: D. Dual-Expert Fusion Module · p. 6
  • The Temporal Expert encodes short-range dependencies between adjacent fixations, whereas the Structural Expert uses AOI-augmented graph message passing to capture cross-temporal regional coherence.Source: D. Dual-Expert Fusion Module · p. 6
  • Learnable gating coefficients softly allocate representational capacity between dynamic transitions and region-level organization on a per-sample basis.Source: D. Dual-Expert Fusion Module · p. 6
  • The two experts encode complementary relational scales: local oculomotor transitions and AOI-semantic attentional organization.Source: D. Dual-Expert Fusion Module · p. 7

E. Model Training and Inference

Model training combines class-distribution-aware representation learning with expert-load regularization. The resulting fused representation is used for ASD classification while addressing class skew and preventing routing dominance.

IV. EXPERIMENT

The experiments evaluate AOI-Net across detection performance, interpretability, ablations, efficiency, temporal scale, and generalization. Supplementary analyses additionally examine training, participant statistics, sensitivities, and fusion choices.

  • The evaluation covers comparative ASD detection, AOI importance, prototype behavior, ablation, GNN depth, efficiency, temporal modeling scale, and dataset-exclusion generalization.Source: §IV · p. 8
  • Supplementary materials provide the training algorithm, participant statistics, η and AOI-importance sensitivity analyses, and fusion-strategy and gate-activation ablations.Source: §IV · p. 8

A. Experimental Setup

The experiments use eight clinically collected eye-movement datasets and evaluate AOI-Net against time-series and medical time-series baselines with multiple classification metrics.

Model Configuration

  • AOI-Net analyzes eye-movement data through temporal and structural experts integrated with Class-Distribution-Aware Learning.η is set to 0.25, µ to 0.01, and Adam is used across all datasets.Source: A. Experimental Setup · p. 8

Counterparts and Metrics

  • Compared methods include general time-series models, a medical time-series classifier, and eye-movement or clinical-signal baselines.Evaluation uses F1-score, Accuracy, Sensitivity, and Specificity.Source: A. Experimental Setup · p. 8

Dataset

  • The study uses eight datasets—Speaking, Walking1, Walking2, Helicopter, Baby, Tablet, Attention, and Sad—from Shenzhen Maternity and Child Healthcare Hospital.ASD participants were clinically diagnosed, while TD participants were screened for known neurodevelopmental disorders.Source: A. Experimental Setup · p. 8

B. Comparative Study of ASD Detection

AOI-Net is evaluated across eight datasets and representative ROC analyses, with results indicating strong overall detection, screening sensitivity, and discrimination relative to competing methods.

Overall Performance

  • AOI-Net consistently achieves superior ACC and F1-score across all eight datasets, often exceeding the second-best methods by substantial margins.The reported result is described as robust across heterogeneous eye-movement patterns elicited by different paradigms.Source: B. Comparative Study of ASD Detection · p. 8, Table I · p. 9

Model Comparison

  • General-purpose Transformer models underperform AOI-Net, while Detach Rocket performs relatively better but still lags behind it.The comparison attributes AOI-Net’s advantage to jointly modeling local temporal dynamics and AOI structural organization.Source: B. Comparative Study of ASD Detection · p. 8

Screening Reliability

  • AOI-Net achieves the best or near-best Sensitivity across most datasets while maintaining competitive Specificity.This trade-off is presented as appropriate for screening, where missed ASD cases can delay early intervention and diagnosis.Source: B. Comparative Study of ASD Detection · pp. 8–9

Behavioral Graphs

  • Behavioral graphs show denser, more balanced transitions across socially salient facial regions for TD participants than for ASD participants.ASD graphs show weaker connectivity among socially relevant AOIs and stronger concentration toward peripheral or non-social regions.Source: Figure 4 · p. 9

ROC Analysis

C. Analysis of AOI Importance and Prototype Behavior

AOI-Net’s prototype graphs distinguish denser, more balanced TD gaze transitions from restricted ASD facial exploration, while ablations show that AOI and temporal relational modeling both contribute to performance. Predefined AOI semantic edges provide stable structural guidance compared with alternative edge-construction strategies.

Prototype behavior

  • TD participants show denser, more balanced transitions across socially salient facial regions, whereas ASD participants show more restricted facial exploration.The strongest contrast involves transitions between the two eyes and other central facial areas.Source: C. Analysis of AOI Importance and… · p. 10

AOI importance

  • Removing AOI-based edges causes discernible performance degradation, indicating that explicit AOI relationships support ASD detection.AOI edges encode scanpath strategies describing transitions between semantically significant regions.Source: D. Ablation Study · p. 10

Temporal and structural modeling

  • Removing temporal edges causes a more pronounced degradation than retaining temporal connections without AOI edges, highlighting the structural expert’s role in fixation relations.Without temporal edges and graph message passing, the model becomes purely sequential and lacks explicit fixation-to-fixation interactions.Source: D. Ablation Study · p. 10

Joint modeling

  • Simply providing AOI features to a temporal module is insufficient; explicit GNN relational modeling captures structured dependencies that sequential modeling alone cannot fully recover.Integrating these components captures joint spatial-semantic and relational dynamics in autistic gaze behavior.Source: D. Ablation Study · p. 10

AOI edge construction

  • Predefined AOI edges outperform KNN, dynamic, and weighted alternatives by preserving stable semantic organization among attentional regions.KNN connections may include less relevant fixations, while flexible learned edges can weaken consistent AOI-level semantic relationships.Source: D. Ablation Study · pp. 10–11

E. Impact of GNN Propagation Depth

GNN propagation depth has a non-monotonic effect: shallow propagation improves detection, whereas excessive depth degrades performance by diluting local attentional patterns. AOI-Net also maintains competitive training efficiency and low inference latency across datasets.

Propagation depth

  • Detection performance first improves and then declines as the number of GNN layers increases.Moderate structural aggregation strengthens AOI-level attentional representations, while excessive propagation expands feature mixing.Source: E. Impact of GNN Propagation Depth · p. 11
  • Shallow graph propagation is sufficient to aggregate behaviorally meaningful information without diluting discriminative local attentional patterns.The findings support controlled graph propagation for localized ASD-related gaze behaviors.Source: E. Impact of GNN Propagation Depth · p. 11

Efficiency

  • AOI-Net demonstrates competitive training efficiency across datasets while TimesNet and MPTSNet incur significantly higher training costs.Most transformer-based baselines maintain relatively low and stable training times.Source: F. Efficiency Analysis · p. 11
  • AOI-Net achieves stable, low inference latency comparable to or better than several existing methods.This efficiency is maintained while modeling both temporal dynamics and AOI-level structure.Source: F. Efficiency Analysis · p. 11

G. Impact Analysis of Temporal Modeling Scales

Temporal-scale analysis supports localized modeling: expanding the localized temporal kernel from 3 to 9 continuously reduces performance. The framework’s current scope is short, sparse hospital-cohort gaze data, with broader temporal dependencies and community-screening shifts left for future evaluation.

Temporal modeling scale

  • Performance continuously degrades as the localized temporal kernel size expands from 3 to 9.The analysis uses the temporal branch alone to assess the utility of longer-range modeling.Source: G. Impact Analysis of Temporal… · p. 12

Generalization

  • The generalization analysis evaluates missing-stimulus conditions to assess whether learned ASD-related gaze representations are shared across paradigms.A designated paradigm is removed from the test set while retained during training.Source: H. Generalization Analysis under… · p. 12, Figure 8 · p. 12

Temporal modeling scale

  • AOI-Net’s temporal strategy focuses on local fixation transitions and recurrent gaze patterns because current task recordings are generally short and sparse.The paper does not exclude potential value from long-range dependencies and plans multiscale temporal modeling as future work.Source: §I · p. 12

Generalization

  • The evaluation is based on hospital cohorts, so community screening may introduce differences in participant distributions, ASD prevalence, and recording conditions.These variations could create distribution shifts affecting model calibration and generalization.Source: §I · p. 12

Framework scope

  • AOI-Net couples short-term gaze dynamics with semantically meaningful AOI topology and adaptively fuses the resulting behavioral signals.The framework is validated on a clinical database containing more than 1,300 participants across eight visual paradigms.Source: §V · pp. 12–13

Related papers

Loading 2608.29289v1…