PaperlayerView exact revision on arXiv

Source-linked AI summary

ViTexSZ: Heterogeneous Vision-Text Knowledge Distillation for EEG Seizure Detection

Chenxi Liu, Mingzhao Li, Yicong Liu, Hao Miao, Hongyuan Zhang, Ziyi Chen, Gaofeng Meng

arXiv:2608.21445v1cs.CV

TL;DR

EEG seizure detection must handle heterogeneous channel layouts and subtle subclinical events. ViTexSZ aligns waveform-image representations, grounds them with clinical prompts through a multimodal teacher, and distills the resulting knowledge into a lightweight student. Across four datasets, it achieved the highest accuracy on all datasets and reported relative improvements of up to 12.9% over second-best baselines.

  • Problem

    Fixed-channel EEG methods have limited applicability to heterogeneous recordings, while aligning EEG representations with clinical semantics remains challenging for subtle subclinical seizures.

  • Method

    ViTexSZ converts EEG into waveform images, aligns modality-specific features into unified tokens, and distills a clinical-prompt-conditioned multimodal teacher into a lightweight student.

  • Results

    12.9% and 7.9% relative improvements over second-best baselines were reported in sensitivity and F1 score, respectively, while accuracy was highest on all four datasets.

  • Takeaways & Limitations

    The framework generalizes across heterogeneous EEG settings and is particularly effective for subclinical seizure detection.

  • Takeaways & Limitations

    The study’s dataset collection and use were governed by institutional-review-board approvals and the original ethical approvals of the external datasets.

Abstract

from arXiv · show

Automated seizure detection from electroencephalography (EEG) is essential for continuous neurological monitoring, particularly for subclinical epileptic seizures that may exhibit only subtle electrographic changes. Existing time-series methods are often designed for fixed EEG channel configurations, thereby limiting their applicability to heterogeneous EEG recordings with irregular channel layouts. Although visual and language modeling offer promising alternatives, aligning heterogeneous EEG representations with clinical semantics remains challenging. We introduce ViTexSZ, a heterogeneous Vision-Text knowledge distillation framework for EEG seizure detection. ViTexSZ converts EEG recordings into structured waveform images and introduces a query-based multi-channel alignment module that maps source-dependent visual features into a unified token space. A heterogeneous teacher further integrates the aligned EEG representations with clinical prompts through a multimodal large language model, associating high-level clinical semantics with seizure-related evidence. Vision-text knowledge distillation then transfers the teacher representations to a lightweight student during detection. Experiments on four EEG seizure datasets demonstrate the generalizability of ViTexSZ across both subclinical and general seizure detection scenarios, achieving the highest accuracy on all datasets and relative improvements of up to 12.9% over the second-best baselines, showing its effectiveness.

Introduction

ViTexSZ addresses the difficulty of detecting seizures across heterogeneous EEG channel configurations, especially when subclinical events produce subtle electrographic changes. It combines waveform-image representations, multi-channel alignment, clinical prompts, and vision–text distillation to improve cross-setting detection.

  • Motivation: Subclinical seizures may show only subtle electrographic changes, making timely automated detection important for continuous neurological monitoring.Reliable detection can support subsequent clinical assessment and intervention.
  • Motivation: Existing EEG methods model cross-channel or multi-scale temporal dependencies but often depend on fixed channel configurations.Vision and language modeling provide complementary ways to represent EEG and incorporate textual knowledge.
  • Challenge: Heterogeneous EEG recordings vary in channel configurations and spatial coverage, producing inconsistent visual structures and feature distributions across acquisition settings.These differences limit the generalization of standard architectures without costly redesign.
  • Framework: ViTexSZ converts EEG signals into waveform images and uses query-based multi-channel alignment to project source-dependent visual features into a unified token space.The alignment module is designed to support knowledge transfer across varying channel configurations.
  • Framework: A heterogeneous teacher integrates aligned EEG representations with clinical prompts through a multimodal large language model before transferring multimodal knowledge to a lightweight student.The framework grounds seizure-related visual patterns in high-level clinical semantics and retains lightweight detection.
  • Results: 12.9% and 7.9% relative improvements over second-best baselines were achieved in sensitivity and F1 score, respectively, on the reported heterogeneous evaluation.Experiments covered four EEG seizure datasets and both subclinical and seizure detection tasks.

Related Work

Prior EEG seizure methods capture channel interactions and temporal dependencies, but their learned spatial structures are often tied to specific montages. This sensitivity to channel variation motivates approaches that can generalize across heterogeneous EEG configurations.

  • Time-series methods: Adaptive graphs and synchronization-aware aggregation model cross-channel dependencies, while attention-based methods capture multi-scale temporal dependencies.These approaches represent spatial interactions and seizure evolution at different temporal resolutions.
  • Limitations: Spatial structures learned by time-series methods are often coupled to specific channel configurations, making them sensitive to montage variation and missing channels.This limits robustness when EEG layouts differ across recordings.

Preliminaries

The paper formulates seizure detection over heterogeneous, potentially unpaired scalp and intracranial EEG segments augmented with clinical prompts. It distinguishes normal activity, subclinical seizures, and clinical seizures while defining teacher-to-student prediction transfer.

  • Heterogeneous EEG: Each EEG segment is a temporally continuous multichannel time series, with scalp and intracranial modalities represented by modality-specific channel-by-time matrices.The dataset contains N EEG segments, and channel counts may differ between modalities.
  • Heterogeneous EEG: The two EEG modalities may be unavailable in some sources, although at least one modality is observed for each segment.This captures the heterogeneous and potentially unpaired nature of the recordings.
  • Clinical prompts: A clinical prompt is a textual sequence combining relevant electronic health-record context with detection instructions and seizure-event information.The prompt supplies clinically relevant context for the detection task.
  • Seizure events: A normal segment has seizure label y_n=0, whereas y_n=1 denotes a seizure; seizure segments are further typed as subclinical or clinical events.Subclinical seizures lack observable clinical manifestations and may be subtle in scalp EEG, unlike clinical seizures with overt signs.
  • Detection objective: The teacher predicts seizure state from available EEG modalities and clinical prompts, while its knowledge is transferred to a lightweight detector for inference.The problem definition separates teacher inputs from the student’s deployment-time prediction.

Overall Framework

ViTexSZ combines a heterogeneous teacher, vision–text knowledge distillation, and a lightweight student. The teacher produces multimodal representations using EEG images, clinical prompts, and heterogeneous inputs, while the student learns efficient detection from privileged training information.

  • Heterogeneous teacher: The heterogeneous teacher uses an MLLM, multi-channel alignment, and a vision–text encoder to generate high-quality vision–text representations.Its inputs include EEG images and clinical prompts.
  • Knowledge distillation: Vision–text knowledge distillation transfers teacher representations to the student using iEEG and MLLM-derived knowledge as privileged information available only during training.The transfer bridges heterogeneous teacher information and lightweight student inference.
  • Lightweight student: The lightweight student combines a pretrained encoder, FiLM module, and vision encoder to learn from the teacher’s privileged representations.This design enables efficient seizure detection.

Heterogeneous Teacher Model

The heterogeneous teacher converts available EEG modalities into visual representations, aligns them into fixed-length tokens, and integrates them with clinical prompts through a frozen multimodal LLM. Transformer refinement and a projection head produce seizure predictions from the aligned representations.

  • Multimodal Encoding: Each available scEEG or iEEG time series is converted into an EEG image and jointly encoded with clinical prompts by a frozen multimodal LLM.The resulting hidden states use visual and text encoders plus a projector.
  • Multi-Channel Alignment: Learnable queries aggregate modality-specific EEG tokens into Nq aligned tokens despite differing scalp and intracranial channel configurations.Trainable query, key, and value projections compute query-to-token similarities for modality-specific retrieval.
  • Multi-Channel Alignment: Retrieved seizure-relevant information from each modality is fused into a fixed-length aligned sequence through projection and residual addition.The aligned representation is denoted Am.
  • Teacher Refinement: A trainable vision encoder with L Transformer layers refines aligned tokens to capture seizure-task-specific interactions.The final output is the teacher representation used downstream.
  • Prediction: A trainable projection head maps the final teacher tokens directly to seizure predictions.

Vision-Text Knowledge Distillation

Vision-text knowledge distillation transfers multimodal seizure representations from a heterogeneous teacher to a lightweight student. Training uses aligned token matrices, while cross-patient inference retains only the student and scEEG input.

  • Teacher–Student Roles: The teacher uses MLLM-derived visual-text knowledge and, when available, iEEG information, whereas the student uses only scEEG and clinical prompts.
  • Knowledge Transfer: Token-level distillation transfers seizure-relevant teacher representations to the lightweight student without adding inference cost.
  • Distillation Objective: For each available EEG modality, teacher and student token matrices are aligned with a normalized Smooth L1 objective.The Smooth L1 penalty is applied element-wise, with stop-gradient used in the formulation.
  • Distillation Objective: The optimization balances seizure detection and vision-text knowledge distillation using λVTKD, while optimized teacher parameters remain frozen during student training.
  • Inference: During cross-patient inference, the heterogeneous teacher branch is removed and the student predicts from scEEG and the prompt.

Lightweight Student Model

The lightweight student performs efficient, noninvasive seizure detection using scEEG and clinical prompts. It converts image patches into matched tokens, modulates them with prompt-derived parameters, refines them with a compact Transformer encoder, and predicts seizures.

  • Student Design: The student uses scEEG as its only physiological input and injects clinical semantic priors through prompt-conditioned feature modulation.This design targets efficient and noninvasive detection with limited computational overhead.
  • Tokenization: Non-overlapping scEEG image patches are projected into visual tokens and merged into Nq student tokens.
  • Tokenization: The student token count matches the teacher representation, enabling direct token-level knowledge transfer.
  • Prompt Conditioning: A pretrained text encoder and trainable multilayer perceptron generate channel-wise modulation parameters γ and β from the clinical prompt.
  • Prompt Conditioning: Feature-wise linear modulation broadcasts γ and β across tokens to adapt seizure-relevant feature dimensions to the clinical prompt.
  • Student Prediction: A lightweight encoder with LS Transformer layers refines the modulated tokens, whose representation is supervised by LVTKD and mapped to seizure predictions.

Experiments

ViTexSZ is evaluated across four EEG seizure-detection datasets, with cross-subject testing, ablations, sensitivity analysis, representation visualization, and a real-time clinical case study. The results show strong performance across heterogeneous and subclinical seizure settings, while component analyses identify the teacher, channel alignment, and privileged iEEG supervision as important contributors.

  • Datasets: The evaluation covers four datasets spanning paired iEEG–scEEG recordings, pediatric scEEG, and clinical scEEG, with heterogeneous channel configurations and expert seizure annotations.Dual-SCS targets subclinical seizures, Dual-CS targets clinical seizures, CHB-MIT provides pediatric scEEG, and TUSZ contains 2,663 seizure events from 238 patients.
  • Overall performance: ViTexSZ achieves the highest accuracy on all four datasets and ranks first in 11 of 16 dataset–metric combinations.Performance is reported under a cross-subject setting using five patient-independent data splits.
  • Overall performance: 12.9% and 7.9% relative improvements over the corresponding second-best baselines are reported on Dual-CS for sensitivity and F1 score, respectively.The reported gains concern paired heterogeneous EEG data.
  • Ablation study: Removing the heterogeneous teacher decreases AUC by 7.9 percentage points, while removing multi-channel alignment decreases sensitivity by 13.1% in Dual-SCS.Removing iEEG reduces AUC by 6.6% and 2.6% across the evaluated datasets, and clinical prompts provide complementary semantic guidance.
  • Hyperparameter analysis: The selected configuration uses 10,496 queries, 6 Transformer layers, 8 attention heads, λVTKD = 1.0, and hidden dimension D = 256.Increasing the query count from 656 to 10,496 improves Acc, F1, and AUC, while further increasing Transformer depth or attention heads provides no clear gain.
  • Real-time case study: In a previously unseen 3-hour clinical scEEG recording, ViTexSZ achieves an AUPRC of 0.83 and an F1 score of 0.85 with approximately 9.8 ms inference per 10-s window.The case study evaluates chronological online predictions for real-time subclinical seizure detection.

Conclusion

ViTexSZ combines heterogeneous EEG waveform representations with clinical semantics and transfers the resulting vision–text representations to a lightweight seizure-detection student. Experiments on heterogeneous EEG datasets demonstrate its effectiveness and generalizability, particularly for subclinical seizure detection.

  • ViTexSZ converts heterogeneous EEG recordings into waveform images and projects source-dependent representations into a unified token space through multi-channel alignment.
  • A heterogeneous teacher associates seizure-related visual patterns with clinical semantics through a multimodal large language model.
  • Token-level knowledge distillation transfers the resulting vision–text representations to a lightweight student for seizure detection.
  • Experiments on heterogeneous EEG datasets demonstrate ViTexSZ's effectiveness and generalizability, particularly for subclinical seizure detection.
Loading 2608.21445v1…