PaperlayerView exact revision on arXiv

Source-linked AI summary

EEG-to-Report: An Annotation and Feature-Text Framework for Training Language Models on Clinical EEG

Xuan-The Tran, Le Trung Kien Nguyen

arXiv:2608.26153v1cs.AI

TL;DR

Clinical EEG interpretation is time-consuming and specialized, while existing tools provide limited support for aligned feature–text supervision. EEG-to-Report combines browser-based EEG review, multimodal annotation, quantitative feature extraction, structured export, and report drafting; pilot use produced segment-level corpora and editable draft-report functionality, but evaluation remains preliminary and the planned feature-to-text model is not yet implemented.

  • Problem

    Clinical EEG interpretation is complex and time-consuming, and existing software ecosystems provide limited support for generating aligned feature–text data for language-model training.

  • Method

    EEG-to-Report integrates multi-format EEG ingestion, interactive segment annotation with typed or transcribed text, automated feature extraction, JSON export, and an auto-report module.

  • Results

    Pilot annotations produced 112 segment-level annotations across 36 recordings and 12 patients, with 1322-feature vectors for typical 36-channel montages.

  • Takeaways & Limitations

    The framework provides a reusable, vendor-agnostic workflow for local centres to build and curate EEG feature–text corpora while keeping clinicians in the loop.

  • Takeaways & Limitations

    Evaluation is preliminary, limited to one institution and few annotators and cases, and no model has yet learned narratives directly from the segment-level feature–text corpus.

Abstract

from arXiv · show

Clinical electroencephalography (EEG) reporting remains largely manual and time-consuming, and current EEG software ecosystems do not produce the structured EEG-text supervision needed for training modern language models. Most toolboxes focus on visualization or preprocessing, providing limited support for workflows that generate high-quality datasets for AI. We introduce EEG-to-Report, a browser-based annotation and feature-text framework that links routine EEG review with the construction of AI-ready datasets. The framework integrates multi-format EEG ingestion, channel standardization, and an interactive viewer with a multimodal annotation layer that combines typed text and transcribed voice notes. For each annotated segment, a feature extraction engine computes a standardized set of spectral, temporal, entropy, Hjorth, connectivity, and spike-related descriptors, stored alongside clinical descriptions in a portable JSON schema. This yields aligned feature-text pairs designed to supervise multimodal EEG-language models. The framework also includes an auto-report module that couples an ensemble of convolutional networks with a large language model to draft clinical narratives for neurologist review. Using pilot annotations, we describe how EEG-to-Report streamlines annotation workflows and produces editable draft reports, providing a reusable foundation for automated EEG reporting systems.

1 Introduction

EEG-to-Report addresses gaps in clinical EEG tooling by combining vendor-agnostic review, segment-level annotation, quantitative feature extraction, and structured clinical text for AI-ready datasets. It also provides an auto-report demonstrator while keeping development human-in-the-loop.

  • Clinical EEG interpretation is complex, time-consuming, and dependent on unevenly distributed specialized expertise, motivating computer-assisted reporting tools.
  • Existing systems often depend on proprietary curated databases or commercial platforms, while common open-source toolboxes prioritize signal processing rather than aligned feature–text data.
  • EEG-to-Report integrates multi-format ingestion, channel standardization, interactive segment selection, multimodal annotation, feature extraction, and dataset export in one browser-based workflow.
  • Each annotated segment is paired with quantitative time–frequency and connectivity features and typed or transcribed clinical text in a reusable JSON representation.
  • The framework includes an auto-report module that combines convolutional-network ensembles with a large language model to draft clinical reports for review.
  • The framework is intended to connect standardized annotation traditions such as SCORE with foundation and clinical language models for interpretable, human-in-the-loop EEG AI.

2 Results

On the Siena database, EEG-to-Report supported standardized ingestion, segment-level annotation, feature–text export, and qualitative inspection of auto-generated reports. The demonstration produced a structured corpus and draft-report workflow, while voice annotation, multi-expert annotation, and quantitative report evaluation remained future work.

  • Data Ingestion and Annotation Workflow: The application parsed Siena EDF recordings, standardized channel names to a 10–20-based convention, and excluded non-EEG channels from the main viewer while retaining inspection access.Non-EEG channels such as ECG or trigger channels remained available through the channel-selection dialog.
  • Data Ingestion and Annotation Workflow: 112 segment-level annotations were created across 36 recordings and 12 patients, covering ictal, peri-ictal, and interictal windows.Typical annotation windows were 20.0 s, with shorter segments selected for focal patterns.
  • Data Ingestion and Annotation Workflow: All demonstration annotations used direct text rather than voice input, and the demonstration used a single annotator despite support for speech-to-text and multi-expert annotation.Speech-versus-text usability, medical speech-to-text accuracy, and inter-rater variability studies were planned for future work.
  • EEG Feature–Text Corpus Statistics: Each saved annotation became a JSON segment object containing timing, channels, quantitative EEG features, and free-text clinical descriptions, with doctor’s notes stored separately.The schema supports portable feature–text corpus construction for downstream machine learning.
  • EEG Feature–Text Corpus Statistics: The feature engine computed spectral, statistical, Hjorth, entropy, spike-related, and coherence descriptors, producing 1322 features per segment for typical 36-channel montages.Computed features included delta through gamma band powers, Shannon and approximate entropy, spike counts, and pairwise channel coherence.
  • EEG Feature–Text Corpus Statistics: Siena segment descriptions were seeded programmatically from seizure annotations and computed features because the database lacked narrative clinical descriptions.The seed text was intended for clinician review and refinement rather than treatment as expert-authored narrative.
  • Auto-Report Generation: The auto-report module estimated quantitative descriptors with a convolutional ensemble and prompted a large language model to generate narrative drafts inspected by three expert readers.Because paired ground-truth reports were unavailable, assessment was qualitative rather than based on BLEU, ROUGE-L, or BERTScore.
  • Auto-Report Generation: Systematic categorization of generated-text failure modes and formal real-time clinician usability assessment were left to future research.Examples include omitted subtle findings, over-generalized phrasing, and rare institutional terms.

3 Discussion

EEG-to-Report presents a vendor-agnostic, human-in-the-loop workflow that links segment-level clinical annotation, quantitative feature extraction, and language-model report generation. Its feature–text schema is intended to support portable, shareable EEG corpora, while preliminary scope and modeling limitations remain.

  • Discussion: EEG-to-Report prioritizes an integrated, vendor-agnostic workflow and data representation over a single black-box classifier.The framework combines multi-format ingestion, clinician-centred annotation, feature extraction, and language-model reporting in one application.
  • Implications for EEG Auto-Reporting and Neuroinformatics: Clinician-authored annotations and editable draft reports keep experts involved during corpus creation and final report review.The auto-report module returns Word or PDF drafts for clinician revision rather than fixed verdicts.
  • From Standardised Reporting to AI-Ready Feature–Text Corpora: The feature–text schema represents each annotated segment as a JSON object containing timing, channel context, structured features, and clinical text.Its model-agnostic interface can support architectures beyond the BART-based encoder–decoder, including T5, LLaMA-based encoders, and multimodal transformers.
  • From Standardised Reporting to AI-Ready Feature–Text Corpora: Feature–text objects may be more compact and easier to anonymise than raw EEG, supporting multi-centre sharing when raw signals cannot be exchanged.Shared feature–text pairs and annotation metadata could support collaborative training, cross-site evaluation, and analysis of inter-rater variability.
  • Implications for EEG Auto-Reporting and Neuroinformatics: Feature-based models may produce clinically plausible draft descriptions for a non-trivial fraction of segments, potentially reducing documentation time and standardising phrasing.The paper frames generated text as a scaffold for clinician refinement, not necessarily as text ready for direct use.
  • Limitations: The empirical evaluation is preliminary, limited to one institution, few annotators, and limited patient cases, without testing across hospitals, acquisition systems, or languages.The authors call for larger multi-centre studies to assess generalisation, efficiency gains, and potential biases.
  • Limitations: The current feature set is hand-crafted and may be less expressive than raw-waveform or end-to-end learned representations.Connectivity and spike detection use relatively simple algorithms, while richer methods could increase representation quality at greater complexity.
  • Limitations: The paper has not yet trained a model directly from the segment-level feature–text corpus to generate narratives or hierarchical study-level reports.Such feature-to-text and hierarchical modeling are identified as important next steps.

4 Methods

EEG-to-Report is implemented as a browser-based client–server application with modular EEG processing, annotation, feature extraction, dataset export, and auto-report components. Its two modules communicate through REST endpoints and can operate independently of the user interface.

  • System Overview: The browser-based client–server application runs on modern operating systems without local installation while keeping signal processing on the server.The frontend uses React and TypeScript with Plotly.js for interactive multi-channel rendering.
  • System Overview: The backend is a Python FastAPI service using MNE-Python for EEG processing, NumPy/SciPy for quantitative features, and locally hosted Whisper for speech-to-text.The frontend and backend communicate through a REST API, while EEG data and derived artefacts remain under institutional server control.
  • System Modules: The annotation module imports and standardises multi-format EEG, supports interactive segment- and channel-level multimodal annotation, extracts per-segment features, and exports aligned JSON feature–text objects.Typed text and transcribed voice are included in the annotation workflow.
  • System Modules: The auto-report module computes spectral features, applies an ensemble of convolutional networks, and generates reports through a language model.The demonstrator includes CNN, GoogleNet, and ResNet components.
  • System Architecture: The annotation and auto-report modules are decoupled through REST endpoints, allowing feature extraction and dataset export to be scripted independently of the interface.The annotation module is organised into presentation, application, domain, and infrastructure layers.

4.2 Data and Participants

The demonstration used a de-identified public Siena scalp EEG dataset, while EEG-to-Report supports standardized ingestion and configurable preprocessing across multiple clinical EEG formats.

  • Data and Participants: The demonstration used de-identified scalp EEG recordings from 14 patients evaluated for epilepsy in Siena’s neurology unit.Recordings used a standard 10–20 referential montage and 512 Hz sampling rate.
  • Data and Participants: All signal processing runs server-side, keeping raw EEG under institutional control.The browser frontend communicates with a FastAPI backend through a REST API.
  • Data and Participants: EEG-to-Report loads EDF/BDF/GDF, EEGLAB, MNE-FIF, BrainVision, Neuroscan CNT, and MATLAB array files through format-specific loaders.Each file is standardized into an MNE Raw object while preserving channel names, sampling frequency, and basic metadata.
  • Data and Participants: During import, channel labels are mapped to standardized 10–20 names where possible, while non-EEG channels remain available through channel management.EOG, EMG, ECG, and trigger channels are excluded from the default EEG view based on name patterns and channel-type hints.
  • Data and Participants: The default preprocessing pipeline applies configurable filtering, notch removal, re-referencing, and resampling operations to the active recording.Pipeline parameters are exposed in the interface for reproducibility across recordings.

4.4 Interactive Visualisation and Annotation Workflow

The interactive workflow combines multi-format review, segment-and-channel selection, typed or dictated annotation, and navigation through saved annotations to create aligned dataset entries.

  • Interactive Visualisation and Annotation Workflow: Module 1 carries recordings from ingestion and preprocessing through interactive review, multimodal annotation, per-segment feature extraction, and JSON export.The dataset-creation pipeline stores each annotated segment as an aligned feature–text pair.
  • Interactive Visualisation and Annotation Workflow: The viewer displays stacked EEG traces with adjustable time windows, vertical scaling, and channel-selection colors.Users can view windows from 1–60 seconds, with a default of 10 seconds.
  • Interactive Visualisation and Annotation Workflow: Navigation controls support stepping through recordings, changing the time window, and jumping across the full duration with a timeline overview.The viewer adapts to the number of channels in the current montage.
  • Interactive Visualisation and Annotation Workflow: Clinicians select a temporal interval and one or more channels by dragging across the viewer and clicking traces before opening the annotation dialog.If no channel is explicitly selected, the annotation applies to all displayed channels.
  • Interactive Visualisation and Annotation Workflow: Annotations can contain typed descriptions or recorded voice notes transcribed by Whisper into an editable text field.The original audio may optionally be stored for later review.
  • Interactive Visualisation and Annotation Workflow: Saved annotations retain segment times, channels, annotation type, display color, timestamps, and a unique identifier, while bars and a list panel support later navigation.The list panel includes short text snippets for annotations in the current recording.

4.5 Quantitative Feature Extraction

For each saved annotation, EEG-to-Report computes standardized spectral, temporal, entropy, connectivity, and spike-related features from selected channels and intervals.

  • Quantitative Feature Extraction: For every saved annotation, the system extracts a quantitative feature vector from selected channels and the selected time interval.Features are computed on preprocessed data using NumPy, SciPy, and MNE routines.
  • Quantitative Feature Extraction: Spectral features include absolute or relative delta, theta, alpha, beta, and gamma band powers derived with Welch PSD estimation.The engine also derives the dominant frequency band and, where applicable, alpha peak frequency.
  • Quantitative Feature Extraction: Channel-level temporal features comprise mean amplitude, standard deviation, Hjorth mobility, Hjorth complexity, and optional linear trend.These descriptors summarize amplitude and variability characteristics within each segment.
  • Quantitative Feature Extraction: Connectivity is represented by magnitude-squared coherence across channel pairs, stored as a flattened upper-triangle vector or summary statistics.Optional summaries include average intrahemispheric and inter-hemispheric coherence.
  • Quantitative Feature Extraction: Spike-related features count samples exceeding a configurable z-score threshold, defaulting to 3 standard deviations.Counts are stored per channel and per segment.

4.6 Feature–Text Schema and JSON Dataset

EEG-to-Report stores annotated segments and case-level doctor’s notes in a portable JSON schema that aligns clinical descriptions with structured quantitative features.

  • Feature–Text Schema and JSON Dataset: The exported dataset contains segment-level annotation objects and a separate case-level doctor’s-notes object.The JSON-based format is designed for portability and model training.
  • Feature–Text Schema and JSON Dataset: Each segment object records identifiers, times, channels, annotation type, display color, clinical description, and grouped quantitative features.Feature groups include spectral powers, temporal statistics, Hjorth parameters, entropy, coherence, and spike-related descriptors.
  • Feature–Text Schema and JSON Dataset: EEG-to-Report can import and export the JSON files and merge temporally adjacent annotations using a configurable time-gap threshold.Merging concatenates descriptions and takes the union of channels; richer multi-expert conflict resolution is planned.

4.7 Report Generation and Feature-to-Text Modelling

EEG-to-Report links current auto-reporting with a planned feature-to-text training route. It exports aligned quantitative feature–clinical text pairs while keeping corpus-trained report generation as future work.

  • Report Generation and Feature-to-Text Modelling: The framework connects EEG features to language models through an available in-application auto-report demonstrator and a planned multimodal EEG–language-model training route.The exported dataset is intended to support feature-to-text modelling independently of the deployed application.
  • Report Generation and Feature-to-Text Modelling: The auto-report module averages predictions from three convolutional architectures to estimate quantitative EEG descriptors for a structured large-language-model prompt.The architectures are a plain CNN, an Inception-style GoogleNet, and a residual ResNet.
  • Report Generation and Feature-to-Text Modelling: Each annotated segment is stored as an aligned feature–text pair containing clinical description and structured descriptors such as band powers, temporal statistics, entropy, coherence, and spike counts.The schema stores timing, channels, segment text, quantitative features, and separate case-level doctor’s notes.
  • Planned Feature-to-Text Modelling: The planned route would normalize and partition each feature vector, project feature groups into encoder space, and fine-tune a BART encoder–decoder to generate narrative descriptions.Training would use patient-level splitting, teacher forcing, cross-entropy loss, and BART tokenization.
  • Planned Feature-to-Text Modelling: A full implementation, hyperparameter search, and evaluation of corpus-trained feature-to-text generation remain beyond the present paper’s scope.The paper focuses on the annotation tool and dataset representation that enable such training.

4.8 Evaluation Protocol

The evaluation protocol tests technical feasibility and characterizes annotation behavior using public clinical EEG data. Clinical report quality, text-generation metrics, and broader usability or generalisability evaluations are deferred to future work.

  • Evaluation Protocol: The application was tested for importing Siena Scalp EEG Database recordings and standardizing channel labels and metadata.This assesses technical feasibility on public clinical data.
  • Evaluation Protocol: Annotation behavior was characterized through annotation counts, segment durations, text lengths, vocabulary distributions, and voice-versus-text input proportions.These statistics describe how annotations were created and distributed across recordings.
  • Evaluation Protocol: Expert qualitative assessments of auto-report drafts and reference-based text-similarity metrics are planned because paired reference reports are unavailable.Planned analyses also include ablation and subsampling studies of a corpus-trained feature-to-text model.
  • Evaluation Protocol: Dedicated usability studies and larger multi-centre evaluations are required to characterize workflow benefits, performance, and generalisability.The current protocol is intended to demonstrate technical feasibility rather than fully establish these outcomes.

Declarations

The study reports no competing interests and uses the publicly available, fully de-identified Siena Scalp EEG Database for secondary analysis without new human-participant data collection.

  • Declarations: The authors declare that they have no competing interests.
  • Declarations: The analysis used the publicly available, fully de-identified Siena Scalp EEG Database and involved no new human-participant data collection.The stated use complied with the database licence and PhysioNet data-use policy.
Loading 2608.26153v1…