PaperlayerView exact revision on arXiv

Source-linked AI summary

AutoBCI: Forecast-Guided Agentic Neural Architecture Discovery for EEG-Based Brain--Computer Interfaces

Muyun Jiang, Yi Ding, Wei Zhang, Jinbo Chen, Chenyu Liu, Zhenjie Yang, Yuxin Li, Jingyuan Chen, Yuhao Lu, Yong Li, Shuailei Zhang, Cuntai Guan

arXiv:2609.35456v1cs.AI

TL;DR

EEG tasks differ in spatial, temporal, and label structure, and selecting suitable architectures across tasks remains a manual design burden. AutoBCI uses PGAD to generate and refine architectures using cross-task validation, and PEEK to forecast full-budget performance from architecture code, training protocol, and early learning curves. AutoBCI’s strongest discovered architecture reached 64.16% average test bAcc, 0.29 pp above REVE; after ten epochs, PEEK reduced forecast MAE by 38.1%.

  • Problem

    EEG tasks differ in spatial, temporal, and label structure, and selecting suitable architectures across tasks remains a manual design burden.

    Source: §1 · p. 2

  • Method

    AutoBCI uses PGAD to generate and refine architectures using cross-task validation, and PEEK to forecast full-budget performance from architecture code, training protocol, and early learning curves.

    Source: §1 · p. 2

  • Results

    AutoBCI’s strongest discovered architecture reached 64.16% average test bAcc, 0.29 pp above REVE; after ten epochs, PEEK reduced forecast MAE by 38.1%.

    Source: §5 · p. 10

  • Takeaways & Limitations

    AutoBCI’s search improved the best validation score in each of six LLM-guided searches, and its strongest architecture exceeded the strongest baseline on average test bAcc.

    Source: §5 · p. 10

    Architecture selection requires completed full-budget training runs across all task families; partially trained candidates are ineligible for parent or champion selection.

    Source: §2.1 · p. 3

Abstract

from arXiv · show

EEG-based brain-computer interfaces support a broad range of applications, yet designing decoding architectures that perform well across diverse tasks remains challenging. We introduce AutoBCI, an agentic framework in which a Designer Agent and a Forecaster Agent support the discovery and selection of EEG decoding architectures across tasks. The Designer Agent performs Pool-Guided Architecture Discovery (PGAD), generating and refining architectures through training and validation across multiple EEG tasks, such as emotion recognition, motor imagery, and sleep staging. The Forecaster Agent performs Performance Estimation from Early Knowledge (PEEK), using architecture code, the training protocol, and early learning curves to predict full-budget validation performance and select promising candidates for continued training. Across 14 EEG datasets spanning motor imagery, emotion recognition, and sleep staging, we evaluate AutoBCI with six LLMs, including Opus 5.5 and GPT 5.6 Sol, and compare the architectures selected by the search procedure against ten baselines: six conventional EEG models and four foundation models. The architecture discovered by AutoBCI with Claude Opus 5.5 achieves 64.16% average test balanced accuracy (bAcc), compared with 63.87% for REVE, the strongest baseline on this metric. Using ten observed epochs, PEEK reduces mean absolute error in predicting average validation bAcc from 2.20 to 1.36 percentage points, a 38.1% reduction relative to the best-observed-score baseline.

1 Introduction

EEG decoding architectures must accommodate tasks with different spatial, temporal, and label structures, yet choosing suitable designs across tasks remains a manual burden. AutoBCI combines agentic architecture discovery and early-performance forecasting to search for shared architectures across EEG tasks.

Motivation

  • Architecture search remains manual across EEG tasks despite EEG-specific models and agent systems that automate analysis workflows rather than model design.EEG decoding spans different spatial coverage, temporal scales, and label spaces, and architecture design reflects human choices.Source: §1 · p. 2
  • Automated search on separately optimized tasks does not establish whether one shared architecture performs well across tasks, motivating joint validation and selection.The study evaluates emotion recognition, motor imagery, and sleep staging; evaluating proposals across tasks increases discovery training cost.Source: §1 · p. 2
  • Forecast accuracy alone does not establish reliable candidate selection; screening must retain candidates that perform well after full training.AutoBCI predicts full-training performance from architecture code and early learning curves, then assesses both forecast accuracy and retention of strong candidates.Source: §1 · p. 2

Contributions

  • AutoBCI pairs PGAD, which generates and refines EEG architectures using cross-task validation, with PEEK, which forecasts full-budget performance to guide continued training.PGAD ranks a persistent candidate pool using joint task evidence; PEEK combines architecture code, training protocol, and early learning curves.Source: §1 · p. 2
  • 269 architectures were searched across six LLM-guided runs; the strongest champion reached 64.16% average test bAcc, 0.29 percentage points above the strongest baseline.PEEK reduced forecast MAE by 38.1% after ten epochs; retaining three candidates per round preserved a full-budget winner in 91.7% of rounds, with an estimated 44.9% fewer training epochs.Source: §1 · p. 3

2 Method

AutoBCI joins pool-guided architecture discovery with early-performance forecasting to generate, evaluate, refine, and screen EEG architectures. PGAD selects shared candidates using cross-task validation, while PEEK forecasts full-budget performance to allocate continued training.

2 Method

  • AutoBCI’s Designer Agent uses PGAD to generate and refine architectures, while its Forecaster Agent uses PEEK to predict full-budget validation performance and select candidates for continued training.PEEK combines architecture code, the training protocol, and early learning curves.Source: §2 · p. 3

2.1 Pool-Guided Architecture Discovery

  • PGAD evaluates each architecture separately across task families, gives datasets equal weight within each task, and assigns equal weight to task scores.A task score is the highest equal-dataset mean validation bAcc at a common checkpoint over the full training run; different tasks may select different checkpoint epochs.Source: §2.1 · p. 3
  • Only candidates completing the full training budget on all tasks are eligible; each round refines top-scoring pool parents and adds fresh proposals.Refinements use parent code, learning curves, and task-specific validation results; validation selects parents and the champion, while test data are reserved for final evaluation.Source: §2.1 · pp. 3–4, Figure 2 · p. 4

2.2 Performance Estimation from Early Knowledge (PEEK)

  • PEEK combines candidate code, training protocol, and early learning history to forecast full-budget validation scores, then continues training the highest-ranked candidates.Forecast targets match the best equal-dataset validation bAcc at a common checkpoint within the budget; only fully completed candidates enter the PGAD pool.Source: §2.2 · pp. 4–5

3 Experimental Setup

The experiments assess architecture discovery and early candidate selection across 14 EEG datasets grouped into motor imagery, emotion recognition, and sleep staging. AutoBCI uses a shared search and evaluation protocol and compares discovered architectures with conventional and foundation-model baselines.

3.1 Dataset construction

  • The 14 datasets are grouped into motor imagery, emotion recognition, and sleep staging, with task-wise channel layouts and labels harmonized while validation and test scores remain dataset-specific.MI and emotion recordings use a 65-channel template, while sleep recordings use an eight-channel template; fixed partitions and sleep subsets are shared across candidates.Source: §3.1 · p. 5, Table 1 · p. 6

3.2 Architecture search protocol

  • Each search runs six discovery rounds of eight proposals across three task families, with fewer than one million trainable parameters per task-specific model and recurrent architectures excluded.Later rounds refine two selected parents and add fresh proposals; only candidates completing all three tasks are eligible for selection.Source: §3.2 · p. 5
  • Six LLMs conduct architecture searches using their respective command-line interfaces, with DeepSeek and Qwen accessed through Anthropic-compatible endpoints via Claude Code.Initial proposals use task specifications and constraints; later refinements draw on selected parents’ architecture code and validation curves, while fresh proposals retain exploration.Source: §3.2 · pp. 5–6, Table 2 · p. 6

3.3 Training and evaluation protocol

  • Candidates are trained independently for 40 epochs on each task family using AdamW, batch size 512, and full BF16 precision, with validation performance selecting task checkpoints.Validation balanced accuracy uses equal dataset weights for checkpoint selection and equal task weights for architecture selection; test data are reserved for final evaluation.Source: §3.3 · p. 6

3.4 Baseline models and comparison settings

  • The comparison includes ten baselines: six conventional EEG models trained from scratch and four pretrained foundation models fine-tuned using the same task partitions and validation-based checkpoint selection.Test evaluation reports balanced accuracy, class-support-weighted F1, and Cohen’s Kappa with equal dataset averaging.Source: §3.4 · p. 6, §3.3 · p. 6

4 Results

Across held-out tests, Opus leads on average bAcc, while task leaders differ. PEEK improves ten-epoch forecasts, and retaining multiple candidates can preserve winners while reducing estimated training cost.

4.1 Performance of discovered architectures

  • Opus achieves the highest average bAcc (64.16%), exceeding REVE by 0.29 pp, while REVE leads average wF1 and Cohen’s Kappa.Source: §4.1 · p. 7, Table 3 · p. 7

4.2 Architecture discovery across language models

  • 66.26% combined validation bAcc is Opus’s final score; Gemini finishes at 63.59%, DeepSeek at 63.10%, and Qwen gains 16.90 pp but finishes lowest (58.43%).Source: §4.2 · p. 7, Figure 3 · p. 8
  • Opus refines multiscale convolutions with channel gating and mean–standard-deviation pooling, while Gemini’s fresh design combines filter banks, dilated residual blocks, and attention pooling.Source: §4.2 · p. 7
  • The pool preserves prior incumbents through DeepSeek and Sonnet round-best declines (0.030 and 0.221 pp); all six searches reuse older parents, while Gemini and Qwen find fresh round-6 champions.Source: §4.2 · pp. 7–8, Table 4 · p. 8

4.3 Early performance forecasting and candidate selection

  • At ten epochs, PEEK cuts combined-score MAE from 2.20 to 1.36 pp across 36 paired rounds (38.1%), anticipating gains beyond observed scores.Source: §4.3 · p. 8, Figure 4 · p. 9
  • At ten epochs, PEEK lowers MAE in five of six searches but raises DeepSeek’s from 1.97 to 3.63 pp; by epoch 16, baseline aggregate MAE is lower (0.88 versus 1.18 pp).Source: §4.3 · p. 8, Figure 5 · p. 9
  • At ten epochs, keeping two candidates raises task-average winner retention from 56.5% to 80.6%; external forecasters reach 72.2% versus 75.0% for self-forecasting.Source: §4.3 · pp. 8–9, Figure 4 · p. 9, Table 5 · p. 9

4.4 Search cost and winner retention

  • Selecting two candidates retains a full-budget winner in 83.3% of rounds with 54.9% estimated task-epoch savings; selecting three reaches 91.7% retention and 44.9% savings.Source: §4.4 · p. 9, Table 6 · p. 10

5 Conclusion

AutoBCI combines multi-task architecture discovery with early performance forecasting. Its strongest discovered architecture leads REVE on average test bAcc, while PEEK reduces forecast error but retains fewer combined-score winners in its top-two selection.

5 Conclusion

  • 64.16% average test bAcc, 0.29 pp above REVE, is achieved by AutoBCI’s strongest discovered architecture.REVE leads on wF1 and Cohen’s Kappa.Source: §5 · p. 10
  • 38.1% lower forecast MAE after ten epochs is achieved by PEEK.Its top-two selection retains fewer combined-score winners than the best-observed-score baseline.Source: §5 · p. 10
  • Every one of the six LLM-guided searches improves its best validation score.AutoBCI combines PGAD architecture discovery across motor imagery, emotion recognition, and sleep staging with PEEK early forecasting.Source: §5 · p. 10

AI Use Disclosure

LLMs support architecture generation and performance prediction in the study, while AI assistants also aid literature review and text polishing. The authors retain responsibility for the manuscript, claims, and artifacts.

AI Use Disclosure

  • Six LLMs propose and refine executable EEG architectures from task specifications and validation feedback, and predict full-budget validation performance from early training evidence.Source: AI Use Disclosure · p. 10
  • Generated architectures undergo the implementation checks described in Supplementary A.6, and reported decoding results come from training and evaluation on the stated data partitions.Source: AI Use Disclosure · p. 10
  • AI assistants were also used for literature review and text polishing, while the authors take responsibility for the final manuscript, experimental claims, and artifacts.Source: AI Use Disclosure · p. 10

Ethics Statement

The study analyzes publicly available EEG recordings computationally, without recruiting participants or collecting new human data. The authors identify no additional ethical concerns arising from these analyses.

Ethics Statement

A.1.1 EEG Representation Learning and Architecture Design

EEG architecture design encodes assumptions about temporal and spatial structure, while foundation models study transferable representations through pretraining. AutoBCI selects architecture code using combined validation scores across three task families, with independent training and separate parameters for each.

A.1.1 EEG Representation Learning and Architecture Design

  • EEGNet uses depthwise and separable convolutions, while TSception and DeepSleepNet encode task-specific temporal, spatial, and sleep-stage structure.TSception captures multiple temporal scales and hemispheric asymmetry; DeepSleepNet combines convolutional feature extraction with recurrent modeling of sleep-stage dependencies.Source: §A.1.1 · p. 14
  • LaBraM, CBraMod, and LEAF study transferable EEG representations through neural tokens, spatial-temporal attention, and language-aligned task instructions, respectively.LaBraM uses neural-spectrum prediction and masked token prediction; CBraMod models spatial and temporal dependencies in heterogeneous recordings; LEAF aligns representations across tasks and label spaces.Source: §A.1.1 · p. 14
  • AutoBCI selects architecture code using combined validation scores across emotion recognition, motor imagery, and sleep staging, with independent training and separate parameters for each task family.Source: §A.1.1 · p. 14
  • PGAD uses executable model proposals and an accumulated pool ranked by an equal-weight aggregate of task scores.Task-specific learning curves and validation results for selected parents provide feedback for subsequent code-level refinements; CTNAS-EEG is a precedent with a multi-task-compatible search space and constrained procedure.Source: §A.1.1 · pp. 14–15

A.1.2 LLM-Guided Architecture Search and Early Performance Prediction

Language models and agentic methods have been used to propose and refine architectures, while early-performance prediction and resource allocation address the cost of evaluating candidates.

A.1.2 LLM-Guided Architecture Search and Early Performance Prediction

  • EvoPrompting, GENIUS, and NADER use language models or collaborating agents to propose, iteratively refine, or guide modifications to architectures (Chen et al., 2023; Zheng et al., 2023; Yang et al., 2025).PGAD combines parent refinements and fresh proposals with a shared selection objective across independently trained EEG task models.Source: §A.1.2 · p. 15
  • Partial learning curves, architecture features, hyperparameters, and validation trajectories can support performance prediction during architecture search (Baker et al., 2017; Ding et al., 2025).Hyperband uses successive halving to allocate training budgets, while BOHB combines this strategy with model-based configuration selection (Falkner et al., 2018).Source: §A.1.2 · p. 15

A.2 Evaluation details and supplementary comparisons

The supplementary evaluation specifies how forecasts are produced and compared, and accounts for screening costs, compute estimates, and token use.

A.2 Evaluation details and supplementary comparisons

  • Forecasts target each task’s maximum equal-dataset mean validation bAcc within 40 epochs, with task maxima averaged equally even when checkpoint epochs differ.Primary forecasts use each generating LLM’s isolated, tool-disabled session and include training protocol, parameter counts, losses, and validation prefixes; code-and-curve and curve-only conditions differ only in architecture code.Source: §A.2 · p. 15
  • All 36 additional cross-model forecast calls succeeded, using frozen ten-epoch code-and-curve prompts and Opus and Sonnet self-forecasts across 12 rounds.Forecasters did not receive generator identities, later validation epochs, or test results.Source: §A.2 · p. 15
  • 140 candidate-epochs per task versus 320 under full training yields a nominal 56.25% reduction for eight candidates, excluding forecasting overhead and failed attempts.The nominal schedule observes all eight candidates for ten epochs and continues two to epoch 40.Source: §A.2 · p. 15
  • The recorded 36-round cohort yields 54.9% savings at k = 2, versus 56.25% for an idealized eight-candidate round.These estimates evaluate recorded batches; effects on parent selection, later proposals, and final search performance remain unevaluated.Source: §A.2 · p. 15
  • GPU job-hour estimates use recorded 40-epoch task durations and assume constant average epoch time, but are not physical GPU occupancy or elapsed search time.Token accounting includes generation and accepted forecasting calls, with common generation budgets of 0.281M input and 0.501M output tokens and PEEK adding 1.167M input and 0.120M output tokens.Source: §A.2 · pp. 15–16

A.3 Round-by-round architecture details

This section documents the search, data, training, and baseline-evaluation protocols, the effects of training precision, and round-by-round architecture details. It also describes champion EEG designs and PGAD/PEEK prompts, including a numeric-value constraint in the PEEK prompt schema.

A.3 Round-by-round architecture details

  • Figure 6 tracks evaluated candidates, round medians, cumulative best bAcc, and the incumbent architecture after each of rounds R1–R6.The figure covers the six searches shown in Figure 3, with the final architecture marked in bold.Source: Figure 6 · p. 16, §A.3 · p. 16

A.4 Baseline training and evaluation

  • Baselines were trained independently on pooled task-family datasets, with equally weighted dataset validation bAcc selecting checkpoints and equal task weights in the combined test results.Conventional models were trained for 100 epochs and foundation models for 40; pretrained encoders and task heads were fine-tuned, with sleep-input adaptations varying by model.Source: §A.4 · p. 17

A.5 Data preparation and split construction

  • Candidates were trained from scratch as separate 40-epoch models for each task, with shared architecture code, separate weights and heads, and architecture selection based on equally weighted dataset and task validation scores.Test splits were reserved for final evaluation; emotion used predefined trial splits, MI held out test subjects, and sleep used subject-disjoint partitions.Source: §A.5 · p. 17, §A.5.2 · p. 19

A.5.3 Signal representation and label handling

  • Emotion and MI use 65-channel, 800-sample inputs, while sleep uses 8-channel, 3,000-sample inputs; each task has one output head spanning its union of catalog labels.Examples are standardized per channel along time, and validation and test predictions mask classes absent from each source dataset.Source: §A.5.3 · p. 20

A.6 Search implementation and execution details

  • Candidate proposals were generated in batches of eight, with later rounds receiving parent code and measured task-level validation evidence; only candidates completing all three tasks could enter selection.Static interface and import checks, numerical-output checks, and synthetic optimization steps ran before candidate training.Source: §A.6 · pp. 20–21

A.8 Training precision: computational cost and predictive performance

  • Full BF16 reduces peak allocated memory versus FP32 by 46.4% for EEGNet, 19.0% for DeepConvNet, and 40.8% for Opus, but does not accelerate training.FP32 has the lowest mean epoch time for all three models; full BF16 is close to FP32 for Opus.Source: Table 10 · p. 23, §A.8 · p. 23
  • Relative to FP32, full BF16 changes test bAcc by −1.68, −3.01, and +0.81 pp for EEGNet, DeepConvNet, and Opus; mixed precision changes it by −0.98, −0.95, and −0.43 pp.The comparison uses three seeds on one fixed BCI Competition IV-2a split.Source: Table 10 · p. 23, §A.8 · p. 23

A.9.1 Claude Sonnet 5

  • The Sonnet champion projects input channels into 48 features, applies two multiscale convolution blocks with kernel sizes 3, 7, 15, and 31, then uses squeeze-and-excitation gates, global average pooling, and linear classification.The same code is instantiated independently for motor imagery, emotion recognition, and sleep staging, with task-specific input channels, class counts, and learned weights.Source: §A.9 · pp. 23–24, §A.9.1 · p. 24

A.9.2 DeepSeek V4.1 Flash

  • The DeepSeek champion uses four depthwise temporal convolution branches with kernels 51, 25, 13, and 7, then projects to 48 channels and classifies normalized upper-triangle temporal covariance features.Source: §A.9.2 · pp. 24–25

A.9.3 Gemini 3.8 Flash

  • The Gemini champion combines four temporal convolution branches with channel mixing and downsampling, four squeeze-and-excitation residual blocks at dilations 1, 2, 4, and 8, and learned attention pooling.The pooled representation feeds a linear classifier.Source: §A.9.3 · pp. 25–26

A.9.4 Qwen3.8 Flash

  • The Qwen champion projects channels to 128 features, applies residual depthwise–pointwise convolution blocks at dilations 1 and 3, and classifies concatenated temporal means and standard deviations.Source: §A.9.4 · pp. 26–27

A.9.5 GPT-5.6 Sol

  • The GPT champion applies parallel temporal convolutions with kernels 11 and 47, a residual depthwise–pointwise block, and multiresolution average and maximum pooling to form a 1,920-dimensional vector.A 256-unit classification head follows the pooled representation.Source: §A.9.5 · p. 27

A.9.6 Claude Opus 5.5

  • The Claude Opus champion standardizes inputs, uses four multiscale convolution blocks with squeeze-and-excitation gates and intermediate pooling, then classifies concatenated temporal means and standard deviations.Its convolution branches use different kernel sizes across blocks, and the final representation passes through dropout before the linear classifier.Source: §A.9.6 · p. 28

A.10 Complete prompts and structured interfaces

  • PGAD uses the same instruction block for initial proposals and parent refinements, while round-specific allocation and supplied parent evidence change; PEEK is a retrospective three-task forecaster.Source: §A.10 · p. 28

A.10.1 PGAD: complete first-round prompt

  • The first-round PGAD prompt requests exactly eight fresh explorations, each with a testable EEG representation hypothesis, and imposes implementation, parameter-count, and training-protocol constraints.Refinements must change one component of a named parent; explorations have no parent, and duplicate model sources are disallowed.Source: §A.10.1 · pp. 29–30

A.10.2 PGAD: subsequent-round prompt

  • Later PGAD rounds request three single-component refinements per supplied parent and two fresh explorations, then rank architectures by equal-weight mean task best validation BACC after full-budget independent task training.The prompt supplies two selected parents with their source, scores, task curves, parameter counts, and per-dataset scores.Source: §A.10.2 · pp. 30–31

A.10.3 PEEK: complete forecasting prompt template

  • The PEEK prompt schema constrains a numeric value to the range 0–1.The passage specifies type "number", minimum 0, and maximum 1.Source: §A.10.3 · p. 34
Loading 2609.35456v1…