PaperlayerView exact revision on arXiv

Source-linked AI summary

NEUROTOKEN: Joint Source and Directional AAD with Envelope Decoding via Conditional Flow Matching

Ali Alavi, Donald S. Williamson

arXiv:2610.00397v1cs.LG

TL;DR

AAD from EEG could help hearing devices identify the speaker a listener attends to, but existing methods divide direction classification, source selection, and envelope reconstruction. NEUROTOKEN combines these tasks and uses conditional flow matching to score candidate envelopes; it improves per-segment source-AAD across three datasets and reduces across-subject variance.

  • Problem

    Existing AAD approaches separate direction classification from source identification, while single-Pearson envelope ranking is noisy at the 1–5 s windows relevant to devices.

    Source: §1 · pp. 1–2

  • Method

    NEUROTOKEN shares EEG features across direction, envelope-reconstruction, and conditional flow-matching heads, with ATTUNEFLOW scoring candidate envelopes by a likelihood ratio.

    Source: §1 · p. 2, §3.3 · p. 4

  • Results

    ATTUNEFLOW-Flow is 9 to 16 pp better than QUADTRACK-Multi per segment across three datasets, with 3–4× smaller standard deviation; trial-level fusion exceeds 93% on KU Leuven and NJU.

    Source: §4.4 · p. 8

  • Takeaways & Limitations

    The authors frame candidate-envelope ranking as a conditional likelihood-ratio test rather than reliance on any single statistic of one realization.

    Source: §5 · p. 9

    On unseen subjects, envelope-side rules plateau at 51–60% per-segment on cross-subject DTU and NJU.

    Source: J Limitations · p. 47

Abstract

from arXiv · show

Identifying which speaker a listener is attending to in a noisy room -- the cocktail-party problem -- is the missing ingredient for next-generation hearing aids and brain-computer interfaces: it tells the device whose voice to amplify. Auditory attention decoding (AAD) reads this answer from EEG, but the literature splits into disconnected pieces: directional-AAD classifies side but does not map side to stream; regression-based source-AAD ranks candidate streams by a single Pearson correlation that is intrinsically noisy at the 1-5 s windows real devices need; and envelope reconstruction has no native AAD rule. We argue the right object is not any single statistic but the conditional likelihood of the attended envelope given EEG, and we make this practical with NEUROTOKEN: a single network whose three heads share one EEG front-end, with a conditional flow-matching head (ATTUNEFLOW) that scores candidates by an integrated velocity-residual likelihood ratio. Two inference-time ensembles -- QUADTRACK (four complementary statistics) and ENV-FLOW (z-normalised QUADTRACK+ATTUNEFLOW) -- absorb per-statistic failure modes for free. On KU Leuven, DTU, and NJU at 5 s, ATTUNEFLOW lifts per-segment source-AAD by 9%-16% over the strongest non-generative baseline and shrinks across-subject variance by ~3x; trial-level fusion exceeds 93% on two of three datasets. In parallel reproductions we show that canonical 95-97% direction-AAD numbers collapse by 17%-45% under a strict trial-disjoint protocol, clarifying both the true ceiling and why a likelihood-based formulation is needed.

1 Introduction

AAD could guide hearing devices toward the attended speaker, but existing approaches separate short-window decisions, source identification, and envelope reconstruction. NEUROTOKEN combines these tasks and uses conditional likelihood scoring to address noisy short-window comparisons.

1 Introduction

  • At 1–5 s, prior AAD approaches either classify direction rather than source or rank reconstructed envelopes with a noisy single correlation.The reported standard error of Pearson ρ is 0.05–0.08, while the attended-minus-unattended r-gap is 0.01–0.02.Source: §1 · p. 2
  • Reproductions of five recent direction-AAD models under a strict trial-disjoint protocol show their canonical 95–97% results fall by 17–45%.The authors present these reproductions as an independent contribution to interpreting the benchmark.Source: §1 · p. 2
  • NEUROTOKEN shares an EEG front-end across direction-AAD, envelope reconstruction, and ATTUNEFLOW source-AAD, which scores candidates with a conditional likelihood ratio.The paper also introduces QUADTRACK and ENV-FLOW as inference-time ensembles over complementary scores.Source: §1 · p. 2
  • ATTUNEFLOW improves per-segment source-AAD by 9–16% across three datasets and reduces across-subject variance by about 3×.The paper attributes the gain to marginalizing the score over flow time and noise.Source: §1 · p. 2

2 Background and Related Work

Prior AAD work includes direction classifiers and envelope-reconstruction systems, each with distinct strengths and limits. NEUROTOKEN addresses their gaps with source scoring and a shared architecture, evaluated under a strict trial-disjoint protocol.

2.1 Discriminative direction-AAD

  • Direction-AAD models classify attended side from short EEG windows and report 95–97% at 1 s on KU Leuven, but side labels cannot identify streams when spatial mapping is unknown.The reproduced models lose 17–45 percentage points under a strict trial-disjoint protocol.Source: §2.1 · p. 3

2.2 Source-AAD and envelope reconstruction

  • Envelope-reconstruction systems rank candidate audio using Pearson correlation, whose 0.05–0.08 standard error at 1–5 s exceeds the 0.01–0.02 attended-minus-unattended r-gap.NEUROTOKEN replaces single-correlation ranking with a Monte-Carlo likelihood ratio under a learned conditional density and keeps the target as a low-rate envelope.Source: §2.2 · p. 3

3 NeuroToken: a unified network with the AttuneFlow score

NEUROTOKEN combines direction classification, envelope reconstruction, and conditional-flow source scoring in a staged network. Its inference rules compare candidate envelopes using complementary statistics and likelihood scores.

3.1 Notation

  • Trials contain 64-channel EEG sampled at 128 Hz, competing audio streams, attention and direction labels, and 1- or 5-second windows.The audio streams are converted to broadband gammatone envelopes at 32 Hz to match the network’s temporal-feature rate.Source: §3.1 · p. 3

3.2 Architecture

  • Two EEG front-ends support three heads: a closed-form CSP direction head, an envelope-reconstruction head, and ATTUNEFLOW, with envelope and flow paths sharing learned features.The narrowband front-end targets 0.5–9 Hz envelope tracking; the broadband path feeds the subject adapter and predicts a listener-specific lag.Source: Figure 1 · p. 4, §3.2 · p. 4

3.3 ATTUNEFLOW: conditional flow matching for envelope source-AAD

  • ATTUNEFLOW conditions on narrowband EEG features and learns a velocity field that transports Gaussian noise toward the attended envelope.At inference it avoids solving the ODE and scores candidate envelopes using a Monte-Carlo velocity-residual score.Source: §3.3 · pp. 4–5

3.4 Stage 1 losses (front-end & envelope regression)

  • Stage 1 trains spatial classification and envelope prediction with spatial, warm-up, margin, contrastive, spectral, and spatial–envelope consistency losses.The warm-up bootstraps envelope prediction; the consistency loss couples the spatial and envelope paths’ attended-side decisions.Source: §3.4 · pp. 5–6

3.5 Stage 2 losses (ATTUNEFLOW fine-tune)

  • Stage 2 freezes Stage-1 modules and trains only the ATTUNEFLOW velocity and projection modules.The two-stage parameter updates are summarized in Table 1.Source: Table 1 · p. 6, §3.5 · p. 6

3.6 Inference: ensembles

  • QUADTRACK sums Pearson, Spearman, shared-lag cross-correlation, and δ–θ coherence, while ATTUNEFLOW scores candidates with a conditional likelihood ratio.ENV-MULTI-FLOW z-normalizes and combines the two score gaps because their scales differ; its stated role is robustness to per-subject failures, not uniform improvement.Source: §3.6 · p. 7
  • ENV-TRIAL-FUSED pools per-segment decisions from three envelope-side rules using confidence-weighted majority voting; the reported alternative ATTUNEFLOW-EM-OR is an oracle upper bound.The paper recommends ENV-TRIAL-FUSED as the deployable trial-level rule.Source: §3.6 · p. 7

4 Experiments

Experiments evaluate NEUROTOKEN on three EEG benchmarks under short-window protocols and compare its scores with reproduced direction-AAD baselines. ATTUNEFLOW improves 5 s per-segment source-AAD over QUADTRACK while reducing across-subject variance, and trial-level results vary across datasets.

4.1 Datasets and protocols

  • The benchmarks are KU Leuven (16 subjects, ±90°), DTU (18, ±60°), and NJU (21, ±60°); recordings use 64-channel EEG, or 32 channels for NJU.The table reports intra-subject single-fold AAD as mean ± standard deviation, with dataset-specific subject counts by window.Source: §4.1 · p. 7, Table 2 · p. 8

4.2 Implementation

  • The 5 s evaluation uses deterministic 5-fold segment-random splits with 50% overlap, matching the ListenNet/DARNet protocol.Training uses two stages, including an ATTUNEFLOW fine-tune; one subject takes about 25–35 minutes on a single A100.Source: §4.1 · p. 8, §4.2 · p. 8

4.3 Strict-vs-paper reproduction of prior baselines

  • Under the strict protocol, reproduced direction-AAD baselines show architecture-vs-pipeline gaps from +16.6 pp for DenseNet-3D to +44.6 pp for LISTENNET.The authors identify the pipeline as the dominant explanatory variable; their closed-form CSP+BandPower spatial head still reaches 90–95% on KU Leuven.Source: §4.3 · p. 8

4.4 Main result: 5 s intra-subject single-fold

  • ATTUNEFLOW-Flow alone is +9 to +16 pp better than QUADTRACK-Multi per segment: KU Leuven rises 76.1→86.4, DTU 74.4→83.6, and NJU 70.1→86.0.Pearson alone is uninformative at 5 s, with ρ at 59–60% across datasets. Trial-level fusion reaches 99.1% on KU Leuven and 93.9% on NJU, versus 86.6% on DTU.Source: §4.4 · p. 8

4.5 Window scaling: where the flow head pays off

  • ATTUNEFLOW-Flow gains 30.5 pp from 1 s to 5 s, while at 1 s every score is within ±5 pp of chance and flow’s lift is 2 pp.The flow score gains about 3× more from increasing window length than single Pearson.Source: §4.5 · p. 9, Table 3 · p. 9

4.6 Per-subject variance: flow tightens the distribution

  • ATTUNEFLOW-Flow’s 5 s across-subject standard deviations are ±2.6/3.3/5.1 pp on KUL/DTU/NJU, versus QUADTRACK-Multi’s ±9.5/9.2/14.1 pp.The authors report 3–4× smaller variance and say QUADTRACK-Multi’s σ>10 pp can leave an unlucky subject at chance.Source: §4.6 · p. 9

4.7 Trial-level deployment numbers (cross-subject)

  • Cross-subject trial-level fusion pools about 50 s of evidence; KU Leuven reaches its reported spatial-transfer ceiling, DTU recovers 83%, and NJU reaches 74%.The authors attribute DTU’s narrower spatial separation and NJU’s 32-channel cap and Mandarin domain shift as relevant dataset conditions.Source: §4.7 · p. 9, Table 4 · p. 9

5 Discussion

The discussion frames candidate-envelope ranking as a likelihood-ratio test and relates the approach to cortical tracking and existing AAD paradigms. The loss appendix explains the spatial-head objective and reports what happens when it is removed.

5 Discussion

  • The discussion casts ranking candidate envelopes against EEG as a likelihood-ratio test under p(e | c), with the flow score marginalising learned density over flow time and noise.The authors connect this formulation to the reported lift and 3–4× variance shrinkage.Source: §5 · p. 9

A Extended related work

  • Low-frequency (1–8 Hz) cortical activity tracks attended-speech envelopes; MEG offers about 3× higher source-space SNR than scalp EEG, and the evaluated datasets are EEG.The discussion cites cortical-tracking work and notes that strong published reconstruction results are driven by MEG [11].Source: A Extended related work · p. 13
  • The related-work overview distinguishes AAD paradigms along discriminative-versus-reconstructive and linear-versus-nonlinear axes.It notes that align [11] uses a separate pretrained vocoder on predicted wav2vec representations rather than generating audio directly from the brain decoder.Source: A Extended related work · p. 13, Table 5 · p. 13
  • LaBraM [21], NeuroLM [20], and BIOT are described as EEG foundation models not yet adapted to AAD; contrastive approaches [7] and [2] use envelope-NCE as the AAD criterion.The authors describe fine-tuning foundation-model checkpoints as a natural follow-up and report that pure-NCE training overfits subject-specific envelope-amplitude patterns.Source: A Extended related work · p. 13
  • EEG reconstruction at intelligible speech quality has not yet been demonstrated, and NEUROTOKEN reconstructs envelopes rather than waveforms.Integrating a flow-matching codec decoder is left to future work.Source: A Extended related work · p. 13

B Loss-term breakdown

  • The loss-term appendix expands six Stage 1 terms and three Stage 2 terms with their motivations, computations, and removal costs.Quoted hyperparameters are production values unless an ablation explicitly varies them.Source: B Loss-term breakdown · p. 14

B.1 Lsp: spatial cross-entropy (w1 =2.0)

  • Lsp trains the spatial head on left-versus-right direction labels using smoothed cross-entropy, keeping the spatial path productive when the envelope path underfits.Its binary target uses smoothing ε=0.05, assigning 0.975 probability to the true class and 0.025 to the other class.Source: §B.1 · p. 14
  • Removing Lsp collapses spatial-head accuracy to chance and drops cross-subject SPATIAL-TRIAL accuracy on KU Leuven by about 8–10 pp, while leaving the envelope path unaffected within ±1 pp.The authors use w=2.0 because a larger end-to-end spatial-loss weight can steal shared-front-end gradient capacity and collapse the envelope path on DTU S1.Source: §B.1 · p. 14

B.2 Lwarm: envelope warm-up Pearson (w1 =3.0)

The appendix motivates five loss terms that balance envelope tracking, candidate discrimination, spatial–envelope agreement, and conditional-flow calibration. Their ablations show that removing key terms can reduce accuracy, destabilize training, or leave the flow without useful AAD discrimination.

B.2 Lwarm: envelope warm-up Pearson (w1 =3.0)

  • Removing the Pearson-margin term drops per-segment ENV-R by 5–10 pp on every dataset; removing warm-up Pearson prevents convergence on smaller-data subjects.The margin reduces correlation with the unattended candidate, while the warm-up term first teaches the envelope head to track the attended envelope.Source: §B.2 · pp. 14–15

B.5 Lspec: multi-resolution STFT magnitude (w1 =0.5)

  • Removing Lspec leaves average per-segment ENV-R unchanged but increases per-subject standard deviation by ∼1 pp.The term stabilizes long-tail subjects whose early envelope reconstructions converge to phase-inverted solutions.Source: §B.5 · p. 16

B.6 Lcons: spatial–envelope consistency (w1 =0.1)

  • Removing the spatial–envelope consistency term drops per-segment SPATIAL and ENV-R accuracy by ∼1 pp and trial-level fusion accuracy by ∼3 pp.The term couples the signed direction and envelope gaps so the two paths make coherent decisions.Source: §B.6 · p. 16

B.7 Lflow: ATTUNEFLOW flow matching (w2 =0.3)

  • Removing Lflow drops ATTUNEFLOW test-window accuracy 5–10 pp, despite apparently perfect training AAD scores, because the velocity field loses conditional calibration.The flow-matching term regularizes the velocity field toward a faithful conditional flow; heavier weighting recovers only ∼4 % AAD lift over the Stage 1 baseline.Source: §B.7 · p. 17

B.9 Laad: ATTUNEFLOW AAD margin (w2 =10.0, the largest weight in Stage 2)

  • 9–16 pp lift over ENV-MULTI is attributed to Laad, which directly optimizes the ATTUNEFLOW score gap between attended and unattended envelopes.Laad alone can produce a degenerate flow that ignores conditioning, so the full Stage 2 objective retains flow and embedding regularizers.Source: §B.9 · pp. 17–18

C Full per-subject intra-subject results

The appendix lists per-subject results by dataset and window length, then outlines the inference-scoring rules and their intended failure modes. The supplied passages identify table headings but do not provide their result values.

KU Leuven (window 1s)

  • The KU Leuven per-subject results are organized for 1 s windows across ENV-R, ENV-MULTI, ATTUNE-FLOW, fusion, and spatial rules.The supplied passage gives the column headings, not the result values.Source: KU Leuven (window 1s) · p. 18

KU Leuven (window 5s)

  • The KU Leuven per-subject results are also organized for 5 s windows across envelope, fusion, and spatial rules.The supplied passage gives the column headings, not the result values.Source: KU Leuven (window 5s) · p. 18

DTU (window 1s)

  • The DTU per-subject results are organized for 1 s windows across envelope, fusion, and spatial rules.The supplied passage gives the column headings, not the result values.Source: DTU (window 1s) · p. 19

DTU (window 5s)

  • The DTU per-subject results are also organized for 5 s windows across envelope, fusion, and spatial rules.The supplied passage gives the column headings, not the result values.Source: DTU (window 5s) · p. 19

NJU (window 1s)

  • The NJU per-subject results are organized for 1 s windows across envelope, fusion, and spatial rules.The supplied passage gives the column headings, not the result values.Source: NJU (window 1s) · p. 20

NJU (window 5s)

  • The NJU per-subject results are also organized for 5 s windows across envelope, fusion, and spatial rules.The supplied passage gives the column headings, not the result values.Source: NJU (window 5s) · p. 20

D Inference scoring rules: motivation, intuition, and decision logic

D.1 ENV-R: single Pearson correlation

This section compares envelope-matching, EEG-conditioned likelihood, spatial-classification, and trial-level fusion rules, whose strengths vary by subject and window, motivating complementary scores and pooling. It distinguishes the diagnostic, ground-truth-dependent ATTUNEFLOW-EM-OR upper bound from the deployment-relevant ENV-TRIAL-FUSED rule.

D.1 ENV-R: single Pearson correlation; D.2 QUADTRACK multi-measure (ENV-MULTI)

  • Pearson alone is often sampling-noise dominated per segment at 5 s, while QUADTRACK (ENV-MULTI) adds 10–16 pp over ENV-R by summing complementary measures.At 32 Hz, a 5 s window has 160 samples and Pearson standard error ≈0.08; QUADTRACK’s unweighted sum reaches 70–75% versus 58–67% for individual measures.Source: §D.1 · pp. 21–22, §D.2 · p. 23

D.3 ATTUNEFLOW: ATTUNEFLOW flow likelihood ratio

  • At 5 s on KU Leuven, ATTUNEFLOW outperforms ENV-MULTI by ∼9 pp and is the only score reported to gain 30 pp from 1 s to 5 s.ATTUNEFLOW scores each candidate under a learned conditional envelope distribution given EEG, using a velocity-residual likelihood ratio.Source: §D.3 · p. 23

D.4 ENV-MULTI-FLOW: z-normalised ensemble of ENV-MULTI and ATTUNEFLOW

  • ENV-MULTI-FLOW lifts the worse of ENV-MULTI and ATTUNEFLOW on every dataset, matches the better on KU Leuven, and has 2–4× lower per-subject standard deviation than ENV-MULTI.The z-normalised ensemble is positioned as a robustness rule, not a uniform mean-lift; on DTU and NJU, ATTUNEFLOW alone is slightly higher.Source: §D.4 · p. 24

D.5 SPATIAL: closed-form CSP + band-power + alpha-asymmetry classifier

  • SPATIAL reaches ∼70% mean accuracy on KU Leuven, but cross-subject accuracy on DTU and NJU drops to chance ∼50%.The classifier combines CSP, band-power, and alpha-asymmetry features; the passage attributes cross-subject transfer failure to subject-varying alpha-lateralisation polarity.Source: §D.5 · p. 25

D.6 SPATIAL-TRIAL: trial-level majority vote of SPATIAL; D.7 ENV-TRIAL: per-rule trial-level majority vote (envelope side)

  • Trial-level majority voting raises SPATIAL to ∼85% on KU Leuven but remains at chance on DTU and NJU; envelope-rule voting typically adds +15 to +30 pp on KU Leuven and +5 on DTU/NJU.Pooling exploits the trial-constant attention label, but cannot help when the underlying per-segment rule is at chance.Source: §D.6 · p. 25, §D.7 · p. 26

D.8 ENV-TRIAL-FUSED: confidence-weighted trial fusion across envelope rules

  • ENV-TRIAL-FUSED matches the better per-rule trial score on every subject and beats it by 1–3 pp when two or more rules are above chance.It weights each rule using its validation accuracy on the current subject, so near-chance rules contribute little.Source: §D.8 · p. 26

D.9 ATTUNEFLOW-EM-OR: trial-level OR ensemble of ENV-MULTI-FLOW and ATTUNEFLOW

  • Under the stated independent-error example, ATTUNEFLOW-EM-OR yields 91% segment accuracy and about 98.1±1.9% cross-subject KU Leuven trial accuracy after majority voting.The example assumes ENV-MULTI-FLOW and ATTUNEFLOW are each correct on 70% of segments and their errors are independent.Source: §D.9 · p. 27
  • 98.1/83.1/73.7% (KUL/DTU/NJU) are the OR-derived oracle operating points; on DTU and NJU, a deployable tie-break trails by a few percentage points.The oracle resolves disagreements using the ground-truth attended side, unavailable at deployment; the appropriate deployment number is ENV-TRIAL-FUSED confidence-weighted majority. On KU Leuven, the paths almost always agree, so the published 99.1% intra-subject and cross-subject KUL trial number are unaffected.Source: §D.9 · p. 27
  • On KU Leuven cross-subject, per-segment errors correlate ∼0.1, so OR captures disjoint failures and trial-level Bernoulli aggregation pushes accuracy near 100%.The passage estimates per-segment OR performance as 1−P(both wrong) ≈ 1−0.3 · 0.3 ≈91%; AND would discard the disjoint-failure signal.Source: §D.9 · p. 27
  • The oracle bound is informative for cross-subject KU Leuven and DTU, where individual rules hit the per-segment SNR ceiling but retain roughly disjoint errors; on NJU, both rules are at chance.Because even oracle OR cannot create above-chance segment evidence on cross-subject NJU, the passage identifies per-subject calibration—not a richer test-time ensemble—as the appropriate response.Source: §D.9 · p. 27

E QUADTRACK per-measure breakdown

QUADTRACK combines four envelope-similarity measures with complementary error modes, while ATTUNEFLOW has smaller across-subject spread; the section also reports an envelope–spatial trade-off and a persistent cross-subject segment ceiling.

E QUADTRACK per-measure breakdown

  • A learned BandPower spatial head reduced DTU S1 envelope-multi accuracy from 79% to 51%; the closed-form CSP+BandPower head avoids end-to-end gradient conflict.The learned spatial classifier’s gradients swamped shared envelope features.Source: E QUADTRACK per-measure breakdown · p. 28
  • The lag-shifted ρX is the strongest single component on every dataset; dropping coherence κ costs ∼3 pp on KU Leuven and ∼4 pp on DTU.Subject-specific neural-response lags shift the cross-correlation peak away from zero in 30–40% of subjects.Source: E QUADTRACK per-measure breakdown · p. 28
  • +9 pp QUADTRACK-Multi gain over single Pearson is largest when coherence is clear despite noisy amplitude or the optimal cortical-response lag varies.The four measures have largely independent error modes, supporting the unweighted sum.Source: E QUADTRACK per-measure breakdown · p. 28

E.1 Across-subject variance of ATTUNEFLOW-Flow vs. QUADTRACK-Multi

  • 2–4× smaller across-subject standard deviation is reported for ATTUNEFLOW-Flow than QUADTRACK-Multi on every dataset, with spread under ±6 pp even on NJU.Table 8 compares per-segment AAD accuracy at 5 s windows.Source: Table 8 · p. 29
  • ATTUNEFLOW-Flow averages its likelihood score over 16 Monte-Carlo draws, reducing per-subject variance while leaving residual variation tied to δ–θ tracking quality.The passage attributes QUADTRACK-Multi’s variance to whichever of its four statistics fails for a subject.Source: §E.1 · p. 29
  • ENV-MULTI-FLOW has ±4.8/3.5 pp spread on KUL/DTU and ±7.3 pp on NJU, so its spread is not always below the better parent.It is described as a robustness-against-per-subject-failure rule rather than a uniform variance reducer.Source: §E.1 · p. 29

F Cross-subject LOSO ablation study

  • Per-segment envelope-multi accuracy plateaued at ∼60% across eight interventions, although the best KU Leuven S1 trial-concatenated LOSO results reached 90–100%.The passage characterizes the per-segment ceiling as structural rather than a tuning artefact.Source: F Cross-subject LOSO ablation study · p. 29

F.1 Ablation protocol

The ablations found a stable near-60% per-segment envelope-AAD ceiling across two held-out KU Leuven subjects, while several attempted changes failed to break it. The protocol also distinguishes this envelope-path ceiling from subject-dependent spatial transfer.

F.1 Ablation protocol

  • The single-fold, single-seed ablations used a 15-subject training set and S1’s ∼1,800-segment, 5-s test set; the D2 reference combined CSP shrinkage 0.3 with envelope-margin weight 8.0.S1 was chosen for rapid iteration and as a representative LOSO test subject; the protocol held code, data, duration, and seed constant across intervention cells.Source: §F.1 · pp. 29–30

F.3 What did not break the ceiling

  • Across the tested interventions, KU Leuven LOSO S1 per-segment env-multi stayed between 55.9% and 59.5%, while trial-concatenated accuracy reached 95–100%.The ablation table reports a 3.6-pp spread in per-segment results.Source: Table 9 · p. 31

F.2 Why the per-segment ceiling is structural

  • On KU Leuven S1, predicted-envelope correlations of 0.162 versus 0.153 left a 0.008 gap and an estimated ∼60% ceiling for correlation-based per-segment AAD.The reconstruction tracks generic speech modulation that is correlated across competing streams, limiting the attended-specific signal.Source: §F.2 · p. 31

F.3 What did not break the ceiling

  • Replacing BatchNorm with GroupNorm reduced env-multi by 1.6 pp and trial-concat by 5 pp, consistent with BatchNorm retaining useful cross-subject amplitude calibration.The authors kept BatchNorm as the default.Source: §F.3 · p. 31
  • Relative-margin and hard-negative NCE losses improved training but lowered validation env-multi by 3.4 pp and env-concat by 20 pp, consistent with overfitting subject-specific amplitude patterns.The modified losses produced a 3× stronger gradient in the small-correlation regime.Source: §F.3 · p. 32
  • Neurophysiological FiLM added 1 pp to E1b but did not recover the D2 baseline, while the match-mismatch head reached 48.5% held-out accuracy and changed env-multi by −1.5 pp.The match-mismatch head reached ∼56% training accuracy, indicating a large train–test gap.Source: §F.3 · p. 32
  • Adding 10% of S1’s trials to training moved env-multi −0.7 pp; oversampling a 20% calibration set lowered it 3.6 pp and collapsed spatial accuracy to 50.1%.The oversampled calibration set comprised roughly two of S1’s eight trials.Source: §F.3 · p. 32

F.4 Generalisation to a different held-out subject (S16)

  • 59.6% on S16 versus 59.5% on S1 kept per-segment env-multi stable, while S16 spatial AAD reached 85.3% and trial spatial-majority 75.0%.The cross-subject envelope ceiling persisted despite S16’s substantially stronger spatial transfer.Source: Table 10 · p. 32, §F.4 · pp. 32–33

F.5 Short-window training ablation (1 s segments)

Training on 1-s segments did not learn attended–unattended separation, because the per-segment correlation noise overwhelmed plausible cross-subject EEG-to-envelope signal. Across the reported KU Leuven LOSO interventions, per-segment envelope-multi remained near 60%, while trial-concatenated accuracy sometimes reached 100%.

F.5 Short-window training ablation (1 s segments)

  • With 1-s training segments, the Pearson-r standard error was ≈0.18 at 32 samples, far above plausible cross-subject EEG-to-envelope correlation (r ≲0.02), so training did not learn attended–unattended separation.Five concatenated 1-s segments do not reproduce one 5-s segment’s statistics when supervision is per segment.Source: §F.5 · p. 33, Table 11 · p. 33

F.6 Summary

  • Across nine KU Leuven LOSO interventions at 5 s, per-segment envelope-multi stayed within 55.9–59.6%, while trial-concatenated accuracy reached 100% in two configurations.The authors conclude that per-segment LOSO AAD is near the correlation-based ceiling and favor trial-concatenation or spatial-majority metrics when ∼50 s of integration is acceptable.Source: §F.6 · p. 33

F.7 Cross-subject 3-holdout sweep on all three datasets

In the cross-subject three-holdout sweep, trial-level ATTUNEFLOW em-trial-OR outperformed per-segment transfer across KU Leuven, DTU, and NJU, though absolute performance varied by dataset. Spatial transfer was strong only on KU Leuven, while NJU remained near chance per segment.

F.7 Cross-subject 3-holdout sweep on all three datasets

  • The sweep trained the full NEUROTOKEN pipeline on remaining subjects and evaluated three held-out subjects jointly under a trial-disjoint, segment-overlap-capped protocol.It used six KU Leuven windows, six DTU windows, and seven NJU windows.Source: §F.7 · pp. 33–34
  • ATTUNEFLOW em-trial-OR reached 98.1±1.9% on KU Leuven, 83.1±4.1% on DTU, and 73.7±5.2% on NJU; per-segment spatial transfer was dataset-dependent.The sweep covered 19 held-out validation cells across the three datasets.Source: Table 12 · p. 34, §F.7 · p. 34

F.8 KU Leuven: per-window breakdown and discussion

  • On KU Leuven, spatial accuracy varied from 59.5% to 84.6% across windows, while envelope-multi and ATTUNEFLOW were more stable at ±1.9 and ±2.9 pp; em-trial-OR reached at least 95% in every window.Windows with canonical contralateral alpha suppression had stronger spatial transfer than windows containing subjects with ipsilateral suppression.Source: §F.8 · pp. 34–35, Table 13 · p. 34

F.9 DTU: per-window breakdown and discussion

  • On DTU, QUADTRACK-Multi reached 59.6±2.4% and ENV-MULTI-FLOW 59.2±1.8%, while spatial transfer was 50.0±1.8%; em-trial-OR ranged from 76.1% to 87.8% across windows.The authors attribute chance-level spatial transfer to DTU’s narrower ±60° separation; ATTUNEFLOW alone reached 54.0±1.3%.Source: §F.9 · p. 35, Table 14 · p. 35

F.10 NJU: per-window breakdown and discussion

  • On NJU, every per-segment metric averaged within ±2 pp of chance, but em-trial-OR averaged 73.7±5.2% and reached 81.2% in its best cell.The authors cite NJU’s 32-channel cap and Mandarin speech domain shift as structural challenges for cross-subject decoding.Source: §F.10 · pp. 35–36, Table 15 · p. 35

G Layer-by-layer module specification

The appendix specifies NEUROTOKEN’s modules and two-stage recipe: Stage 1 trains the shared AAD and envelope components, while Stage 2 adds ATTUNEFLOW’s flow and contrastive heads.

G Layer-by-layer module specification

  • Stage 1 trains the shared front-end, envelope and spatial heads, and adapter; Stage 2 freezes them and trains only ATTUNEFLOW’s flow and contrastive heads.The in-paper stages correspond to codebase stages 1 and 6; the other generative-pipeline stages are not required for the reported AAD numbers.Source: G Layer-by-layer module specification · p. 36
  • Separate EEG front-ends serve envelope tracking and direction decoding because their optimal spatial filters are spectrally and topographically distinct.The shared front-end uses multiscale temporal branches and adaptive pooling; the envelope-specific front-end filters 0.5–9 Hz and downsamples its dilated features to 32 Hz.Source: G Layer-by-layer module specification · pp. 36–37
  • The spatial head combines frozen, train-split CSP filters with band-power and alpha-asymmetry features, then classifies their concatenation with a small MLP.The authors report that CSP+BandPower outperformed the learned alternatives they tested on small intra-subject training sets.Source: G Layer-by-layer module specification · p. 39, Figure 8 · p. 39
  • The envelope head sums a dilated CNN and a linear mTRF branch, with near-zero initialization letting the linear branch dominate early training.The authors say this staging lets the linear TRF converge before the CNN learns nonlinear residual corrections.Source: G Layer-by-layer module specification · pp. 39–40
  • ATTUNEFLOW uses an eight-block conditional velocity network whose 780 ms receptive field spans envelope context, with flow loss constraining the AAD-margin objective.The flow head is zero-initialized to start as identity transport; the authors report that removing the flow loss permits a degenerate conditioning-independent solution.Source: G Layer-by-layer module specification · p. 41, Figure 10 · p. 41
  • The contrastive projector maps eight-bin pooled EEG and envelope inputs into separate, L2-normalized 128-dimensional embeddings for InfoNCE scoring.The authors report about 6 percentage points of benefit at eight bins, compared with 0 at one bin and 3–4 at four bins.Source: G Layer-by-layer module specification · pp. 41–42, Figure 11 · p. 42
  • The parameter-free audio target averages eight gammatone subbands into one broadband envelope; auxiliary band-power features are retained but omitted from the canonical CSP+BandPower spatial head.The auxiliary module is consumed by the optional transformer-style head; its three bands are theta, alpha, and beta.Source: G Layer-by-layer module specification · pp. 42–43, Figure 12 · p. 43, Figure 13 · p. 43

H Prior-work reproduction protocol and tables

Under trial-disjoint evaluation, all five reproduced direction-AAD baselines score lower than under their published pipelines; the leaky-minus-strict gaps range from 16.6 to 44.6 percentage points.

H Prior-work reproduction protocol and tables

I Alpha-lateralisation sign analysis per subject

The alpha-band asymmetry sign separates subjects following canonical contralateral suppression from those showing ipsilateral suppression.

I Alpha-lateralisation sign analysis per subject

  • About 63 % of subjects follow canonical contralateral alpha suppression, while the remaining 37 % show ipsilateral suppression and contribute to the observed bimodality.The sign is defined by whether left-attention asymmetry is more than 1 SD below the within-subject resting baseline comparison.Source: I Alpha-lateralisation sign analysis… · p. 47

J Limitations

The appendix bounds the evaluated setting and identifies calibration, trial-level evaluation, and online use as important constraints, alongside proposed hearing-aid, BCI, and methodological applications.

J Limitations

  • 51–60 %: every envelope-side rule plateaus at this per-segment range for unseen subjects, and the authors say they do not break that ceiling.Trial-level ATTUNEFLOW-EM-OR reaches 73.7 % on NJU, while the authors direct readers to treat trial-level rather than per-segment results as the deployable operating point.Source: J Limitations · p. 47
  • Evaluation covers only two-talker, fixed-azimuth stimuli and normal-hearing participants; the authors report no evidence for scaling to real-world multi-talker settings.They also note that online use is unevaluated: the 20-step flow ODE runs once per 5 s segment, not within a short causal decision window.Source: J Limitations · p. 48
  • The benchmark requires substantial compute: each subject takes 25–35 minutes on one A100, and the project used about 700 A100-hours.The authors characterize this as lab-scale but non-trivial, limiting groups without GPU access to inference-only reproductions.Source: J Limitations · p. 48
  • Intra-subject results assume per-subject calibration, and trial-level fusion assumes attention remains on the same speaker throughout a trial.The authors note real attention switches occur on roughly 1–10 s timescales; their fusion rules do not include a switch detector.Source: J Limitations · p. 48
  • The fusion rule’s confidence weights use validation accuracy, which the authors acknowledge inflates trial-fusion results relative to strict held-out evaluation.They mark each affected fusion-rule cell with a dagger in the released per-fold sidecars.Source: J Limitations · p. 48

K.1 Positive impacts

  • The authors identify neuro-steered hearing aids and cochlear implants, as well as assistive BCI, as potential applications of AAD.They report 98 % KU Leuven and 83 % DTU trial-level results as within the operating range manufacturers have publicly cited for clinical translation.Source: §K.1 · p. 49
  • The authors propose conditional flow-matching score ratios as a general recipe for cortical decoding when per-window signal-to-noise is too low for a fixed-statistic discriminator.They identify motor-imagery BCI and sleep-stage decoding as possible areas for transfer.Source: §K.1 · p. 49

K.2 Negative impacts and dual-use concerns

The discussion identifies privacy, involuntary monitoring, fairness, safety, and compute risks, while distinguishing envelope reconstruction from intelligible speech decoding. It recommends privacy-preserving processing, population-specific validation, safety overrides, and transparent metrics.

K.2 Negative impacts and dual-use concerns

  • EEG may reveal information beyond attended speakers, so transmitting raw EEG creates a cognitive-privacy attack surface distinct from audio-and-accelerometer wearables.The authors recommend on-device EEG processing and transmitting only the final binary attention decision.Source: §K.2 · p. 49
  • More wearable dry-EEG and ear-EEG could lower the cost of involuntary attention monitoring, including second-by-second logs of speakers or advertisements.The authors identify visible indicators, hardware recording-off switches, and regulated use as mitigations.Source: §K.2 · p. 49
  • The flow head reconstructs cortical envelopes, not phonemes or content; the authors say EEG bandwidth is insufficient for intelligible speech reconstruction.They caution that this envelope-only boundary does not establish that future non-invasive EEG cannot decode intelligible speech.Source: §K.2 · pp. 49–50
  • Performance is unknown for hearing-impaired, older, and neurologically diverse listeners, among other populations, because benchmarks skew toward young, normal-hearing university adults.The authors warn that deploying without per-population calibration risks worse performance for under-represented groups.Source: §K.2 · p. 50
  • A false AAD decision could suppress the attended source, and the reported 98% KU Leuven trial-level result does not translate to unconstrained acoustic scenes.The authors say those scene-specific failure modes remain uncharacterised and recommend audio-domain safety overrides.Source: §K.2 · p. 50
  • The project consumed roughly 700 A100-hours; the authors expect EEG foundation-model scaling work to require an order of magnitude more compute.They document the compute budget to support collective accounting of research costs.Source: §K.2 · p. 50

K.3 Mitigations we recommend

  • Recommended mitigations include on-device EEG processing, visible recording indicators, target-population clinical validation, audio-domain safety overrides, and open per-fold metrics.The authors state that variance shrinkage is necessary but not sufficient for clinical validation.Source: §K.3 · p. 50
Loading 2610.00397v1…