PaperlayerView exact revision on arXiv

Source-linked AI summary

Decoding dynamic brain patterns from evoked responses: A tutorial on multivariate pattern analysis applied to time-series neuroimaging data

Tijl Grootswagers, Susan G. Wardle, Thomas A. Carlson

arXiv:1606.02840v2q-bio.NC

TL;DR

Time-series decoding has only recently been applied to Cognitive Neuroscience questions. This tutorial reviews analysis options and illustrates them with MEG data, where classifier performance rose above chance for almost the entire window from about 80 to 600 ms.

  • Problem

    Time-series decoding has only recently been applied to Cognitive Neuroscience questions, leaving a comparatively limited evidence base for its use in this area.

    Source: §1 · pp. 3–4

  • Method

    The paper provides a tutorial-style guide to time-series decoding analysis, illustrated with example MEG data.

    Source: §1 · p. 3

  • Results

    From about 80 to 600 ms after stimulus presentation, classifier performance rose significantly above chance for almost the entire time window, allowing MEG decoding.

    Source: §2.2 · pp. 13–14

  • Takeaways & Limitations

    The guide is intended to inform future time-series decoding studies from a Cognitive Neuroscience perspective.

    Source: §1 · p. 3

    An onset in the analysis may indicate possible contamination from double dipping.

    Source: §6.1 · p. 32

Abstract

from arXiv · show

Multivariate pattern analysis (MVPA) or brain decoding methods have become standard practice in analysing fMRI data. Although decoding methods have been extensively applied in Brain Computing Interfaces (BCI), these methods have only recently been applied to time-series neuroimaging data such as MEG and EEG to address experimental questions in Cognitive Neuroscience. In a tutorial-style review, we describe a broad set of options to inform future time-series decoding studies from a Cognitive Neuroscience perspective. Using example MEG data, we illustrate the effects that different options in the decoding analysis pipeline can have on experimental results where the aim is to 'decode' different perceptual stimuli or cognitive states over time from dynamic brain activation patterns. We show that decisions made at both preprocessing (e.g., dimensionality reduction, subsampling, trial averaging) and decoding (e.g., classifier selection, cross-validation design) stages of the analysis can significantly affect the results. In addition to standard decoding, we describe extensions to MVPA for time-varying neuroimaging data including representational similarity analysis, temporal generalisation, and the interpretation of classifier weight maps. Finally, we outline important caveats in the design and interpretation of time-series decoding experiments.

1 Introduction

This tutorial provides guidance for applying MVPA to time-series neuroimaging data, using example MEG data to explain decoding of evoked responses. It covers the analysis pipeline and how analytic choices may affect results, introduces extensions to decoding, and outlines caveats and limitations.

1 Introduction

  • The article addresses the lack of tutorial guidance for time-varying brain activity with a Cognitive Neuroscience-focused guide to time-series decoding.It notes that these methods had only recently been applied in Cognitive Neuroscience, despite their use in Brain Computing Interfaces.Source: §1 · pp. 3–4
  • The tutorial uses example MEG data to demonstrate time-series decoding, examine how analysis choices affect results, and outline interpretive caveats.Its aims include explaining differences between time-series and spatial neuroimaging and informing future studies without prescribing one pipeline.Source: §1 · p. 4
  • The tutorial focuses on evoked responses, with group-level statistical inference at single time points or small windows; its principles also generalize beyond MEG.It presents a broad demonstration rather than a prescribed pipeline and avoids formal mathematics and implementation details.Source: §1 · p. 4

1.1 Multivariate pattern analysis (MVPA) for MEG/EEG

  • MVPA considers relationships among multiple neuroimaging variables, and decoding predicts experimental conditions from data patterns rather than predicting data from a model.The article centers on decoding evoked responses and also covers representational similarity analysis.Source: §1.1 · p. 5
  • Successful classification of held-out brain patterns above chance supports the conclusion that the data contain information distinguishing experimental conditions.Repeating classification across MEG/EEG time points can probe the time-course of information processing.Source: §1.1 · pp. 6–7
  • Neuroscience decoding uses statistical inference about available information, whereas BCI work emphasizes prediction accuracy, robust real-time classification, and generalizability.The tutorial notes that accuracy differences between neuroscience studies are generally not treated as meaningful.Source: §1.1 · p. 8
  • Time-series decoding enables examination of evolving brain representations, with extensions including temporal generalization, representational similarity analysis, and classifier-weight projection.The tutorial also discusses combining decoding with source reconstruction or comparing interactions between sensor subsets.Source: §1.1 · pp. 9–10
  • Although relatively few time-series studies had used decoding, those studies had already yielded valuable insights and illustrated its potential for future applications.The cited approaches include temporal generalization, source reconstruction, and comparisons of sensor-subset interactions.Source: §1.1 · p. 9
  • MVPA complements univariate analysis: classifiers can detect information missed by averaged signals, increasing sensitivity to condition differences, including on single trials.The appropriate method depends on the experimental question; MVPA addresses information-based rather than activation-based questions.Source: §1.1 · pp. 9–10
  • The tutorial compares analysis choices across preprocessing and decoding stages, including PCA, subsampling, averaging, and decoding, and describes extensions and limitations.The article also covers example MEG data, temporal generalization, representational similarity analysis, and classifier-weight projection.Source: §1.1 · pp. 10–11

2 Description of experiment

The section introduces an animacy-decoding MEG experiment used to illustrate how analysis choices affect time-varying decoding, and reports above-chance decoding after stimulus onset under the default pipeline.

2 Description of experiment

  • Twenty healthy volunteers participated in a study whose reported analysis used clear-state stimuli from 48 object exemplars (24 animate, 24 inanimate), yielding 32 trials per exemplar, 768 per category, and 1,536 per participant.Source: §2 · pp. 11–12

2.1. Data Collection

  • MEG was continuously sampled at 1,000 Hz from 160 axial gradiometers, filtered online with a 0.03-Hz high-pass filter, and epoched from −100 to 600 ms relative to stimulus onset.The 100 ms of pre-trial signal served as a sanity check for decoding accuracy.Source: §2.1 · p. 12

2.2. Analysis Summary

  • Because analysis parameters may interact, effects may vary across datasets, and performance comparisons require equal noise, the authors frame results as illustrative rather than prescriptive.They vary one parameter at a time relative to fixed parameters.Source: §2.2 · p. 12
  • The default pipeline retained 99% of variance with PCA, used leave-one-exemplar-out cross-validation, and reported animacy decoding accuracy over time.Higher accuracy indicated better prediction; chance was assessed at each time point using a Wilcoxon signed-rank test with false-discovery-rate correction.Source: §2.2 · p. 13
  • Before onset, accuracy was at chance (50%); from approximately 80 ms after stimulus onset, it rose significantly above chance for almost the entire window through 600 ms.This indicates that MEG activation patterns contained information about whether a stimulus was animate or inanimate.Source: §2.2 · pp. 13–14

3. Preprocessing

This section examines preprocessing strategies for noisy, high-dimensional neuroimaging data, including dimensionality reduction and trial averaging, and how their effects on decoding depend on the method and data. It considers the trade-off between improving signal-to-noise and retaining trials, including implications for decoding performance and temporal signatures.

3.1 Data transformation and dimensionality reduction

  • In the 160-channel example, an average of 48.16 components (SD=7.05, range 26–79) accounted for 99%; for this data set and classifier, PCA performed much better than raw channels.The differences were classifier dependent.Source: §3.1 · p. 15

3.2 Improving signal to noise

  • Subsampling modestly affected decoding while reducing computation, whereas sliding windows improved performance only marginally despite significantly increasing computation.Sliding windows give the classifier access to all time points in a window, while subsampling provides their average and preserves the number of features per time point.Source: §3.2 · p. 17
  • Both temporal-collapsing methods affect decoding-onset and peak estimates; use the sliding window’s last time bin for onset and low-pass filter before resampling to avoid aliasing.Low-pass filtering can itself produce significant decoding when no signal exists in the original data.Source: §3.2 · p. 17
  • Without artefact rejection, example decoding remained above chance, but condition-correlated artefacts can become discriminatory cues and prevent attributing decoding to the experimental condition.The authors note that the importance of artefact correction can vary across data sets.Source: §3.2 · p. 18
  • Averaging trials within an exemplar improves decoding performance and makes temporal signatures more pronounced, while the first decoding onset remains similar across averaging levels.Source: §3.2 · p. 18
  • In the example data, averaging 4 trials gives the greatest performance increase; more averaging yields smaller gains, while fewer trials per exemplar typically increase within-subject classifier-performance variance.Source: §3.2 · pp. 18–19
  • The optimal number of trials to average differs across datasets; Isik et al., (2014) are given as an averaging example, and when too few trials are available, training trials could be sampled with replacement (bootstrapped).Source: §3.2 · p. 19
  • Trial averaging does not affect model testing such as RSA, because relative decoding performance is scaled similarly between exemplars or time points.Source: §3.2 · p. 19

4. Decoding

This section explains how to evaluate time-resolved decoding of stimulus classes in MEG, including how classifier and cross-validation choices affect results. It covers measuring classifier performance and testing above-chance decoding while accounting for class imbalance and multiple comparisons across time, including temporal-clustering methods.

4. Decoding

  • The example preprocessing epochs MEG from -100 to 600ms, downsamples to 200Hz, and averages groups of four trials into eight pseudo-trials per exemplar.Source: §4 · p. 19
  • The workflow classifies animate versus inanimate MEG patterns at each time point, treating above-chance cross-validated performance as evidence of class-specific information.Decoding time courses can be assessed by the peak performance or earliest significant time point, and compared across conditions.Source: §4 · p. 19

4.1 Classifiers

  • LDA, GNB, and SVM had the best overall performance; GNB and LDA train faster than SVM, while correlation classifiers performed less well on these MEG data.The comparison used built-in Matlab implementations with default parameters.Source: §4.1 · pp. 20–21
  • Classifier choice and preprocessing interact: omitting PCA strongly affected GNB but had smaller effects on LDA and SVM, limiting universal recommendations.The authors recommend choosing the cross-validation scheme to fit the experimental design; for categories with many exemplars, leave-one-exemplar-out validation avoids sharing exemplar trials between training and test sets.Source: §4.1 · p. 21, §4.2 · p. 22

4.2 Cross-validation

  • Without cross-validation, decoding appeared above chance before stimulus onset because test data were also used for training, violating independence.Above-chance pre-stimulus performance is a warning that preprocessing or cross-validation may contain an error.Source: §4.2 · pp. 22–23
  • Ten-fold and leave-one-trial-out cross-validation yielded similar results, while k-fold versus leave-one-exemplar-out differences were largest early and diminished later.The early difference was consistent with early visual-feature processing; comparing these splits can indicate how much performance depends on exemplar properties.Source: §4.2 · p. 23

4.3 Evaluation of classifier performance and group-level statistical testing

  • Group-level statistical evaluation of decoding remains complex, with no consensus on the optimal approach.Source: §4.3 · p. 23
  • With unbalanced classes, accuracy can be inflated by predicting the majority class, so D-prime or balanced accuracy is preferable because it is unaffected by class imbalance.Balanced accuracy averages the accuracies for each class.Source: §4.3 · pp. 23–24
  • Above-chance performance can be tested with Wilcoxon signed-rank tests that make minimal distributional assumptions (Wilcoxon, 1945), Student's t-tests, or slower permutation tests using a shuffled-data null distribution.Permutation tests are especially useful when assumptions about the null distribution cannot be made, such as with biased classifiers or unbalanced data.Source: §4.3 · p. 24
  • Because tests are conducted across time points, multiple-comparison correction is needed; FDR is straightforward but does not account for relations between time points.The example analysis used FDR adjustment.Source: §4.3 · p. 24
  • Cluster-based correction tests clusters of time points for above-chance decoding and can increase sensitivity to smaller but more sustained effects.Source: §4.3 · pp. 24–25

5. Additional analyses

This section presents additional analyses for time-series decoding, including temporal generalization, representational similarity analysis, and methods for interpreting classifier weights as activation patterns. It describes how these analyses address representational change, structure, and information sources, while noting that activation-pattern reliability depends on decoding performance.

5.1 The temporal generalization method

  • Temporal generalization trains a classifier at one time point and tests it at others: successful cross-decoding suggests similar activation-pattern geometry, while failure suggests it has changed.The method can test cognitive models that predict how generalizable representations will be.Source: §5.1 · pp. 25–26
  • Temporal cross-decoding generalized well for neighbouring time points and between 150–200 and 300–500 ms, indicating that MEG activation patterns were similar across those periods.Significant points occurred off the diagonal.Source: §5.1 · p. 26

5.2 Representational Similarity Analysis (RSA)

  • RSA indexes pairwise stimulus dissimilarity by decodability, forming representational dissimilarity matrices (RDMs) that can be computed over time and compared with model RDMs.The empirical-to-model RDM correlation indicates how well a model’s predicted representational structure is reflected in brain activation patterns.Source: §5.2 · p. 27
  • The Animacy model fit the MEG data better than the Natural model, both fit better than the Silhouette model later, and Silhouette fit best early, supporting animacy as an organising principle.The authors interpret the early Silhouette fit as expected because it reflects early visual features.Source: §5.2 · p. 28
  • A current RSA limitation is that valid statistical comparisons between candidate models are difficult; a proposed alternative evaluates model performance against a noise ceiling.The noise ceiling estimates the highest possible performance given the data’s noise and can be evaluated over time for MEG models.Source: §5.2 · pp. 28–29

5.3 Weight projection

  • Weight projection examines which voxels or sensors drive classifier performance by projecting the feature weights assigned during classification.The supplied passage introduces weight assignment but does not provide further projection steps or results.Source: §5.3 · p. 29
  • Although raw weights may seem to index how much sensors contain class-specific information, higher weights do not directly imply more information than lower weights, and non-zero weights do not imply class-specific information.Source: §5.3 · p. 29
  • Reconstructed activation patterns require caution when decoding performance is low, because suboptimal weights produce unreliable patterns.Haufe et al. (2014) describe the dependence of pattern reliability on weight quality.Source: §5.3 · p. 30
  • For classifiers that use feature covariance, activation patterns are computed as A = cov(X)*w; the method cannot be directly applied to multi-time-point classification.The example uses LDA, and the resulting patterns can be projected onto sensor locations.Source: §5.3 · p. 30
  • Reconstructed activation patterns localized information around occipital sensors at 100 ms and temporal sensors at 300 ms, a pattern less evident in raw weight topographies.The authors note that this shift accords with the visual processing hierarchy.Source: §5.3 · p. 30

6. General discussion

Time-series decoding can reveal temporal dynamics of brain information processing, but its results require careful interpretation. The discussion highlights pitfalls involving data leakage, signal strength, filtering, experimental confounds, and the source of decodable information.

6. General discussion

  • Time-series decoding investigates temporal brain dynamics; the tutorial illustrates analysis choices and extensions including temporal generalisation, RSA, and weight projection.It also emphasizes that preprocessing and decoding choices can affect results.Source: §6 · p. 31

6.1 Common pitfalls

  • Classifiers must not access class-specific test-set information, which inflates performance; above-chance prestimulus accuracy can flag possible double dipping.The authors use the no-cross-validation analysis in Figure 10 to illustrate this confound.Source: §6.1 · pp. 31–32
  • Even with identical true decoding onsets, stronger simulated signals produced earlier significant decoding than weaker signals, so onset differences do not establish differences in information availability.The authors caution that significant-decoding onset depends on signal strength.Source: §6.1 · p. 32
  • A 30Hz low-pass filter made the simulated signal significant 40ms before its 50ms true onset; higher cut-offs such as 200Hz substantially reduced this distortion.The authors advise against timing interpretations with low-cut-off filters.Source: §6.1 · pp. 32–33
  • Because classifiers exploit any class-correlated signal, response-related brain activity or differences such as attention can confound decoding of the intended manipulation.The example experiment switched response buttons across blocks to avoid confounding button presses with stimulus category.Source: §6.1 · pp. 33–34
  • Even in carefully controlled studies, successful decoding shows that conditions differ in decodable information but usually does not identify the information’s source.The source of decodable signals remains a central challenge in applying MVPA to understand brain information processing.Source: §6.1 · p. 34
Loading 1606.02840v2…