Source-linked AI summary
Machine learning for neural decoding
Joshua I. Glaser, Ari S. Benjamin, Raeed H. Chowdhury, Matthew G. Perich, Lee E. Miller, Konrad P. Kording
TL;DR
Neural decoding still relies heavily on traditional methods, motivating clearer guidance on modern machine-learning approaches. This tutorial provides methods, code, and comparisons showing that neural networks and ensembles outperform traditional decoders on typical spiking datasets.
Problem
Neural decoding needs clearer evidence and guidance on how modern machine-learning methods compare with traditional decoders across typical datasets.
Method
The paper presents a tutorial, code package, best practices, and held-out-data comparisons of neural networks, ensembles, and traditional decoding methods.
Results
Neural networks and ensembles achieved the best held-out R2 performance across motor, somatosensory, and hippocampal datasets, while Wiener and Kalman filters performed worst.
Takeaways & Limitations
Modern machine-learning methods can be advantageous for neural decoding, including when recordings contain limited neurons or limited training time.
Takeaways & Limitations
The comparisons used spiking data, so their relative performance may differ for other modalities with substantially different noise levels.
Abstract
from arXiv · showhide
Despite rapid advances in machine learning tools, the majority of neural decoding approaches still use traditional methods. Modern machine learning tools, which are versatile and easy to use, have the potential to significantly improve decoding performance. This tutorial describes how to effectively apply these algorithms for typical decoding problems. We provide descriptions, best practices, and code for applying common machine learning methods, including neural networks and gradient boosting. We also provide detailed comparisons of the performance of various methods at the task of decoding spiking activity in motor cortex, somatosensory cortex, and hippocampus. Modern methods, particularly neural networks and ensembles, significantly outperform traditional approaches, such as Wiener and Kalman filters. Improving the performance of neural decoding algorithms allows neuroscientists to better understand the information contained in a neural population and can help advance engineering applications such as brain machine interfaces.
Caution in interpreting machine learning models of decoding
Decoding performance reveals how much information neural activity contains about a variable, but it does not establish that the decoded variable is directly processed or purposefully represented by the brain area. Interpreting complex decoders also requires held-out evaluation and caution about overfitting, data limitations, hyperparameters, runtime, and modality-specific noise.
- Understanding how information in neural activity relates to external variables: High decoding accuracy does not show that transformations within an ML decoder match those in the brain.This caution applies even when the decoder resembles neural function, such as a neural network.
- Understanding what information is contained in neural activity: Decoding measures how much information a neural population has about X, not whether a brain area directly processes X or serves that purpose.The passage emphasizes careful scientific interpretation for both ML and non-ML models.
- Model evaluation and overfitting: Held-out test data and cross-validation are crucial because expressive decoders can overfit training-specific noise and idiosyncratic datapoint information.Simpler algorithms or regularization can reduce overfitting by limiting model complexity.
- Practical and methodological limitations: Interpretation of performance should account for training cost, hyperparameter sensitivity, optimization limits, and differences across data modalities and noise levels.For 30 minutes of data on CPUs, fitting took less than 1 second for a Wiener filter, less than 10 seconds for a feedforward neural network, and less than 8 minutes for an LSTM.
Extended Data
The extended data examines decoder performance across temporal bin sizes and Kalman filter variants, while documenting implementation choices and modeling assumptions for the tested methods.
- Decoder results with different bin sizes: Decoder performance was evaluated across varying bin sizes using Wiener and Kalman filters, feedforward neural networks, and LSTMs across three brain areas.The analyses used the same testing set and largest training set as Fig. 5.
- General information: The decoding demonstrations assume normally distributed output noise, with outputs represented as vectors or matrices depending on whether components are predicted jointly.The input matrix is denoted X and the predicted output is denoted Y.
- Feedforward Neural Net: The feedforward neural network used two fully connected hidden layers with ReLU activations and equal hidden-layer widths.All neural network implementations used the Adam optimization algorithm and Keras; the network jointly predicted two output units.
- Kalman filter variants: The Kalman filter variant with a transition-noise scaling parameter was used in the main text to weight neural evidence and kinematic transitions differently.The parameter scales the noise matrix associated with transitions in kinematic states, and its introduction made a big difference for the hippocampus dataset.
- Naïve Bayes: The Naïve Bayes decoder assumes neurons’ spike counts are conditionally independent given the output variables and uses a uniform prior.Including a prior based on the training-set output distribution did not improve validation performance.