PaperlayerView exact revision on arXiv

Source-linked AI summary

The Roles of Supervised Machine Learning in Systems Neuroscience

Joshua I. Glaser, Ari S. Benjamin, Roozbeh Farhoodi, Konrad P. Kording

arXiv:1805.08239v2q-bio.NCcs.LGstat.ML

TL;DR

Neuroscientists seek to identify informative variables and assess whether simple models capture neural data. The review describes feature-importance methods and ML benchmarks, reporting that neural networks often outperform simple models in predicting neural activity. It cautions that predictive accuracy alone does not establish model correctness.

Abstract

from arXiv · show

Over the last several years, the use of machine learning (ML) in neuroscience has been rapidly increasing. Here, we review ML's contributions, both realized and potential, across several areas of systems neuroscience. We describe four primary roles of ML within neuroscience: 1) creating solutions to engineering problems, 2) identifying predictive variables, 3) setting benchmarks for simple models of the brain, and 4) serving itself as a model for the brain. The breadth and ease of its applicability suggests that machine learning should be in the toolbox of most systems neuroscientists.

Role 1: Solving engineering problems

Supervised ML turns neuroscience problems into prediction tasks, learning relationships from labeled measurements when the goal is an accurate estimate rather than a detailed explanation. The review describes applications to neural activity, behavior, neuroanatomy, and practical constraints on successful use.

Role 1: Solving engineering problems

  • Engineering problems seek accurate estimates of a desired quantity from related measurements, without requiring a detailed account of their relationship.ML learns that relationship from labeled measurements and then predicts outputs for new measurements.Source: Role 1: Solving engineering problems · p. 5

Neural Activity / Function

  • ML applications include decoding neural signals for medical use, predicting neural activity from external variables, quantifying behavior, and reconstructing activity from imaging measurements.Examples include brain-computer interfaces, seizure prediction, video-based behavior tracking, imaging inverse problems, and inferring spike timing from calcium traces.Source: Neural Activity / Function · pp. 5–6

Neuroanatomy / Structure

  • For neuroanatomy, ML can automate image segmentation and labeling as imaging data grow, and can support disease classification from anatomical measurements.Applications include labeling white matter tracts and reconstructing neuronal connections and morphologies from electron microscopy.Source: Neuroanatomy / Structure · pp. 6–7

Caveats

  • Successful ML requires methods suited to the data; overfitting is especially concerning with small datasets and complex models, while computation can challenge real-time use.Training may require suitable hardware, and some brain-machine-interface predictions must be made within tens of milliseconds.Source: Caveats · pp. 7–8

Role 2: Identifying predictive variables

Supervised ML can identify predictive variables in neuroscience without assuming a simple form for their relationships, while feature-importance methods measure input relevance in different ways. The review describes applications to neural activity, disease, and neuroanatomy, alongside caveats about interpretation and scope.

Role 2: Identifying predictive variables

  • Feature-importance methods examine which inputs support predictive accuracy; leave-one-out and best-first measure different notions of importance.Leave-one-out measures a variable's information beyond the others, whereas best-first reflects its total learnable task information.Source: Role 2: Identifying predictive variables · pp. 8–9
  • Unlike simple coefficient-based models, ML can assess variable importance in unknown nonlinear relationships and use bootstrapping to estimate confidence intervals.The review also notes that selecting important features can support simpler models using fewer inputs.Source: Role 2: Identifying predictive variables · p. 9

Caveats

  • Feature-importance methods assess raw model inputs, not necessarily the scientific features of interest; finding relevant feature combinations is a separate problem.In vision, raw pixels may be selected rather than higher-level features; combinations may require inspecting the model or using unsupervised learning.Source: Role 2: Identifying predictive variables · p. 9, Caveats · p. 12

Neural Activity / Function

  • XGBoost and ensemble methods improved encoding over Generalized Linear Models across motor cortex, somatosensory cortex, and hippocampus, while modern ML methods increased decoding accuracy [Glaser et al. 2017].Encoding models can support feature-importance analysis, and more accurate behavioral models can help researchers investigate neural activity–behavior relationships.Source: Neural Activity / Function · pp. 9–11
  • In a mouse depression model, neural activity features predictive of pathological behavior informed a stimulation paradigm that restored normal behavior.The reviewed medical applications test predictive relationships without assuming their functional form.Source: Neural Activity / Function · p. 11

Neuroanatomy / Structure

  • ML classifiers and feature-importance analyses can identify neuroanatomical features predictive of disease and help distinguish neuron types using morphology.Reviewed examples include depression and Alzheimer’s disease classifiers and predictions of cell type from morphology.Source: Neuroanatomy / Structure · pp. 11–12

Caveats

  • Overfitting remains a concern, so method and regularization choices should maximize accuracy on held-out data.The review states that higher test-set accuracy yields greater statistical power for estimating variable predictiveness.Source: Caveats · p. 12
  • Predictive relationships do not establish causality and can depend on included variables; mutual information is interpretable but can be prohibitively time-consuming for high-dimensional data without assumptions.The review notes that these interpretive issues also apply to simple regression methods.Source: Caveats · p. 12

Role 3: Benchmarking simple models

The review proposes using ML benchmarks to test whether simple, hypothesis-driven models capture the predictable structure in data, while noting that predictive performance alone cannot establish model validity. It extends this approach to neural activity and neuroanatomy and discusses alternative benchmarks and practical caveats.

Role 3: Benchmarking simple models

  • Because ML primarily predicts rather than automatically proposing mechanistic hypotheses, it cannot replace simpler models for that purpose until interpretability methods progress.Biophysical models such as Hodgkin–Huxley specify how biological mechanisms produce spikes, whereas ML models generally do not automatically build such hypotheses.Source: Role 3: Benchmarking simple models · pp. 12–13
  • A hypothesis-driven model should at least match ML performance on the same task, though matching it does not guarantee that the model captures meaningful concepts.A model that performs much worse may be missing important principles; matching ML makes its posited concepts more likely to be meaningful, but does not guarantee it.Source: Role 3: Benchmarking simple models · p. 13
  • Comparing only with earlier simple models can miss predictable complexity; benchmarks on data subsets can identify which tasks or model components need revision.A model close to the ML benchmark on task A but not task B may need revision to better account for task B.Source: Role 3: Benchmarking simple models · p. 13

Neural Activity / Function

  • Benchmarking simple neural activity models against ML could reveal how much activity or behavior remains unexplained, but such comparisons are uncommon.The review cites V1 receptive fields and population vectors as examples of simple models for neural activity or behavior.Source: Neural Activity / Function · pp. 13–14
  • Neural networks often predict neural activity better than traditional simple models, and a performance gap can point to missing dynamics or nonlinearities.Examples include retinal ganglion cells, primate V4 and IT, and auditory cortex; the review suggests updating a static linear model when an RNN performs much better.Source: Neural Activity / Function · p. 14

Neuroanatomy / Structure

  • ML applied to raw anatomical data could provide upper performance bounds for simple models of neuronal morphology, making anatomical modeling more meaningful.Examples include models of branch diameters, branch length, and neuronal fractal dimensions.Source: Neuroanatomy / Structure · p. 14

Caveats

  • Human behavior can be a more direct benchmark for models of human performance, while repeated-trial noise estimates require identical iterations and may be complicated by repetition effects.Repeated stimuli can change neural responses, and delivering identical stimuli to a moving animal can be difficult.Source: Caveats · p. 15
  • Predictive performance is necessary but not sufficient for model validity; ML benchmarks themselves require attention to overfitting, model choice, and dataset size.A high-accuracy model may not represent the true system, and multiple models may achieve similar performance.Source: Caveats · pp. 14–15

Role 4: Serving as a model for the brain

The review presents supervised machine learning as a way to model brain response properties, while emphasizing that these similarities do not settle how biologically faithful neural networks are. It also outlines four roles for supervised ML in systems neuroscience and notes practical opportunities alongside interpretive and causal limits.

Role 4: Serving as a model for the brain

  • Trained networks show response-property parallels across visual and nonvisual systems, with reported similarities spanning more visual cortex than competing model classes.Examples include network activations predicting responses across visual areas and trained networks showing activity similar to auditory and motor cortex neurons.Source: Role 4: Serving as a model for the brain · p. 16
  • Although these models can predict activity well, they often do not reveal the neural computations performed; interpretability methods and more interpretable architectures are being developed.Examples include feature visualization, methods for understanding recurrent-network dynamics, and architectures designed to be easier to interpret.Source: Role 4: Serving as a model for the brain · pp. 16–17
  • Response similarities do not establish shared learning rules; the review urges attention to brain cost functions and learning rules, while limiting networks to high-level functional models.The authors note that supervised learning in brains is disputed, and that neural networks differ from brains in implementation and other biological details.Source: Role 4: Serving as a model for the brain · pp. 17–18, Caveats · p. 18

Caveats

  • The review identifies four roles for supervised ML in systems neuroscience: engineering, predictive-variable identification, benchmarking simple models, and modeling the brain.The authors expect ML's influence on neuroscience to continue growing.Source: Discussion · p. 20
  • High predictive accuracy does not establish causality or a causally correct model of brain activity, especially when neuroscience data contain unobserved variables.The authors distinguish the use of ML for modeling and identifying predictive variables from engineering and benchmarking applications.Source: Discussion · p. 20
  • The review focuses on supervised learning and excludes unsupervised learning and reinforcement learning, which also have neuroscience applications.The authors describe unsupervised methods for dimensionality reduction, clustering, generative models, and feature extraction, and reinforcement learning as a model of reward-based learning and decision making.Source: Discussion · p. 20
  • As datasets grow and become more complex, ML may help extract meaningful relationships; prediction-oriented experiments can record more variables and use ML to optimize stimuli.The authors contrast prediction-focused data collection with traditional variable interpretation and multiple-comparison corrections.Source: Discussion · pp. 20–21
  • As ML becomes easier to apply, the authors expect it to become a central neuroscience tool, while noting that neuroscience has long inspired ML development.They describe automated ML as reducing choices about methods, preprocessing, and hyperparameter optimization.Source: Discussion · p. 21
Loading 1805.08239v2…