PaperlayerView exact revision on arXiv

Source-linked AI summary

Analyzing biological and artificial neural networks: challenges with opportunities for synergy?

David G. T. Barrett, Ari S. Morcos, Jakob H. Macke

arXiv:1810.13373v1q-bio.NCcs.AIcs.CVcs.LGstat.ML

TL;DR

Neuroscience and machine learning share the challenge of understanding how complex neural networks represent information and compute. This review surveys analysis approaches across both fields, including ways to compare neural representations. It highlights opportunities for cross-field analysis while noting that biological systems offer less experimental access than DNNs.

  • Problem

    Understanding computations in multi-layered neural networks is a shared conceptual challenge for neuroscience and machine learning.

    Source: §6 · p. 10

  • Method

    The review surveys statistical analysis tools from both disciplines for studying neural activity and comparing representations across networks.

    Source: §1 · p. 3, §5 · p. 8

  • Results

    The reviewed findings link DNN generalization with greater ablation robustness, neural manifolds with constraints on learning, and biological visual areas with different DNN layers.

    Source: §3 · p. 6, §4 · p. 7, §5 · p. 10

  • Takeaways & Limitations

    The fields can redeploy DNN analysis methods to study biological networks and neuroscience methods to analyze DNNs.

    Source: §6 · p. 10

    Biological neural networks lack the complete activity, connectivity, perturbation, and learning-history access available for DNNs, and measurements are typically sparse, noisy, limited, and non-stationary.

    Source: §6 · pp. 10–11

Abstract

from arXiv · show

Deep neural networks (DNNs) transform stimuli across multiple processing stages to produce representations that can be used to solve complex tasks, such as object recognition in images. However, a full understanding of how they achieve this remains elusive. The complexity of biological neural networks substantially exceeds the complexity of DNNs, making it even more challenging to understand the representations that they learn. Thus, both machine learning and computational neuroscience are faced with a shared challenge: how can we analyze their representations in order to understand how they solve complex tasks? We review how data-analysis concepts and techniques developed by computational neuroscientists can be useful for analyzing representations in DNNs, and in turn, how recently developed techniques for analysis of DNNs can be useful for understanding representations in biological neural networks. We explore opportunities for synergy between the two fields, such as the use of DNNs as in-silico model systems for neuroscience, and how this synergy can lead to new hypotheses about the operating principles of biological neural networks.

1 Introduction

Neuroscience and machine learning both struggle to explain how multi-stage networks represent stimuli and perform complex computations. This review considers analysis tools from both fields as a basis for studying neural representations.

1 Introduction

  • Modern recording and perturbation technologies enable investigation of activity across large neural populations, but interpreting the resulting data remains difficult.Electrophysiology and imaging can record hundreds of neurons simultaneously, while optogenetic techniques allow targeted perturbations.Source: §1 · pp. 1–2
  • Despite complete access to DNN connectivity and activations, understanding how and why they work remains challenging, including how they generalize and develop invariant representations.Their millions of parameters contribute to the challenge.Source: §1 · p. 2
  • Both fields face the challenge of explaining how interconnected units transform stimulus representations across stages to support complex computations such as object recognition.The review also asks which tools can characterize representations in high-dimensional spaces.Source: §1 · p. 2
  • The review examines statistical tools from neuroscience and machine learning for analyzing activity measurements of single neurons and neural populations.Neuroscience and cognitive science influenced DNN development, while convolutional networks drew inspiration from properties of the ventral visual stream.Source: §1 · pp. 2–3

2 Receptive fields: What stimulus-features do single neurons represent?

Receptive field analysis characterizes the stimuli that drive individual neurons, and has informed accounts of increasingly complex and invariant visual representations. Similar analyses in convolutional networks reveal a progression from edge-like responses toward object-related responses.

2 Receptive fields: What stimulus-features do single neurons represent?

  • A receptive field is the stimulus-space region that drives a neuron's firing, often characterized as its preferred, maximally activating stimulus.Receptive field analysis is a canonical method for studying single-neuron activity (Sherrington, 1906, Victor, 2005).Source: §2 · p. 3
  • Along the ventral visual stream, receptive fields are described as becoming larger, more complex, and more invariant, culminating in concept cells tuned to individual objects.Simple cells in visual cortex have spatially localized, edge-like receptive fields.Source: §2 · p. 4
  • Activation maximization synthesizes images that maximally activate CNN units, with earlier units responding to edges and later units to textures and class-specific objects.Intermediate receptive fields may not have a semantically meaningful description.Source: §2 · p. 4

3 Ablations: How relevant are single neurons to the overall computation?

Ablation tests how individual units contribute to network output, while exposing a mismatch between neuronal selectivity and computational importance. In DNN image classification, generalizable networks are more robust to ablation than networks that memorize training data.

3 Ablations: How relevant are single neurons to the overall computation?

  • Ablation analysis silences neurons and measures the effect on network output, but single-neuron ablation in biological networks is technically constrained.Optogenetics and targeted two-photon stimulation may change that constraint; theoretical neuroscience has used ablations in models of intact and damaged networks.Source: §3 · pp. 4, 6
  • Generalizable image-classification networks are more robust to ablation than memorizing networks, while class selectivity is only weakly related to a unit's impact on overall accuracy.Most neurons have little effect on accuracy; using maximum class accuracy drop instead reveals a weak correlation of approximately -0.22.Source: Figure 2 · p. 5, §3 · p. 6
  • Pruning research identifies summed absolute weight values as one of the strongest signals of feature-map importance, while the factors underlying unit importance remain an open question.A better understanding of those factors may be useful for understanding biological neural networks.Source: §3 · p. 6

4 Dimensionality reduction: How can we characterize distributed representations?

Dimensionality reduction can characterize low-dimensional structure in neural populations whose information and dynamics are distributed across many neurons. Evidence from brain-machine interface learning suggests that neural manifolds can constrain which activity patterns animals learn.

4 Dimensionality reduction: How can we characterize distributed representations?

  • Dimensionality reduction seeks low-dimensional structure in population activity, where neural dynamics and information about stimuli and behavior can be distributed across many neurons.Low-dimensional manifolds have also been reported for motor-cortex dynamics.Source: §4 · pp. 6–7
  • Animals learned new brain-machine interface decoders when required activity patterns stayed within the underlying neural manifold, but not otherwise.This finding suggests that low-dimensional manifolds reflect functional structure rather than only statistical correlations; a small number of measurements may suffice to decode low-dimensional distributed dynamics.Source: §4 · p. 7
  • DNN representations show substantial redundancy, but the relationship between DNN performance and stimulus-representation dimensionality remains relatively little understood.Reported examples include predicting most parameters from about 5% of them and removing more than 85% with minimal performance loss; theoretical work is developing frameworks relating manifold geometry to network capacity.Source: §4 · p. 7

5 Cross-correlation: How can we compare representations across networks?

Cross-network comparison methods can track how representations change across layers and time, and compare biological and artificial networks. These analyses reveal both representational correspondences and differences across networks.

5 Cross-correlation: How can we compare representations across networks?

  • Comparing representations across populations is useful for studying transformations across layers or time, and may enable direct comparisons between biological and artificial networks.Figure 3 illustrates SVCCA comparisons across training snapshots, RSA comparisons of stimulus pairs, and linear regression aligning biological and DNN activation vectors.Source: §5 · p. 8, Figure 3 · p. 9
  • CCA-based comparisons found that lower DNN layers stabilize earlier than higher layers, while ResNet and ConvNet representations are similar early but differ late.The early stabilization finding motivated a proposed training approach that sequentially freezes lower layers; networks trained with different random seeds also had units spanning overlapping subspaces despite differing individual neurons.Source: §5 · p. 8
  • CCA identifies directions capturing correlations between datasets, including datasets with different numbers of neurons, and can compare representations across layers or network snapshots.Representational similarity analysis instead compares networks by the pairwise stimulus similarities or dissimilarities encoded in each network or layer.Source: §5 · p. 8
  • Direct comparison of DNN and biological representations creates the possibility of using artificial networks as model systems for biological neural networks.Yamins et al. used linear regression to identify correspondences between biological activity measurements and DNN activations.Source: §5 · pp. 8, 10
  • V4 activity was best reconstructed from intermediate DNN layers and IT activity from top layers, consistent with similarities between DNN and ventral-stream visual processing.The regression-based comparison uses biological and DNN activation vectors; unlike receptive-field analysis, it provides an objective statistical comparison.Source: Figure 3 · p. 9, §5 · p. 10

6 Challenges and opportunities

Neuroscience and machine learning share the challenge of understanding computations in multilayered networks, but differences in access and mechanisms constrain direct transfer. The section describes opportunities for methods and ideas to move between fields.

6 Challenges and opportunities

  • DNN methods can exploit complete activity, connectivity, perturbation, learning-history, and gradient information that is unavailable to biological neuroscience.This unequal experimental access is a direct challenge to transferring some analysis methods.Source: §6 · p. 10
  • Despite conceptual and empirical similarities, biological and artificial networks differ mechanistically, and whether they use similar representations and algorithms remains open.The section notes backpropagation, non-spiking representations, and different constraints and demands as examples of differences.Source: §6 · p. 10
  • Understanding biological computation may be even harder than understanding DNNs, which remain poorly understood despite full access and simpler neural machinery.Better experimental tools are necessary but not sufficient; the section also calls for data-analysis methods and quantitative theories.Source: §6 · pp. 11–12
  • DNN analysis methods can inform biological research, while neuroscience-adapted methods and deep-learning tools may expand neural data analysis.Examples include activity-based dimensionality reduction, black-box variants of gradient-dependent methods, and Bayesian inference approaches.Source: §6 · p. 11
  • DNNs can serve as in-silico systems for rapidly developing, testing, and applying analysis techniques to complex sensory tasks.Neuroscience benchmarks may use neuron-response predictions to compare, select, or rule out models.Source: §6 · pp. 11–12
  • Public, large-scale standardized neuroscience datasets may support benchmarks that compare algorithms in a field where progress can be difficult to quantify.The section gives prediction of response properties along the visual hierarchy as an example benchmark task.Source: §6 · pp. 11–12

Related papers

Loading 1810.13373v1…