Towards an integration of deep learning and neuroscience

1. Introduction

• Machine learning and neuroscience have diverged, but advances in complex cost functions and structured architectures create an opportunity to reconnect them.Machine learning has emphasized function optimization, while neuroscience has focused on detailed neural codes, dynamics, circuits, and mechanisms.Source: §1 · p. 3

1.1 Hypothesis 1 – The brain optimizes cost functions.

• The brain may optimize cost functions that shape neural properties, internal representations, and processes, requiring efficient credit assignment in multilayer and recurrent networks.Behavioral evidence includes near-optimal movement, minimized energy and bodily risk, and maximized financial and movement gains.Source: §1.1 · pp. 3–4

1.2 Hypothesis 2 – Cost functions are diverse across areas and change over development.

• Brain cost functions may be locally generated or imposed across areas, differ by computational role, and change over development to bootstrap increasingly complex knowledge.Examples include movement error, visual surprise, attention allocation, local input-model quality, and developmental progression from visual contrasts to faces.Source: §1.2 · p. 4

1.3 Hypothesis 3 – Specialized systems allow efficiently solving key computational problems.

• Pre-structured specialization may match brain areas to distinct computational problems, including memory, information routing, reinforcement learning, and discrete decision gating.Recurrent circuits may support short-term memory, switchable cell states may support persistent or transient activation, and thalamic and basal-ganglia circuits may route or gate information.Source: §1.3 · p. 4
• Development and reinforcement learning may bootstrap a diverse sequence of internally generated cost functions that anticipates future needs and enables precise behavior from limited supervised information.This bootstrapping can program developmental stages, support familial imprinting and later foraging, and organize memory access and actions beyond curriculum learning.Source: §1.3 · pp. 4–5

Core hypothesis

• The brain may optimize global functions in multilayer networks through credit assignment while specifying tunable cost functions matched to ethological needs.These cost functions can drive development similarly to modern machine-learning systems.Source: §2 · p. 5

Scope of the section

• The paper develops these hypotheses by examining local and multilayer optimization, cost functions that vary across brain areas and time, specialized computational structures, and experimental tests.It also considers implications for neuroscience and machine learning and the evolutionary perspective on this architecture.Source: §2 · p. 5

Optimization mechanisms

• Brain learning may combine genetic circuit specification, local optimization, and biologically realistic approximations of backpropagation to solve distinct computational problems.Proposed mechanisms include error-derivative backpropagation, spike-timing-dependent plasticity, dendritic computation, and local excitatory-inhibitory networks.Source: §2 · pp. 5, 7

Structured learning architecture

• Internally generated cost functions may provide error signals and heuristics that bootstrap increasingly complex learning across specialized brain systems.The proposed architecture combines genetically and learning-specified cost-function modules with trainable cortical networks and recurrent or other neuronal dynamics.Source: Figure 1 · p. 6

2.1 Local self-organization and optimization without multi-layer credit assignment

• Hebbian plasticity can locally self-organize ocular dominance, self-organizing maps, and orientation columns while optimizing objectives such as principal-component extraction and reconstruction error.These mechanisms adjust weights from presynaptic–postsynaptic correlations and can produce neuronal competition.Source: §2.1 · p. 7
• Randomly connected liquid- and echo-state machines generate random temporal filters whose tunable readout weights learn complex temporal patterns without full multilayer backpropagation.Learning only the readout layer substantially simplifies optimization.Source: §2.1 · p. 7
• Echo-state networks can be trained through supervised or reinforcement learning, while feedback-trained variants suppress chaotic spontaneous activity.Readout-only supervised training is equivalent to regression.Source: §2.1 · p. 7

2.2 Biological implementation of optimization

• Local self-organization alone is insufficient to explain the brain’s powerful learning performance, motivating biologically plausible approximations of gradient descent.The discussion places backpropagation in computational context before considering how neural circuits could approximate gradient descent.Source: §2.2 · p. 7

2.2.1 The need for efficient gradient descent in multi-layer networks

• Serial perturbation estimates coordinate descent by perturbing one weight at a time, but it is slow because it requires global coordination across synapses.Parallel weight perturbation reduces coordination but has high-variance gradient estimates and remains inefficient for large networks, whereas node perturbation has lower variance proportional to cell number rather than synapse number.Source: §2.2.1 · pp. 7–8
• Backpropagation rapidly computes layerwise sensitivities with no single-pattern gradient variance, enabling efficient training of deep and wide architectures.Biological perturbation methods use local activity and a global cost signal, potentially supplied by mechanisms such as diffusible neuromodulators, but remain slow; reinforcement learning may require such exploration when costs are not directly differentiable.Source: §2.2.1 · p. 8

2.2.2 Biologically plausible approximations of gradient descent

• Biologically plausible schemes use phasic feedback errors, contrastive Hebbian dynamics, STDP, or random feedback weights to approximate backpropagation.XCAL uses bidirectional settling and local modified Hebbian learning, while STDP can encode error derivatives in temporal firing-rate derivatives; feedback alignment performs nearly as well as backpropagation under normalization and sign-concordance conditions.Source: §2.2.2 · pp. 8–10
• Biological approximations still face realism and credit-assignment challenges, especially recurrent temporal dynamics, spiking communication, and the lack of a natural implementation of backpropagation through time.Memory systems and specialized architectures could spatialize or shorten temporal credit assignment, while recurrent-network proposals offer alternatives to unfolding networks across time.Source: §2.2.2 · pp. 10–11
• Existing evidence suggests biologically plausible spiking networks can efficiently optimize complex functions of temporal history, although the learning mechanisms for such functions remain incompletely understood.Candidate approaches include internal FORCE learning, online predictions of backward-through-time gradients, specialized memory structures, and mult timescale recurrent connections.Source: §2.2.2 · pp. 10–11

2.3.1 Exploiting biological neural mechanisms

• Biological neurons may support learning rules beyond backpropagation through dendritic computation, retrograde signaling, and neuromodulation that expand possible credit-assignment and optimization algorithms.Dendrites can implement nonlinear computation, propagate activity-dependent signals, and potentially store activations alongside error derivatives; neuromodulators can selectively gate plasticity across areas and times.Source: §2.3.1 · pp. 12–13

2.3.2 Learning in the cortical sheet

• Cortical learning models use laminar architecture, dendritic compartments, inhibition, recurrence, oscillations, and thalamic interactions to explain predictive learning beyond backpropagation, but detailed circuit maps remain incomplete.These mechanisms may compare top-down and bottom-up signals, control competition and feedback, and maintain temporal information for prediction.Source: §2.3.2 · p. 13

2.3.3 One-shot learning

• One-shot learning may arise from invariant stored templates, replay into long-term memory, or Bayesian learning of sequential programs, although specialized attention and memory can also support backpropagation-based generalization.Human-like performance in Bayesian sequential-program learning has not yet been reached by current neural-network models.Source: §2.3.3 · pp. 13–14

2.3.4 Active learning

• Active learning selects informative inputs to reduce uncertainty or improve prediction, potentially through reinforcement-learned sampling policies, though brain implementation and the role of explicit probabilistic representations remain unclear.Standard gradient descent lacks adaptive sampling, but stochastic gradient descent can learn to choose data that accelerates subsequent gradient-based learning.Source: §2.3.4 · pp. 14–15

• Supervised learning back-propagates full output-error vectors, whereas reinforcement learning uses scalar evaluations and trial-and-error updates, making reinforcement learning generally less efficient.Scalar evaluation signals do not directly provide a low-variance gradient for multilayer networks.Source: §2.4 · p. 15
• Feedback-based attention mechanisms such as AGREL and KickBack, together with diffusible reward, approximate full-gradient credit assignment in deep reinforcement-learning networks.For networks deeper than three layers, the proposed architecture uses parallel feedforward and feedback pathways; it remains less efficient than backpropagation but improves on naive perturbation methods.Source: §2.4 · p. 15
• Deep Q-learning generates temporally local target patterns from reward decomposition, enabling backpropagation-based learning and a biologically compatible bridge between reinforcement learning and efficient gradient descent.Experience replay, mini-batch training, and stored previous outputs or cloned networks support efficiency and stability.Source: §2.4 · pp. 15–16
• Reinforcement learning still struggles with long timescales and hierarchical action, whereas humans sustain extended adaptive behavior using specialized processes such as prospective memory.Longer-horizon discounting loosens formal optimality guarantees, and current machine-learning methods do not span comparable timescales and scopes.Source: §2.4 · p. 16
• The brain may use diverse scalar reward-like signals, specialized memories, replay, and hierarchical control to train local networks, but whether it uses deep Q-learning, AGREL, or another mechanism is open.The diversity of neuromodulatory signals makes independently mediated reward-like pathways plausible.Source: §2.4 · p. 16

3. The cost functions are diverse across brain areas and time

• The brain may use distinct cost functions for supervised, unsupervised, reinforcement-based, and hybrid learning rather than one end-to-end objective.These functions may be innate properties shaped by evolution and matched to particular computational problems.Source: §3 · pp. 16–17

3.1 How cost functions may be represented and applied

• Supervised costs can compare outputs with targets, clamp networks into target states, or use scalar reinforcement signals requiring deep-network credit assignment.Unsupervised costs may instead arise intrinsically from network dynamics or be explicitly encoded in prediction, reconstruction, sparsity, or continuity errors.Source: §3.1 · p. 17

3.2.1 Matching the statistics of the input data using generative models

• Generative unsupervised learning can make higher areas reproduce input statistics, producing probabilistic models through wake-sleep, autoencoding, message passing, or backpropagation.These models can support compositional scene generation and interact with supervised costs through lateral and top-down connections.Source: §3.2.1 · pp. 17–18

3.2.2 Cost functions that approximate properties of the world

• Unsupervised costs can encode world regularities such as sparsity, temporal continuity, predictable transformations, and cross-modal mutual prediction to accelerate and target learning.Prediction errors can serve as explicit vectorial teaching signals, while continuity can provide implicit labels for object identity and invariance.Source: §3.2.2 · pp. 18–19, §3.3 · p. 19

3.3 Cost functions for supervised learning

• Supervised learning could compare neural states after failed and successful actions, exploit temporally continuous implicit labels, or compress multi-step routines into one-step functions.These mechanisms allow learning without requiring every target to be externally specified in advance.Source: §3.3 · pp. 19–20

3.4 Repurposing reinforcement learning for diverse internal cost functions

• Reinforcement mechanisms may be repurposed beyond global reward to train local networks with internally generated error signals for specific features or computations.Such signals could be genetically specified and may operate in systems including striatum and primary visual cortex, although some invariant recognition may develop without reward.Source: §3.4 · p. 20

3.4.1 Cost functions for bootstrapping learning in the human environment

• Bootstrap cost functions use innate heuristic detectors and mutual supervision to reduce data requirements and guide learning toward biologically relevant features that ordinary unsupervised objectives may miss.Hand, face, motion, appearance, and contextual detectors can bootstrap one another, producing downstream detectors better than any single training heuristic; proposed implementations include local reinforcement signals or input biasing.Source: §3.4.1 · pp. 20–22

3.4.2 Cost functions for story generation and understanding

• The paper proposes extending bootstrap cost functions toward representations useful for constructing stories, whose episodes encode causal relations and agents’ implicit goals.This passage introduces story generation and understanding as a possible culmination of such specialized internal guidance.Source: §3.4.2 · p. 22
• Story representations and primitives may develop through imitation, hierarchical state and action chunking, saliency-directed memory storage and recall, and social-agency priors.These mechanisms could support learning to chain representations and update them through experience and communication.Source: §3.4.2 · p. 23

• Blank-slate optimization is insufficient for complex cognition, even with diverse genetically specified cost functions and local learning rules.Source: §4 · p. 23
• Pre-structured systems can make planning, sequential processing, memory access, and compositional or recursive computation efficient, robust, and less dependent on learning.Source: §4 · p. 23
• Specialized systems may store and route data and flexibly deliver targets, training data, reinforcement, attention, and modulatory signals to the right learning rules, locations, and times.Source: §4 · p. 23
• The brain contains conserved and specialized structures, while cortical microcircuits may also vary across regions through distinct cell types and pre-specified heterogeneity.Source: §4 · pp. 23–24
• The proposed mappings between computational functions and neuroanatomy remain preliminary, and the relative contributions of optimization, self-organization, genetics, development, and their interaction are unresolved.Source: §4 · p. 24

4.1 Structured forms of memory

• Multiple memory types support distinct computational needs: long-term content-addressable retrieval, rapidly rewritable working memory, and implicit memories learned through error-driven changes to network weights.Content-addressable memory can be retrieved across situations resembling the original storage context, whereas working memory supports rapid rewriting.Source: §4.1 · p. 24

4.1.1 Content addressable memories

• Content-addressable memories recognize familiar situations and complete stored patterns from partial or noisy information, supporting database-like retrieval and historical stimulus-response lookup.Hippocampal CA3 is proposed as an auto-associative memory, with dentate gyrus pattern separation enabling complete-memory retrieval from partial cues.Source: §4.1.1 · p. 24
• Adding content-addressable memory enables deep networks to solve problems previously beyond LSTM networks, despite LSTMs already learning long-term dependencies.These memories may also serve as components of broader computational functions.Source: §4.1.1 · p. 24

4.1.2 Working memory buffers

• Working memory is limited yet can maintain scalar variables, high-dimensional vectors, or vector sequences through functionally distinct subsystems and persistent recurrent neural activity.Models associate working memory with recurrent prefrontal networks and characterize multiple specialized subsystems.Source: §4.1.2 · pp. 24–25
• Differentiable addressable memories and gating permit weighted superpositions of memory registers or gates to be queried, but whether the brain uses these mechanisms remains unclear.Neuroscience attractor models traditionally allow access to only one memory at a time, making the biological status of these machine-learning mechanisms uncertain.Source: §4.1.2 · p. 24
• Working memory buffers enable short-term storage and, together with other mechanisms, generalize operations across whatever representations occupy the buffer, supporting human-like reasoning.Their computational role depends on retaining information while applying operations flexibly to stored contents.Source: §4.1.2 · p. 25

4.1.3 Storing state in association with saliency

• Selective storage and retrieval based on information properties or attached tags provides a computational primitive for learning more complex tasks within a limited time window.This mechanism links memory access to saliency or other task-relevant attributes.Source: §4.1.3 · p. 25

4.2 Structured routing systems

• Structured routing systems operate through different mechanisms and constraints to flexibly route information across multiple temporal, spatial, and control scales.Source: §4.2 · p. 25

4.2.1 Attention

• Attention can focus computation on selected objects, reducing learning sample complexity, improving memory access, and enabling programmatic control of multi-step perceptual tasks.Attention itself, and local responses to attentional signals, can be learned by optimizing cost functions (Mnih et al., 2014).Source: §4.2.1 · p. 25

4.2.2 Buffers

• Memory buffers could store distinct variables, allowing abstractions that generalize operations across interchangeable contents and thereby make computation more flexible.A central challenge is transferring learned representations between separately coded buffers; proposed solutions jointly train buffers before allowing them to operate independently (Hayworth, 2012; Bostrom, 1996).Source: §4.2.2 · p. 26, §4.2.3 · p. 26

4.2.3 Discrete gating of information flow between buffers

• Discrete gating could dynamically connect selected source and destination regions without crosstalk, routing exactly the information required for a task.ACT-R evaluates many symbolic production rules in parallel and conditionally routes information between buffers; basal ganglia, cortex, and thalamic pathways are proposed to control such gating (Anderson, 2007; Stocco et al., 2010; O’Reilly and Frank, 2006).Source: §4.2.3 · p. 27

4.3 Structured state representations to enable efficient algorithms

• Pre-structured neural architectures may enable efficient representations and transformations that would otherwise need to emerge through gradient descent.The section motivates specialized structures for computational problems such as spatial representation, control, planning, binding, and recursion.Source: §4.3 · pp. 27–28

4.3.1 Continuous predictive control

• The cerebellum may continuously update predictive models from motor errors, supporting rapid correction of changing body and environmental dynamics across motor and cognitive tasks.Cerebellar mechanisms include high-dimensional feedforward representations, learned timing, inverse models, and real-time state updating.Source: §4.3.1 · p. 28

4.3.2 Hierarchical control

• Hierarchical control decomposes difficult optimization into levels with distinct cost functions, timescales, representations, and training signals, while the mechanisms producing these decompositions remain unknown.Candidate mechanisms include action and goal chunking, hierarchical reinforcement learning, and specialized structures spanning spinal, cortical, striatal, hippocampal, and amygdala systems.Source: §4.3.2 · pp. 28–29

4.3.3 Spatial planning

• Hippocampal place-cell and grid-cell systems may solve spatial shortest-path problems efficiently through attractor or wave dynamics, with related mechanisms supporting abstract navigation among goals.Specialized path-finding structures could simplify prospective planning by mapping goal and sub-goal spaces.Source: §4.3.3 · pp. 29–30

4.3.4 Variable binding

• Variable binding links arbitrary variables to their instantiations, and may arise from biologically plausible binding, routing, or spatially grounded memory systems.Such mechanisms address how brains apply rules across novel sentence contents and other arbitrary representations.Source: §4.3.4 · p. 30

4.3.5 Hierarchical syntax

• Dynamic hierarchies are needed for arbitrary nested references and recursive syntax, motivating neural mechanisms analogous to pointers or differentiable push-down stacks.Specific fronto-temporal networks may represent and generate hierarchical structure, although the underlying circuit mechanisms remain poorly understood.Source: §4.3.5 · pp. 30–31

4.3.6 Mental programs and imagination

• Sequential programs and hippocampal simulation may support compositional learning, imagination, and rapid evaluation of action sequences for program induction and world modeling.Structured probabilistic programs enable one-shot learning of new geometric and visual concepts, while hippocampal-prefrontal interactions simulate possible trajectories and outcomes.Source: §4.3.6 · p. 31

Specialized structures

• Specialized structures may support routing, control, utility functions, feature detection, attention, and rapid multimodal binding, yet their functional decomposition remains preliminary.The passage highlights the thalamus, cerebellum, amygdala, retina or early visual areas, frontal eye fields, and claustrum, while noting that other structures remain understudied.Source: §4.4 · p. 31

Rapid learning

• The hippocampus may support rapid learning while the cortex learns more slowly, with basal ganglia and prefrontal cortex potentially contributing to sequential one-shot learning.This proposal is illustrated by sequential computations underlying Bayesian Program Learning.Source: §4.4 · p. 31

Conscious thought

• Conscious thought can involve hierarchical planning, suspended and resumed trains of thought, and introspective recognition, but its neural basis is unclear.The passage speculates that the hippocampus may be involved in these processes.Source: §4.4 · p. 31

Related frameworks

• Lake et al. (2016) propose generative world models with compositionality, causal and intuitive inductive biases, and probabilistic inference over structured models to support abstraction and transfer learning.The models may use graphs, trees, or programs and discrete reusable parts.Source: §4.5 · p. 32

Integration with optimization

• Structured neural networks could implement these mechanisms through modular recombination, copied and routed representations, and reinforcement-learned control flow, but the required architectural level remains unresolved.Open possibilities include cost functions and training procedures, specialized computational structures, or underlying neural primitives; Andreas et al. (2015, 2016).Source: §4.5 · p. 32

Neuroscientific implications

• Complex state-machine, message-passing, or Bayesian primitives may support structured inference in the brain, although probabilistic population coding can also emerge from backpropagation without being built in by hand.Mapping Lake et al. (2016) onto neuroscience may involve interacting cost functions, specialized systems, genetic control, and generic neural optimization; Bach and Herger (2015), George and Hawkins (2009), and Orhan and Ma (2016).Source: §4.5 · p. 32

5.1 Hypothesis 1– Existence of cost functions

• Endpoint comparisons, learning-dynamics measurements, perturbations, and imposed local objectives offer complementary tests of whether brain circuits optimize cost functions.These approaches compare measured representations with cost-function simulations, assess gradient-like synaptic vector fields, test recovery toward local optima after perturbation, and use brain-machine interfaces to impose user-defined learning problems.Source: §5.1 · pp. 32–33
• Optimization may occur simultaneously end-to-end, within brain areas or microcircuits, and inside individual neurons, so experiments must identify each trainable unit’s scale.Local perturbations could reveal the extent of induced learning, while large-scale maps can clarify how training signals, connectivity, and diverse cellular properties constrain optimization.Source: §5.1 · pp. 33–35

5.2 Hypothesis 2– Biological fine-structure of cost functions

• Differences in neural architecture should be interpreted through candidate cost functions and their biological correlates, including synaptic rules, neuromodulation, genetically guided wiring, and inter-area interactions.Inverse reinforcement learning may infer cost functions from plasticity, while molecularly identifiable cells, synapses, and developmental programs could reveal how specialized learning objectives are implemented and tuned.Source: §5.2 · p. 35

5.3 Hypothesis 3– Embedding within a pre-structured architecture

• Distinct circuits participating in a shared computation should receive different plasticity signals matched to their computational roles, such as feature detection, attention allocation, or memory.For a joint detection objective, each subsystem should learn from the specific failure—insufficient features, misallocated attention, or forgotten relevant items—that limits overall performance.Source: §5.3 · p. 36

6. Neuroscience inspired machine learning

• Because the brain exhibits intelligent behavior across broad problems using relatively little data, understanding its subsystems and layers may guide more powerful machine learning systems.The section reviews three hypotheses about brain optimization and architecture as possible routes toward data-efficient learning.Source: §6 · p. 36

6.1 Hypothesis 1– Existence of cost functions

• The brain may efficiently optimize diverse problems despite nonlinear, stochastic, spiking dynamics, motivating biologically informed learning rules, hardware, and architectures.Testing whether the brain performs multilayer credit assignment resembling full gradient descent could reveal alternatives to backpropagation or simpler, more powerful optimization methods.Source: §6.1 · pp. 36–37

6.2 Hypothesis 2– Biological fine-structure of cost functions

• Machine learning may increasingly use multiple task-specific cost functions to specialize modules, sequence learning, support transfer, and construct hierarchical behaviors through divide-and-conquer strategies.Visual representations could be shaped by diverse supervised or unsupervised objectives, then shared, retrained, or used to generate training signals for other circuits.Source: §6.2 · pp. 37–38

6.3 Hypothesis 3– Embedding within a pre-structured architecture

• The brain’s heterogeneous, modular circuitry suggests that specialized architectures are needed to solve diverse behavioral challenges without reprogramming, offering machine learning a blueprint for combining them.Understanding both the breadth of specialized structures and the architecture that integrates them may be especially useful because designing such combinations de novo is difficult.Source: §6.3 · p. 38

• Deep learning separates efficient distributed optimization via backpropagation from designing cost functions and training procedures for desired computations.Source: §7 · p. 38
• The brain may likewise use general-purpose cortical optimization mechanisms alongside area-specific cost functions, allowing improvements to each component to generalize across cortex or enhance particular areas.Source: §7 · p. 38
• Evolutionary separation of optimization and cost-function generation could accelerate better computations by combining common unsupervised mechanisms with area-specific supervised or reinforcement-based objectives, as in Rasmus and Berglund (2015).Source: §7 · p. 39
• One proposed implementation treats six-layered cortex as shared optimization machinery receiving different cost functions and training data across cortical areas, unlike a uniform unsupervised algorithm differing only in inputs.Source: §7 · p. 39
• The biological separation remains unresolved: relatively fixed wiring or learning rules could coexist with variable cost-function components, configurable microcircuits, or specialized neurons, as proposed by Marcus et al. (2014a,b).Open questions include when and how the division evolved and how it differs across animals.Source: §7 · p. 39

• Specific systems such as V1 could test whether ventral and dorsal pathways impose distinct objectives on shared circuitry or interact with genetically and locally pre-structured mechanisms.These questions concern spatial and spatio-temporal filtering, downstream pathway roles, and the balance between local optimization and pre-initialized computations.Source: §8 · p. 39
• The proposed framework replaces a single cost function and homogeneous architecture with many specialized subsystems whose internally generated objectives interact across development.It combines diverse optimization mechanisms, representations, and structures rather than seeking one unifying theory.Source: §8 · p. 40
• The framework redirects neuroscience from identifying neural codes toward testing how cost functions, fixed circuitry, structured architectures, and learning rules jointly produce neural responses.It raises questions about which computations are wired in, learned, or produced by mixtures of pre-structuring and optimization.Source: §8 · p. 40
• Closer dialogue between neuroscience and machine learning could reveal biologically evolved optimization strategies, interacting developmental objectives, and specialized structures that improve learning across timescales with little supervised data.Such insights may help machine learning address broad computational demands in structured environments and move toward general intelligence.Source: §8 · p. 40
• The resulting “society” of cost functions and trainable networks resembles internally bootstrapped specialized systems, with both architectures and objectives shaped by evolution like neural-network hyperparameters.This view favors heterogeneous organization over simple end-to-end training.Source: §8 · pp. 40–41