PaperlayerView exact revision on arXiv

Source-linked AI summary

Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks

Andrea Soltoggio, Kenneth O. Stanley, Sebastian Risi

arXiv:1703.10371v3cs.NEcs.AI

TL;DR

The paper addresses how adaptive learning systems can be designed beyond fixed, hand-engineered neural networks. It reviews EPANNs, which combine simulated evolution with plastic neural networks and organize their properties, methods, progress, and research directions. The review reports advances in evolving learning, understanding evolution-learning interactions, multi-signal networks, and evolutionary representations of learning, while identifying autonomous discovery as the field’s central opportunity.

  • Problem

    EPANN research lacked a unified account of its motivations, achievements, methods, and future challenges despite substantial prior work.

  • Method

    The paper reviews EPANN inspiration, properties, evolutionary aims, research advances, and future opportunities.

  • Results

    EPANN research has advanced evolutionary algorithms for learning, understanding of evolution-learning interactions, multi-signal networks, and evolutionary representations of learning.

  • Takeaways & Limitations

    Recent scientific and technical progress creates conditions for EPANNs to discover new principles and systems for general adaptation and intelligent systems.

Abstract

from arXiv · show

Biological plastic neural networks are systems of extraordinary computational capabilities shaped by evolution, development, and lifetime learning. The interplay of these elements leads to the emergence of adaptive behavior and intelligence. Inspired by such intricate natural phenomena, Evolved Plastic Artificial Neural Networks (EPANNs) use simulated evolution in-silico to breed plastic neural networks with a large variety of dynamics, architectures, and plasticity rules: these artificial systems are composed of inputs, outputs, and plastic components that change in response to experiences in an environment. These systems may autonomously discover novel adaptive algorithms, and lead to hypotheses on the emergence of biological adaptation. EPANNs have seen considerable progress over the last two decades. Current scientific and technological advances in artificial neural networks are now setting the conditions for radically new approaches and results. In particular, the limitations of hand-designed networks could be overcome by more flexible and innovative solutions. This paper brings together a variety of inspiring ideas that define the field of EPANNs. The main methods and results are reviewed. Finally, new opportunities and developments are presented.

I. INTRODUCTION

EPANNs extend neuroevolution toward plastic neural systems that discover learning through interaction with environments. This review unifies their biological inspiration, defining motivations, progress, challenges, and future opportunities.

  • I. INTRODUCTION: EPANNs combine evolution and lifelong plasticity so networks can discover learning dynamics while interacting with sensory-motor environments.Evolution determines parts of the design, while plastic components change in response to propagated signals and stimuli.
  • I. INTRODUCTION: EPANNs address autonomous learning discovery, appropriate neural abstractions, large search spaces, computational cost, and broad performance objectives.Their search spaces and research questions are broader than those of algorithms targeting narrow, well-defined problems.
  • I. INTRODUCTION: Recent advances in neural networks, computing hardware, evolutionary computation, neuroscience, and genetics create conditions for renewed EPANN progress.The paper presents these developments as foundations for more flexible and innovative approaches.
  • II. INSPIRATION: Biological intelligence is shaped by interacting evolution, development, and lifelong learning, motivating EPANNs to study comparable adaptive processes.These dimensions are also described as phylogenetic, ontogenetic, and epigenetic processes.
  • II. INSPIRATION: The review organizes EPANN inspiration around evolutionary processes, biological plasticity, artificial plasticity, and natural and artificial learning environments.The figure summarizes these topics without claiming a complete taxonomy.

A. Natural and artificial evolutionary processes

Evolutionary computation provides EPANNs with a mechanism for designing intelligent software with limited human intervention. Biological learning and plasticity supply additional models for connecting neural change to adaptive behavior.

  • A. Natural and artificial evolutionary processes: Evolutionary computation simulates natural evolutionary processes in software to search for intelligent computational systems.This perspective underlies evolutionary robotics, learning classifier systems, and neuroevolution.
  • A. Natural and artificial evolutionary processes: Changes in synaptic connectivity can support learned behavior, including associative conditioning in simple neural systems.The cited biological work links connectivity strength changes to behavioral learning.
  • A. Natural and artificial evolutionary processes: Structural plasticity creates new neural pathways, occurs primarily during development, and can continue into adulthood.Computational models have also represented axon growth regulated by neurotrophic factors.

C. Plasticity in artificial neural networks

Artificial neural plasticity offers many learning rules and architectures, but their effectiveness depends strongly on matching mechanisms to tasks. EPANNs use evolution to explore this design space and its adaptive consequences.

  • C. Plasticity in artificial neural networks: EPANNs can search across learning rules, network topologies, and combinations of mechanisms rather than relying solely on manual design.Evolution can use established rules as ingredients or discover new functions representing learning rules.
  • C. Plasticity in artificial neural networks: Artificial neural networks support supervised, unsupervised, Hebbian, generative, neuromodulated, and spiking plasticity mechanisms.The review presents these algorithms as potential building blocks for EPANNs.
  • C. Plasticity in artificial neural networks: Continuous plasticity can cause catastrophic forgetting when new information overwrites previously acquired skills.This problem is also called the plasticity-stability dilemma.
  • D. Lifelong learning environments: Different environments impose different features, dynamics, goals, and corresponding machine-learning problems.Temporal dynamics, rewards, known target behaviors, and environmental regularities can call for different computational structures or objectives.

III. PROPERTIES, AIMS, AND EVOLUTIONARY ALGORITHMS FOR EPANNS

The review defines EPANNs through evolution, plasticity, evolutionary discovery of learning, and generality. It then distinguishes autonomous design, component testing, and biological-learning hypotheses as central experimental aims.

  • A. EPANN properties: EPANNs are defined by evolutionary design, signal-dependent plasticity, discovery of learning dynamics, and independence from particular problems or mechanisms.Together, these properties support investigation of learning across problems with different natures, complexities, and time scales.
  • A. EPANN properties: EPANN properties enable evolutionary investigation of learning across a large variety of problems using only evolution and plasticity as core assumptions.The framework is intended to support creative designs rather than a single prescribed learning mechanism.
  • B. Aims: EPANN experiments delegate learning-system design to evolution, either bootstrapping learning from scratch or optimizing initialized capabilities.Other aims include recovering performance in unseen conditions and testing computational advantages of neural components.
  • B. Aims: A further aim is to derive hypotheses about how biological learning emerged, especially through experiments that bootstrap learning from scratch.This aim connects EPANN experiments to questions about evolutionary origins of learning.

C. Evolutionary algorithms for EPANNs

EPANN evolutionary algorithms must search dynamic, expanding spaces in which learning increases problem complexity and solution size is often unknown in advance. Key design strategies include indirect encodings, diversity preservation, and mutation or recombination that retain useful structure.

  • EPANN evolution searches dynamic spaces where learning further increases both evolutionary-search complexity and problem complexity.
  • Because solving-network size is unknown in advance, desirable EAs can increase genotype length so evolution may discover increasingly complex strategies and larger networks.
  • Genotype-to-phenotype mappings can improve scalability by compactly encoding larger networks and exploiting regularity, repetition, variation, and architectural motifs.
  • EAs must mutate and recombine EPANN genomes without destroying essential solution properties, although effective recombination of two EPANNs remains unresolved.

5) Genetic encoding of plasticity rules:

EPANNs encode plasticity rules and related network features within evolutionary searches that can span large spaces of architectures, rules, and learning signals. Prior studies show that evolution can optimize, select, and sometimes rediscover learning mechanisms across varied tasks.

  • 5) Genetic encoding of plasticity rules:: Plasticity rules may be encoded as simple parameters or as broader rule spaces and applied selectively across network components, thereby evolving learning architectures.
  • 8) Implementations:: EPANN implementations add evolutionary features to search extremely large spaces more creatively than traditional parameter and hyper-parameter searches.
  • 5) Genetic encoding of plasticity rules:: Local plasticity uses signals such as pre- and postsynaptic activity, weights, modulatory signals, or errors, with Hebbian plasticity formed by presynaptic and postsynaptic activity multiplied by a learning rate.
  • 5) Genetic encoding of plasticity rules:: Evolution can autonomously discover learning mechanisms: one study rediscovered the delta rule in 20% of runs, while another found better generalization after evolution on more tasks.
  • 5) Genetic encoding of plasticity rules:: Evolution selected specific rules from large sets for simple linear problems, while co-evolving multiple Hebbian-like rules improved navigation performance but made analysis more difficult.

B. Evolving learning architectures

Evolving learning architectures combines neural-architecture search with learning mechanisms, enabling EPANNs to exploit dynamics that become difficult to design manually.

  • Evolutionary architecture search can vary hidden units, layers, topologies, modules, connectivity, and associated learning mechanisms.
  • CALM evolved modular networks whose modules performed unsupervised learning and whose intermodule connectivity used Hebbian rules.
  • Evolved recurrent networks with linear output learners were more compact and learned better than randomly initialized echo state networks.
  • Learning mechanisms and neural architectures are strongly interdependent, while diverse dynamics can facilitate the evolution of learning.
  • EPANNs become more effective as manual network design becomes less practical because architectures and dynamics grow richer.

C. EPANNs in Evolutionary Robotics

Evolutionary robotics provided a testbed showing that plasticity can improve adaptation and evolution, while also exposing limits of reward-driven discovery and the need for exploratory stepping stones.

  • Plastic networks adapted better when transferring from simulation to real robots, although the precise principles producing this advantage were difficult to quantify.
  • Networks evolved faster when synaptic plasticity and neural architectures were evolved simultaneously in conditional phototaxis.
  • Plasticity generally helped evolution across varied robotics conditions, motivating focused study of learning discovery, neuromodulation, and indirectly encoded plasticity.
  • Non-stationary reward environments require learning because changing facts cannot be embedded permanently in the genotype.
  • In a variable-reward foraging task, evolution discovered learning through discrete fitness stepping stones and a sudden fitness increase.
  • Reward-based search can be deceptive because stepping stones toward learning may receive lower fitness than brittle non-learning solutions.
  • Novelty search and exploratory behavior provided alternative stepping stones that supported later discovery of reward-based learning.
  • Hypotheses about parental protection, late maturity, and sensitive learning periods extend EPANN research toward developmental conditions for learning.

E. Evolving neuromodulation

Evolved neuromodulation lets networks regulate plasticity through additional signals, and studies associate these dynamics with faster learning and better adaptation in changing environments.

  • Neuromodulatory signals can multiply plasticity, enhancing or reducing learning across different neurons or network regions.
  • In changing-reward T-maze tasks, networks with evolved modulatory neurons learned faster than controls without them.
  • Evolution can use modulatory topology to separate learning circuitry from input-output control, shortening decision pathways.
  • Evolved networks can balance fixed and plastic components through selective modulation of learning targets.
  • Neuromodulation enabled faster learning in dynamic concurrent foraging and improved adaptation to changing conditions in other tasks.
  • Evidence across studies suggests neuromodulation facilitates the evolution of learning and benefits systems using multiple signal types.

F. Evolving indirectly encoded plasticity

Indirect encodings let evolution map compact genetic descriptions into larger networks and distributed plasticity rules, expanding the structures and learning mechanisms it can search.

  • Indirect genotype-to-phenotype mappings expand compact evolutionary representations into fully developed networks and plasticity rules.
  • Adaptive HyperNEAT continually queries an evolved CPPN during an agent’s lifetime to generate connection-specific nonlinear learning rules.
  • Unlike standard HyperNEAT, adaptive HyperNEAT uses neuron locations, pre- and postsynaptic activity, and current weights to determine changes.
  • Indirect encodings producing more regular neural structures improved general EPANN learning abilities in operant-conditioning tasks.
  • Expanding the search space of plasticity rules produced more general solutions than using Hebbian learning alone.
  • Recent advances in AI, machine learning, neural networks, and computing resources create new opportunities for EPANN research.

A. Levels of abstraction and representations

EPANN research emphasizes that abstraction, representation, memory, and learning mechanisms shape what evolution can discover, while general learning remains difficult and costly to achieve.

  • A. Levels of abstraction and representations: Choosing neural abstraction levels trades simulation cost against the richness of dynamics available for evolving learning and intelligence.Low-level models are more expensive but potentially richer, whereas high-level models simulate faster but require selecting essential dynamics.
  • A. Levels of abstraction and representations: Representations of networks, internal models, learning mechanisms, and genetic encodings can affect the algorithms’ ability to evolve learning abilities.Prior work showed that different phenotypical representations can produce considerably different evolutionary results.
  • B. Evolving general learning: General learning aims to acquire associations across input, internal, and output patterns in spatial and temporal dimensions, including problems not encountered during evolution.Achieving it may require richer, stochastic environments and larger evolutionary searches to avoid finite behavior switches and deception.
  • C. Lifelong learning and memory: Evolutionary Neurodynamics proposes that replication and selection could occur within a neural system during learning, accelerating evolutionary dynamics to a lifetime timescale.This offers a distinct route for accelerating learning, alongside complementary learning systems and other memory mechanisms.
  • C. Lifelong learning and memory: EPANNs can explore incremental, social, few-shot, and memory-based learning, including modular systems and external-memory architectures.External memory can separate memory and control, as in Evolvable Neural Turing Machines that avoided catastrophic forgetting in continual learning.

F. EPANNs and deep learning

EPANNs extend evolutionary design into deep learning, hardware, and evaluation, but current deep-learning experiments mainly optimize supervised classification performance rather than discovering learning from scratch.

  • F. EPANNs and deep learning: EPANNs combine evolution with recurrent, convolutional, differentiable, and pathway-based deep-network designs to optimize controllers, representations, and multitask learning.Examples include evolved recurrent controllers with deep max-pooling preprocessing, evolved differentiable pattern-producing networks, and pathways trained through backpropagation.
  • F. EPANNs and deep learning: Current EPANN deep-learning experiments primarily optimize learning in supervised classification tasks, such as final classification accuracy.Future work is proposed to evolve learning from scratch, recover performance, or combine learning rules and dynamics in novel ways.
  • F. EPANNs and deep learning: Progress depends on parallel computation, neuromorphic hardware, standardized tools, and environments that test generalization, recovery, and learning across changing problems.The field can exploit GPUs, neuromorphic systems, and existing intelligent-system platforms even when they were not developed specifically for EPANNs.
  • F. EPANNs and deep learning: Memristive plasticity consistently enabled higher performance than constant-weighted connections in simulated robotic navigation under static and dynamic reward scenarios.GPU implementations also supported evolution strategies for plastic spiking neural networks.
  • F. EPANNs and deep learning: EPANN evaluation should include learning speed, task breadth, skill variety, robustness, generalization, recovery, behavior richness, resources, and deployed solution complexity.These metrics target lifelong-learning capabilities rather than performance on one particular task.

VI. CONCLUSION

EPANNs draw on interdisciplinary biological inspiration to study autonomous neural learning and have made substantial progress toward evolving learning mechanisms. New computational resources and neural technologies may enable more flexible systems for discovering adaptation and intelligence.

  • VI. CONCLUSION: EPANNs pursue long-term objectives for discovering neural learning, with implications for artificial intelligence and biological adaptation.Their motivation draws on broad, diverse, and interdisciplinary sources.
  • VI. CONCLUSION: Over two decades, EPANN research advanced evolutionary algorithms for learning, clarified evolution–learning dynamics, assessed multi-signal networks, and explored evolved representations of learning mechanisms.These results point to the potential for autonomous evolution and discovery of neural learning.
  • VI. CONCLUSION: Increased computational power and renewed neural-computation research create conditions for more flexible EPANN designs that may discover principles for general adaptation and intelligent systems.The paper identifies this combination as a potential step change for the field.
Loading 1703.10371v3…