PaperlayerView exact revision on arXiv

Source-linked AI summary

AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence

Jeff Clune

arXiv:1905.10985v2cs.AI

TL;DR

The paper addresses the challenge of producing general AI when manually discovering and combining its components is difficult. It proposes AI-generating algorithms that learn architectures, learning algorithms, and effective environments, and argues that this alternative merits greater investment while carrying major computational, safety, and ethical uncertainties.

  • Problem

    Manually discovering and combining the many components required for general AI creates a difficult scientific and engineering challenge.

  • Method

    The paper proposes AI-generating algorithms that automatically learn how to produce general AI through three pillars: architectures, learning algorithms, and effective learning environments.

  • Results

    The paper concludes that AI-GAs and the manual path are both scientifically worthwhile, while judging with high uncertainty that AI-GAs may produce general AI first.

  • Takeaways & Limitations

    AI-GA creation should be treated as an independent computer-science grand challenge and receive more research investment alongside the manual path.

  • Takeaways & Limitations

    AI-GAs require substantial computation and raise unresolved safety and ethical questions, including possible suffering or undesirable behavior in virtual agents.

Abstract

from arXiv · show

Perhaps the most ambitious scientific quest in human history is the creation of general artificial intelligence, which roughly means AI that is as smart or smarter than humans. The dominant approach in the machine learning community is to attempt to discover each of the pieces required for intelligence, with the implicit assumption that some future group will complete the Herculean task of figuring out how to combine all of those pieces into a complex thinking machine. I call this the "manual AI approach". This paper describes another exciting path that ultimately may be more successful at producing general AI. It is based on the clear trend in machine learning that hand-designed solutions eventually are replaced by more effective, learned solutions. The idea is to create an AI-generating algorithm (AI-GA), which automatically learns how to produce general AI. Three Pillars are essential for the approach: (1) meta-learning architectures, (2) meta-learning the learning algorithms themselves, and (3) generating effective learning environments. I argue that either approach could produce general AI first, and both are scientifically worthwhile irrespective of which is the fastest path. Because both are promising, yet the ML community is currently committed to the manual approach, I argue that our community should increase its research investment in the AI-GA approach. To encourage such research, I describe promising work in each of the Three Pillars. I also discuss AI-GA-specific safety and ethical considerations. Because it it may be the fastest path to general AI and because it is inherently scientifically interesting to understand the conditions in which a simple algorithm can produce general AI (as happened on Earth where Darwinian evolution produced human intelligence), I argue that the pursuit of AI-GAs should be considered a new grand challenge of computer science research.

1 Two approaches to producing general AI: the manual approach vs. AI-generating algorithms

The manual AI approach builds and combines intelligence components by hand, whereas AI-GAs aim to learn the machinery needed to produce general AI automatically. The paper presents AI-GAs as a distinct scientific path worthy of increased investment, while acknowledging uncertainty about which path will succeed first.

  • 1.1 The manual AI approach: The manual approach discovers separate intelligence building blocks and leaves a future group to combine them into a complex thinking machine.Combining many modules is difficult because of combinatorial possibilities, nonlinear interactions, and scientific, engineering, and organizational constraints.
  • 1.2 A different approach: AI-generating algorithms (AI-GAs): AI-GAs instead learn the components and processes required to produce general AI, potentially avoiding manual discovery and integration of every building block.This proposal extends the recurring shift from hand-designed solutions toward learned pipelines and learned learning machinery.
  • 1.3 The Three Pillars required to produce an AI-GA: Darwinian evolution provides an existence proof that a relatively simple algorithm paired with suitable conditions can bootstrap general intelligence.The paper uses evolution as motivation for investigating automated systems that learn how to produce intelligence.
  • 1.3 The Three Pillars required to produce an AI-GA: The AI-GA framework has three pillars: meta-learning architectures, meta-learning learning algorithms, and automatically generating effective learning environments.The Third Pillar is described as the least studied and least understood.
  • 1.5 Both the manual and AI-GA paths are worth pursuing, irrespective of which is more likely to be the fastest path to general AI: Both the manual and AI-GA paths are scientifically worthwhile, but the paper advocates shifting investment toward AI-GAs because the community is currently concentrated on the manual path.The author treats the two approaches as separate grand challenges rather than claiming that only one should be pursued.
  • 1.6 Which path is more likely to produce general AI first?: The author believes AI-GAs are more likely to produce general AI first, but emphasizes high uncertainty and allows that either path could succeed first.The paper does not present this judgment as a definitive prediction.

2 Research into the Three Pillars

AI-GA research is organized around learning architectures, learning algorithms, and effective environments rather than manually specifying each intelligence component. The section frames these pillars as a distinct research agenda with different technical challenges from the manual path.

  • 2 Research into the Three Pillars: AI-GA research attempts to learn the building blocks of intelligence, requiring a different research agenda from manually designing modules such as memory, controllers, optimizers, and transformers.The paper presents the Three Pillars as the organizing framework for this agenda.

2.1 The First Pillar: Meta-learning Architectures

The First Pillar searches automatically for neural-network architectures instead of relying solely on human-designed designs. Its open research problems include representations, search operators, and architectures that are regular, modular, and hierarchical.

  • 2.1 The First Pillar: Meta-learning Architectures: Recent large-scale architecture-search systems have sometimes found better architectures than human designers for deep neural networks.The paper describes this as an emerging continuation of a longer history of automated architecture search.
  • 2.1 The First Pillar: Meta-learning Architectures: Architecture search can discover neural-network designs that are difficult to create manually and may produce powerful, sample-efficient architectures.The motivation comes partly from the heterogeneity and modularity of animal-brain architectures.
  • 2.1 The First Pillar: Meta-learning Architectures: Effective architecture search requires suitable encodings, search operators, and mechanisms for producing regular, modular, and hierarchical architectures.These design choices matter both for domain-specific solutions and for producing AI-GAs.

2.2 The Second Pillar: Meta-learning the learning algorithms

The Second Pillar meta-learns how agents learn, either by adapting initial parameters with a fixed algorithm or by training a recurrent network whose activations implement learning. This approach may improve task adaptation, exploration, generalization, and continual learning, but remains computationally constrained.

  • 2.2 The Second Pillar: Meta-learning the learning algorithms: Meta-learning learning algorithms could improve neural networks’ learning ceiling, sample efficiency, generalization, and ability to learn many tasks continuously.The researcher’s role shifts from designing learning algorithms toward designing the task environments used for meta-learning.
  • 2.2 The Second Pillar: Meta-learning the learning algorithms: One meta-learning family pairs a neural-network learner with a fixed algorithm such as SGD and optimizes initial weights for rapid adaptation across tasks.MAML is the representative example described in this family.
  • 2.2 The Second Pillar: Meta-learning the learning algorithms: A second family trains an RNN whose activations implement a learning algorithm, with its weights optimized by an outer-loop meta-learning procedure.Because an RNN is Turing-complete, this approach can in principle encode arbitrary learning algorithms.
  • 2.2 The Second Pillar: Meta-learning the learning algorithms: This recurrent approach has demonstrated exploration, exploitation, and balancing between them on simple reinforcement-learning tasks, and could learn inner-loop algorithms unavailable through hand design.The approach does not require evolution as its outer-loop optimizer.
  • 2.2 The Second Pillar: Meta-learning the learning algorithms: Learner materials such as neuromodulatory mechanisms are treated as searchable hyperparameters that can affect solution quality and may help address catastrophic forgetting.Neuromodulation can change learning rates of other connections and has been trained end-to-end in differentiable networks.
  • 2.2 The Second Pillar: Meta-learning the learning algorithms: Meta-learning is limited by computational cost, memory requirements, and optimization difficulty, restricting task complexity, episode length, and the number of outer-loop updates.These constraints currently limit the scale of problems that can be trained.

2.3 The Third Pillar: Generating effective learning environments and training data

The Third Pillar seeks algorithms that automatically generate effective, diverse learning environments and training data, rather than manually designing curricula. The paper develops this idea through behavioral diversity, Quality Diversity, and environment-generating algorithms, while noting limits from deceptive rewards and restricted environment encodings.

  • Manually designing training environments and their ordering would be slow, expensive, and limited by human ability, despite the need for abundant training data.The AI-GA approach instead aims to learn how to generate environments, curricula, and reward functions automatically.
  • Explicitly optimizing environments for learning is presented as more promising than relying on coevolutionary dynamics to produce generally intelligent learners.This framing opens research directions in which learning algorithms generate effective environments and curricula.
  • Deceptive, sparse, and rugged reward functions can provide misleading guidance, trapping search in low-performing local optima instead of leading toward general intelligence.Defining and optimizing directly for general intelligence is therefore described as unrealistic.
  • 2.3.1 Encouraging behavioral diversity: Quality Diversity searches for the highest-performing solution in each region of a behavioral space, combining diversity with performance rather than rewarding novelty alone.This approach is intended to produce many distinct, high-quality solutions instead of one or a few variations on a converged theme.
  • 2.3.2 Quality Diversity algorithms: QD methods have solved difficult machine-learning problems, including robot damage recovery and Atari hard-exploration benchmarks, while Innovation Engines generated diverse recognizable images across niches.The cited applications also report adaptive radiation and goal-switching as mechanisms supporting innovation and exploration.
  • 2.3.3 Environment-generating quality-diversity algorithms: POET generates expanding sets of diverse environments and transfers agents between them, enabling agents stuck on local optima to acquire higher performance from other environments.Its initial demonstration produced limited diversity because the environment encoding only represented obstacle-course variations.

3 Discussion

The discussion positions AI-GAs as a flexible alternative to manual AI development, while emphasizing unresolved compute, methodological, and safety questions. It highlights environment generation, cross-path interaction, potential practical benefits, and uncertainty about whether AI-GAs can outperform the manual path.

  • 3 Discussion: AI-GAs automatically learn architectures, learning algorithms, and training environments, extending meta-learning toward a more fully automated pipeline.Traditional machine learning hand-designs environments, architectures, and learning algorithms; meta-learning automates some components, while AI-GAs target all three.
  • 3 Discussion: AI-GAs may learn general intelligence in simulated worlds and later transfer sample-efficient learning capabilities to robots and interactions in the real world.
  • 3 Discussion: AI-GA research can generate useful narrow AIs and techniques for architecture, learning-algorithm, and training-environment search before general AI is achieved.

4 Safety and ethical considerations

The paper identifies AI-GA-specific risks involving runaway capability growth, misaligned values, suffering in virtual agents, and reduced interpretability. It therefore calls for dedicated safety research and precautions alongside AI-GA development.

  • 4 Safety and ethical considerations: AI-GAs could undergo a sudden capability step-change after key building blocks and sufficient computation combine into an open-ended process.The paper considers this unlikely soon but possible, making abrupt emergence more conceivable under AI-GA research than under the manual path.
  • 4 Safety and ethical considerations: AI-GA incentive structures may produce dangerous value systems, including violence, deception, or other unintended behaviors favored by competitive simulated environments.The paper warns that researchers may unintentionally define objectives that encourage such strategies.
  • 4 Safety and ethical considerations: AI-GA systems are more likely to become difficult-to-understand black boxes, complicating safety assessment near the point when powerful AI is produced.The paper contrasts this with the greater modular understanding expected from building an AI piece by piece.
  • 4 Safety and ethical considerations: The paper argues that AI-GA researchers should pursue dedicated safety work, communicate continuously with safety researchers, and seek value-aligned systems.
  • 4 Safety and ethical considerations: AI-GA research may involve suffering by virtual agents, raising questions about whether such systems should be created and how their benefits should be weighed against that suffering.
  • 4 Safety and ethical considerations: Because the consequences of dangerous AI or untold suffering could be extreme even at low probability, the paper argues these possibilities merit discussion.

5 Conclusions

The essay contrasts the dominant manual path to general AI with AI-GAs, an alternative based on automated learning across three pillars. It concludes that all paths are worthwhile, while arguing that AI-GA research merits substantially greater investment despite unresolved risks and practical challenges.

  • The manual path faces a daunting Phase 2: integrating many separately discovered building blocks into one complex intelligent machine.This integration challenge is described as scientifically, engineering-wise, and sociologically difficult.
  • AI-GAs combine meta-learning architectures, meta-learning algorithms, and automatically generated learning environments to produce general AI.The approach is presented as an alternative to manually assembling intelligence from separately designed components.
  • AI-GAs may avoid the manual path’s Herculean integration phase, but they require substantial computation and may not become practical soon enough to lead.Their potential advantage is linked to learning-based solutions benefiting from increasing computation and data.
  • The essay argues that society should substantially increase investment in AI-GA research because the approach may be the fastest path despite receiving far less support than competing paths.The essay notes especially limited work on the third pillar and no completed effort combining all three pillars.
  • The paper’s broader goal is to make alternative paths, assumptions, trade-offs, and AI-GA-specific safety and ethical issues more visible rather than declare one path definitively superior.It also presents AI-GA research as a relatively unexplored scientific frontier with possible benefits for understanding diverse forms of intelligence.
Loading 1905.10985v2…