Source-linked AI summary
When Brain-inspired AI Meets AGI
Lin Zhao, Lu Zhang, Zihao Wu, Yuzhong Chen, Haixing Dai, Xiaowei Yu, Zhengliang Liu, Tuo Zhang, Xintao Hu, Xi Jiang, Xiang Li, Dajiang Zhu, Dinggang Shen, Tianming Liu
TL;DR
AGI aims to create machines capable of broad intellectual tasks, and brain-inspired AI is explored in relation to that goal. This article surveys the field's progress, technologies, and limitations, concluding that it is promising but substantial work remains.
Problem
AGI aims to create machines that handle a wide range of tasks, learn from experience, and generalize their knowledge to new situations.
Source: §1 · p. 4Method
The article surveys brain-inspired AI in relation to AGI, covering its progress, characteristics, technologies, system evolution, limitations, and future.
Source: §6 · p. 17Results
Brain-inspired AI is a promising field with potential to unlock mysteries of human intelligence and pave the way for AGI, but significant work remains.
Source: §6 · p. 17Takeaways & Limitations
Progress toward AGI will require advances in technology, algorithms, and hardware, alongside continued collaboration across disciplines.
Source: §6 · p. 17Current AGI and brain-inspired systems require far more training data than humans, leaving efficient few-sample learning an open question.
Source: §5.1 · p. 16
Abstract
from arXiv · showhide
Artificial General Intelligence (AGI) has been a long-standing goal of humanity, with the aim of creating machines capable of performing any intellectual task that humans can do. To achieve this, AGI researchers draw inspiration from the human brain and seek to replicate its principles in intelligent machines. Brain-inspired artificial intelligence is a field that has emerged from this endeavor, combining insights from neuroscience, psychology, and computer science to develop more efficient and powerful AI systems. In this article, we provide a comprehensive overview of brain-inspired AI from the perspective of AGI. We begin with the current progress in brain-inspired AI and its extensive connection with AGI. We then cover the important characteristics for both human intelligence and AGI (e.g., scaling, multimodality, and reasoning). We discuss important technologies toward achieving AGI in current AI systems, such as in-context learning and prompt tuning. We also investigate the evolution of AGI systems from both algorithmic and infrastructural perspectives. Finally, we explore the limitations and future of AGI.
1 Brain-inspired AI and AGI
The section traces how ideas from human intelligence have informed AI models and architectures, and surveys brain-inspired algorithms and hardware in relation to AGI. It presents the relationship as potentially reciprocal: brain-inspired AI may contribute to AGI, while AGI may deepen understanding of intelligence.
Human intelligence
- Human intelligence combines multimodal integration, parallel processing, neuroplasticity, and high-level reasoning, though inter-region communication mechanisms are not fully understood.The brain is described as having over 86 billion neurons, each capable of forming up to 10,000 synapses.Source: §1 · p. 2
Brain-inspired AI and AGI
- AGI research has drawn on brain principles, from the MCP artificial neuron and perceptron to backpropagation.Backpropagation adjusts artificial-network weights in a way inspired by the brain's modification of connection strengths.Source: §1 · p. 2
- CNNs draw on hierarchical visual-cortex processing [7], while attention and self-attention underpin Transformers, BERT, GPT, and image-patch-based ViT.The passage also reports ViT state-of-the-art performance across various computer-vision tasks.Source: §1 · p. 3
- Studies report shared structural or representational properties between artificial and biological neural networks, including small-world architectures and similarities in visual and language representations.Examples include comparisons involving CNNs, MLPs, ViT, BERT, Wav2Vec2.0, and biological neural networks.Source: §1 · p. 3
- Brain-inspired approaches include visual-attention and core-periphery models, as well as spiking neural networks applied to brain data, muscle-activity decoding, and brain-machine interfaces.The passage describes core-periphery-guided vision models and CNNs, and SNN applications across several settings.Source: §1 · p. 4
- Neuromorphic chips such as IBM's TrueNorth and Intel's Loihi use spiking neural networks and have been applied to recognition, robotics, and autonomous vehicles.Neuromorphic hardware processes information in parallel and distributed ways, and may support generalized hardware platforms.Source: §1 · p. 4
- Brain-inspired AI is presented as a potential route to adaptable AGI, while AGI may in turn benefit human intelligence and deepen understanding of intelligence.The section links brain-inspired systems to handling complex tasks, learning from experience, and generalizing to new situations.Source: §1 · pp. 4–5
2 Characteristics of AGI
The section frames AGI-relevant intelligence around scale, cross-modal integration, alignment with user intentions, and reasoning. It surveys model examples and techniques associated with these characteristics.
2.1 Scale
- Brain scale is measured by neuron count, which the paper describes as correlated with cognitive abilities; human brains contain over 86 billion neurons.The section also notes that brain-region size and complexity are related to neuron counts.Source: §2.1 · p. 5
- GPT-3 has 175 billion parameters and was trained on 570 gigabytes of text, versus GPT-2’s 1.5 billion parameters and 40 gigabytes; parameter count alone does not determine performance.The paper also reports GPT-3 achieved human-like performance on several language benchmarks.Source: §2.1 · pp. 5–6
2.2 Multimodality
- AGI should integrate knowledge across modalities, connecting shared concepts so information learned from images can support language answers and text can support visual tasks.The proposed multimodal approach aligns representations and training signals across modalities.Source: §2.2 · pp. 6–7
2.2.1 Text-to-image and image-to-text generation
- CLIP learns associations between images and descriptions, DALL-E generates images from text, and GLIDE uses diffusion for image generation.Stable Diffusion works in pretrained autoencoder latent space, while VisualGPT adds a self-resurrecting activation unit to produce sparse activations and prevent accidental overwriting of linguistic knowledge.Source: §2.2.1 · p. 7
2.2.2 Visual question answering
- METER uses flexible vision-language transformer sub-architectures, while VLMo combines dual and fusion encoders with modality-specific experts and shared self-attention.The passage reports benchmark performance but supplies no specific scores.Source: §2.2.2 · pp. 7–8
2.2.3 Video-language modeling
- GPT-4 is described as near human-level across varied domains and better than ChatGPT on textual tasks, and the authors call it an incomplete early AGI.For video-language tasks, ClipBERT uses sparsely sampled frames; later models add masked visual-token modeling or sparse attention.Source: §2.2.3 · p. 8
2.3 Alignment
- The paper attributes unintended LLM behaviors partly to misalignment between language-modeling objectives and safely following user instructions, and describes RLHF as tuning models with human preferences.RLHF is presented as a way to reduce unintended behaviors and improve reliability for complex tasks.Source: §2.3 · p. 9
2.4 Reasoning
- LLM reasoning approaches include prompts that elicit existing abilities and code added during pretraining to enhance reasoning through more diverse training data.The paper distinguishes demonstrating reasoning with prompts from directly enhancing reasoning through code training.Source: §2.4 · pp. 9–10
3 Important Technology
The section surveys prompt-based technologies for AGI, emphasizing in-context learning and instruction alignment. It connects rapid task adaptation to human learning while noting computational and prompt-design costs.
3 Important Technology
- Language-model prompting includes zero-shot, few-shot, in-context learning, and instruction, with the article emphasizing in-context learning and instruction.Instruction provides task descriptions in natural language, while few-shot prompts provide examples as templates.Source: §3 · pp. 10–11
3.1 In-Context Learning
- Large pretrained models can adapt to novel tasks from prompts without extensive labeled fine-tuning data, but their scale raises computational costs and effective prompts may require expertise.The article points to medical and robotics as fields where labeled data may be limited or unavailable.Source: §3.1 · pp. 11–12
- In-context learning uses a task description or a few input-output examples to perform new tasks without further training or fine-tuning.The article relates it to fine-tuning-like changes in prediction, representation, and attention, and says it can reduce overfitting to downstream labeled data.Source: §3.1 · p. 11
3.2 Prompt and Instruction Tuning
- Instruction tuning addresses whether powerful generative models follow user intent and produce truthful, harmless outputs, using human demonstrations, rankings, and preference-based reinforcement learning.InstructGPT also mixes in some original GPT-3 training data to retain classical NLP tasks; larger and more diverse instruction datasets have been reported to enhance performance.Source: §3.2 · p. 12
4 Evolution of AGI
The section traces AGI from early AI ideas through modern deep learning and describes the algorithms and infrastructure behind current systems. It presents AGI as adaptable across tasks, while noting that genuine AGI has not yet been achieved.
4.1 Early Days of AI
- Early AI developed from Turing’s machine-intelligence proposal and the Dartmouth workshop, but limited early systems and unmet expectations contributed to an AI winter.Neural networks and backpropagation later renewed interest and helped establish a foundation for modern neural networks.Source: §4.1 · p. 13
4.2 Deep Learning and Modern AGI
- Deep learning, Transformers, and large language models advanced language, vision, and multimodal modeling, although GPT-2 and GPT-3 are described as progress toward AGI rather than AGI themselves.The section also describes ChatGPT’s use of RLHF and GPT-4’s capabilities and examination performance.Source: §4.2 · pp. 13–14
4.3 The Infrastructure of AGI
- Neural-network infrastructure progressed from simple early models to multilayer training with backpropagation, while GPUs, TPUs, and large-scale computing investments enabled more efficient deep-network training.The section describes an Azure AI supercomputer with over 285,000 CPU cores and over 10,000 GPUs.Source: §4.3 · pp. 14–15
- AGI infrastructure combines large datasets, substantial computation, and distributed systems, as illustrated by GPT training and frameworks for distributing work across devices.The section reports WebText’s 45 terabytes before preprocessing and deduplication and around 40 gigabytes afterward.Source: §4.3 · p. 15
5 Discussion
The discussion identifies technical, ethical, and computational barriers to human-level AGI and outlines research directions involving foundation models, system integration, and brain-inspired learning. It also stresses responsible development as systems become more powerful and pervasive.
5.1 Limitations
- Limited understanding of the brain and the data demands of current systems constrain efforts to replicate human intelligence and match human few-shot learning.The paper identifies learning efficiently from few samples as an open question.Source: §5.1 · pp. 15–16
- Ethical alignment and safety remain concerns because AGI decisions may have far-reaching consequences, while current large language models require substantial computation and energy.The paper says robust safety mechanisms and alignment with human values are essential, and notes that cost may limit who can develop and deploy systems.Source: §5.1 · p. 16
5.2 Future of AGI
- Future work may advance AGI through more powerful foundation models and improvements to hardware and computational algorithms.The article cites ChatGPT and GPT-4 as models with impressive capabilities in various applications.Source: §5.2 · p. 16
- Integrating AI systems across language, vision, and robotics with expert feedback is presented as one approach to more versatile systems and cross-domain knowledge transfer.The section also calls for efficient instruction methods, in-context learning, reasoning, and brain-inspired approaches to learning from unstructured data and generalizing from few examples.Source: §5.2 · pp. 16–17
- The paper calls for responsible AGI development that considers bias, privacy, and security and aligns systems with human values.It frames AGI as still challenging while recognizing its potential benefits to society.Source: §5.2 · p. 17
6 Conclusion
The conclusion presents brain-inspired AI as a promising route toward AGI while emphasizing that substantial work remains. It calls for advances across technology, algorithms, and hardware, alongside interdisciplinary collaboration.
6 Conclusion
- The article provides an overview of brain-inspired AI from an AGI perspective, covering progress, intelligence characteristics, technologies, evolution, limitations, and future directions.Source: §6 · p. 17
- The conclusion describes brain-inspired AI as promising for exploring human intelligence and pursuing AGI, while stating that significant work remains to achieve AGI.Source: §6 · p. 17
- Further progress toward AGI is said to require advances in technology, algorithms, and hardware, as well as continued collaboration across disciplines.Source: §6 · p. 17