Source-linked AI summary
The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence
Gary Marcus
TL;DR
Current AI research emphasizes general-purpose learning, larger datasets, and more compute, but systems remain unreliable outside their training conditions. This paper proposes hybrid architectures combining deep learning with symbolic operations, large-scale knowledge, reasoning, and cognitive models. The proposed four-step program could provide a substrate for richer systems and bring AI closer to a framework that can be trusted.
Problem
Current AI systems remain unreliable outside their training conditions, while robust intelligence requires knowledge, reasoning, and cognitive models for open-ended situations.
Method
The paper proposes a four-step program spanning hybrid neuro-symbolic architectures, rich cognitive frameworks and knowledge databases, abstract reasoning tools, and mechanisms for representing and inducing cognitive models.
Results
The proposed program could provide a substrate for richer, more intelligent systems than are currently possible.
Takeaways & Limitations
Progress on these prerequisites could bring AI much closer to a framework that can be trusted.
Takeaways & Limitations
CYC demonstrates that abstract knowledge and higher-order reasoning can support subtle reasoning, but requires problems in logical form and lacks a natural-language front end and vision.
Abstract
from arXiv · showhide
Recent research in artificial intelligence and machine learning has largely emphasized general-purpose learning and ever-larger training sets and more and more compute. In contrast, I propose a hybrid, knowledge-driven, reasoning-based approach, centered around cognitive models, that could provide the substrate for a richer, more robust AI than is currently possible.
1. Towards robust artificial intelligence
The paper argues that current deep-learning systems remain brittle, data-hungry, and unreliable outside their training conditions, motivating robust AI based on knowledge, internal models, and reasoning.
- Robust artificial intelligence should apply knowledge systematically across problems, reason flexibly about the world, and transfer learning between contexts like an ordinary adult.
- Reliability is necessary for trustworthy AI in open-ended settings, including homes, roads, hospitals, businesses, and communities.
- Current AI systems can perform extremely well on specific tasks yet remain brittle, data-hungry, and unable to handle modest deviations from training assumptions.
- State-of-the-art models often exploit spurious dataset patterns instead of learning meaning in a flexible and generalizable way.
- GPT-2 can generate fluent continuations but its representations are thin and unreliable, failing to track details such as locations or simple counts.Its failures include placing dry-cleaned clothes at the wrong location and reporting seventeen frogs after a 6−2+3 update.
- The proposed response is to develop systems that acquire, represent, manipulate, and reason over abstract knowledge and complex internal models of the external world.
2. A hybrid, knowledge-driven, cognitive-model-based approach
The proposed approach combines perception, cognitive models, and decision-making with mechanisms for abstract generalization, because standard neural networks often fail beyond their training distribution.
- Cognitive systems can be understood as a cycle in which perception builds internal models that guide decisions about entities, properties, and relationships.
- Open-ended reasoning requires detailed, structured internal models informed by substantial knowledge about the world and its dynamics.
- The paper frames its challenge as bridging deep learning's emphasis on learning with classical AI's concerns about knowledge, reasoning, and cognitive models.
- Training a network on the identity function for even numbers does not reliably extend the function to unseen odd numbers.
- Adding hidden layers or more hidden nodes does not change this failure to generalize abstract patterns to novel cases.
- Multilayer perceptrons can memorize training examples and interpolate nearby cases but generalize poorly outside the training space.
2. Excessive dependency on exact details of training regime: whereas all normal
Neural networks can depend heavily on exact training details, whereas human commonsense generalizations apply across broad classes of objects and situations.
- Neural networks can remain sensitive to the order in which training items are presented, unlike human learners acquiring language and world knowledge under varied circumstances.
- Idiosyncratic generalization makes systems unreliable when they must operate beyond their training distribution.
- People generalize that breaking a liquid-filled bottle probably causes some liquid to escape across varied bottle sizes, shapes, colors, and contents.
- The proposed alternative must learn, represent, and extend abstract knowledge beyond the capabilities of backpropagation-trained multilayer perceptrons alone.
2.1. Hybrid architecture
The paper argues that robust AI needs hybrid architectures combining deep learning's data-driven learning with symbolic operations for representing and manipulating abstractions, while acknowledging that hybrids are necessary but insufficient.
- Symbolic operations: Symbolic operations over variables support abstractions by binding variables to instances and applying operations across all members of a class.
- Symbolic operations: The paper argues that symbol-manipulation is required because much useful knowledge is abstract and cannot be handled reliably without representing and manipulating abstractions.
- Hybrid architectures: Hybrid architectures combine deep learning's capacity to learn from large datasets with symbolic methods' capacity to represent and manipulate abstract structures.
- Hybrid architectures: OpenAI's Rubik's solver combines a symbolic algorithm for cognitive aspects with deep reinforcement learning for manual manipulation.
- Hybrid architectures: Other hybrid systems reported benefits in visual question answering, fizz-buzz, mathematics, search, and game playing.
- Evaluation: Determining whether a system is hybrid requires reverse engineering its implementation to identify mappings between low-level computations and algorithmic descriptions.
- Limitations: Hybrid models may be necessary for robust intelligence, but the paper states that they are not sufficient and leaves three further research challenges.
2.2. Large-scale knowledge, some of which is abstract and causal
Robust intelligence requires large-scale structured knowledge, including abstract and causal knowledge, but current systems struggle to acquire, represent, apply, and reason with it reliably.
- Knowledge representation has enabled abstract knowledge in principle, but classical efforts such as CYC required thousands of person-years and achieved limited impact.CYC’s broad commonsense knowledge effort captured facts across many domains in precise logical form, yet evaluation and commercial impact remained limited.
- GPT-2 models word-use statistics rather than concepts, so fluent correlations do not reliably provide conceptual or causal understanding.Its failures include difficulty identifying relevant conceptual properties and distinguishing statistical association from an abstract account of how the world works.
- Weak knowledge representations impair downstream inference, planning, language understanding, and interpretability because systems lack reliable causal, contextual, and plausibility representations.The paper illustrates this with unreliable fire-related reasoning, implausible planning, contextual disambiguation failures, and internally inconsistent associations.
- The paper argues that systematically inducing, representing, and manipulating large databases of structured, often causal knowledge is a prerequisite for robust intelligence.This knowledge must support action, interpretation, decision-making, and reasoning over complex internal models of the external world.
- Robust systems will need substantial external cultural knowledge and causal, counterfactual knowledge, while efficiently connecting abstract knowledge to specific real-world situations remains unsolved.The paper notes that current deep learning systems access external symbolic knowledge only to a limited extent, and that practical integration at scale is difficult.
2.3. Reasoning
The paper argues that robust reasoning requires moving beyond memorization toward systems that apply general knowledge to specific cases. CYC illustrates both the potential of rich knowledge and reasoning and the substantial limitations of manually constructed symbolic systems.
- 2.3. Reasoning: Reasoning can replace exhaustive memorization by applying general truths to specific instances as needed.The paper contrasts memorizing every individual fact with learning a general rule such as all humans are mortal.
- 2.3. Reasoning: Current Transformer systems are too unreliable for sound reasoning when used in isolation from symbol-manipulation tools.The paper attributes this limitation to insufficient robustness, while treating symbol-manipulation as a promising direction when adequate knowledge is available.
- 2.3. Reasoning: CYC combines a sophisticated reasoner with a large-scale knowledge base to derive complex inferences from story and commonsense knowledge.The Romeo and Juliet example includes story summaries, time-specific information, commonsense knowledge, and an inference derived by CYC.
- 2.3. Reasoning: Rich cognitive models encode subtle information about what protagonists do, know, intend, and expect at particular time points.The model is an internal distillation of a complex scenario, while associated commonsense knowledge represents actions, consequences, and human interaction.
- 2.3. Reasoning: CYC provides a best-case proof of concept for rich cognitive models and knowledge, but depends heavily on knowledge engineers who manually formalized Romeo and Juliet.Automating comparable representations and reasoning would be a major breakthrough.
- 2.3. Reasoning: CYC shows that subtle reasoning is possible, yet its practical use is constrained by limited natural-language and visual input and reliance on preformatted logical representations.The system also has unclear capacity for uncertainty and may be brittle because it depends on specific databases and mappings into internal logic.
- 2.3. Reasoning: Robust reasoning requires systems that efficiently leverage large-scale background knowledge even when information is incomplete.The paper also calls for benchmarks that test the sophisticated reasoning exemplified by the Romeo and Juliet scenario.
2.4. Cognitive models
Cognitive models are evolving internal representations of entities, properties, events, and time that support coherent understanding and reasoning. The paper argues that current systems often lack such models, producing fluent local predictions without reliable transfer or consistency across contexts.
- 2.4. Cognitive models: A cognitive model accumulates knowledge about entities, their properties, and time-indexed events as a scenario unfolds.Such models may differ in detail between people but are used routinely to represent particular states of affairs.
- 2.4. Cognitive models: Constructing the right cognitive model for a situation is complex, can have multiple plausible answers, and is not yet automated.The paper presents model construction as a substantial challenge even when complex models can be built manually in principle.
- 2.4. Cognitive models: Model-free reinforcement learning can perform well in closed environments but generalizes poorly when those environments change.In Breakout, moving the paddle a few pixels caused drastic performance reductions, whereas humans could compensate using an internal cognitive model.
- 2.4. Cognitive models: Current Transformers lack explicit cognitive models, limiting their ability to represent, update, and manipulate knowledge for downstream reasoning.The paper contrasts sentence-level prediction with deriving detailed cognitive models from discourse.
- 2.4. Cognitive models: CYC can reason over manually constructed cognitive models and commonsense knowledge, but cannot derive the relevant models from natural-language plot summaries on its own.Automatically producing such models for downstream reasoners would be a major step beyond current AI.
- 2.4. Cognitive models: Rich cognitive models support coherent interpretation by representing individuals and relations, updating those representations, and enabling reasoning about events and actions.The paper argues that pure co-occurrence statistics have not reliably achieved these functions.
- 2.4. Cognitive models: Meena’s fluent, on-topic responses can remain inconsistent because it lacks an enduring record of preferences and an internal model of its own knowledge.The paper gives contradictory claims about whether Meena is learning Cantonese or already natively fluent.
- 2.4. Cognitive models: The paper identifies deriving and updating cognitive models from language or perception, and reasoning over them with commonsense knowledge, as underdeveloped priorities.It recommends making systems that map input into rich, evolving cognitive models a high priority.
3. Discussion
The paper argues that robust AI requires moving beyond data-driven statistical inference toward systems combining learning with knowledge, reasoning, and rich cognitive models. It proposes four cognitive prerequisites and a broader research shift, while acknowledging that these steps would provide only a foundation rather than a complete cognitive being.
- Current limitations: Transformers are unreliable, reason poorly, and fail to build cognitive models of events as they unfold over time.The paper argues that their partial success has created an illusory sense that large-scale machine-interpretable knowledge is unnecessary.
- Four prerequisites: The proposed research agenda combines hybrid learning and symbol-manipulation, large-scale knowledge bases, tractable reasoning, and rich cognitive models.These are presented as four cognitive prerequisites for robust artificial intelligence.
- Architectural scope: Robust AI is predicted to reside in the intersection of learning, symbolic representation, variables, and records for individuals, but that intersection contains vast numbers of possible architectures.Identifying the right region is therefore only a starting point; the appropriate architecture remains unspecified.
- Knowledge and reasoning: The paper calls for systems that represent core frameworks of human knowledge, including time, space, causality, physical objects, and human interactions.The architecture should remain extensible while preserving abstraction, compositionality, and individual tracking.
- Research priorities: The proposed shift does not abandon deep learning but places it within a broader coalition that welcomes prior knowledge, reasoning, variables, and cognitive models.The paper also argues for more heterogeneous architectures and greater exploration beyond heavily studied blank-slate empiricist models.
- Prospects and boundaries: Progress toward the four prerequisites could provide a substrate for richer systems and bring AI closer to being trustworthy, but cannot guarantee a complete cognitive being.The paper presents this program as an important next-decade effort rather than a complete solution.