A Roadmap towards Machine Intelligence

1 Introduction

• The authors propose combining existing ideas into a coherent program of progressively more capable machines, rather than trying to solve AI all at once.Each intermediate target should be a simplified but self-contained version of the intended system.Source: §1 · pp. 1–2
• The proposal focuses on machines that learn new concepts through communication at a rate comparable to humans with similar prior knowledge.The paper illustrates this with learning subtraction after mastering addition.Source: §1 · p. 2
• The roadmap centers on an interactive learning environment that incrementally develops more intelligent behavior, alongside algorithmic capabilities for learning from it.The paper identifies communication and learning as the two characteristics it considers crucial.Source: §1 · pp. 1–2

2 Desiderata for an intelligent machine

• The proposed desiderata aim to help machines autonomously serve human endeavors while minimizing machine complexity and maximizing behavioral interpretability.The authors present these as desiderata, not a formal characterization of intelligence.Source: §2 · p. 2

2.1 Ability to communicate

• Natural language can specify machine goals and make outputs understandable, while also providing access to knowledge encoded in texts.The paper also frames language as a possible interface between natural language and traditional programming.Source: §2.1 · p. 2
• Interactive conversation can gather explanations, clarifications, instructions, and feedback, and the proposed learning environment emphasizes this interaction.The authors contrast conversation with transmitting information through a book alone.Source: §2.1 · p. 3
• A Teacher can use natural language to teach the Learner a limited command language for interacting with its environment through the same channels.The paper also describes language-mediated perceptual input, such as converting pixel data into object labels.Source: §2.1 · p. 3
• Language-mediated training may limit skills usually gained by observing the world, so the authors suggest adding perception-oriented tasks using encoded continuous signals.They say such tasks would require new scripts, not a change to the learning framework.Source: §2.1 · p. 3
• A minimal bit-stream interface is intended to avoid encoding prior assumptions about future challenges into input and output representations.The authors remain agnostic about internal representations; only the machine’s inputs and outputs need be linguistic.Source: §2.1 · pp. 3–4

2.2 Ability to learn

• Learning is needed so a machine can adapt from experience and change its behavior when users correct mistakes.The paper contrasts this with a machine that would react identically to a situation throughout its lifetime.Source: §2.2 · p. 5
• Users can shape behavior for concrete tasks through positive and negative rewards, while human learning is inspiration rather than a psychologically plausible model for machine development.The authors note that machines lack senses and biological constraints, and can receive explicit tutoring at rates children could not tolerate.Source: §2.2 · p. 5

3 A simulated ecosystem to educate communication-based intelligent machines

• The ecosystem is a controlled, manageable but challenging environment intended to expose limits in current incremental learning and develop skills that can later transfer to real-world tasks.Its structure lets experimenters control tasks, ensure they have solutions, and define success criteria before adding more natural signals such as human communication or Internet access.Source: §3 · p. 6

3.1 High-level description of the ecosystem

• The dynamic ecosystem comprises a Learner, a scripted Teacher that assigns tasks and rewards behavior, and a scripted Environment for linguistic observation and navigation.The Teacher uses limited expected responses to focus on the Learner’s learning and generalization abilities, not to make it self-sufficient; the Environment uses more restricted, explicit language.Source: §3.1 · p. 7
• The Learner maximizes average reward over time, so equally rewarded solutions favor faster strategies and time spent on offline computation can be penalized.Reward is assumed binary so users need not set relative reward amounts; rewards are also expected to become sparse as tasks and plans grow more complex.Source: §3.1 · pp. 8–9
• Tasks progress from repeating and manipulating language strings to associating expressions with primitive actions, composing commands for abstract goals, and asking clarification questions.As Teacher instructions grow more general, they can also introduce limited ambiguity, polysemy, vagueness, anaphora, and quantification; later, reading text could support autonomous information seeking.Source: §3.1 · pp. 9–10
• Free periods are intended to support undirected exploration and replay of recent experience, while curiosity could sustain skill acquisition even if explicit external rewards stop.The authors expect exploration to improve later training performance and replay to help update the Learner’s internal structure using a broader view of accumulated knowledge.Source: §3.1 · pp. 8–10
• Learners are evaluated by the number of new tasks completed successfully within a fixed time, and a proposed competition would test tasks substantially different from development tasks.Examples of test differences include a new Teacher language, Environment layout, obstacles or objects, and a new domain such as commerce.Source: §3.1 · p. 11

3.2 Early stages of the simulation

• The illustrated early curriculum moves from issuing Environment commands toward understanding their effects, generalizing command structure, and handling higher-level orders through interactive communication.The examples assume the Learner already attends to the Teacher and manipulates character strings; the broader task progression is organized in levels without requiring a fixed order within them.Source: §3.2 · pp. 11–12
• Commands must produce intended actions in the world to earn reward, and varied trials are meant to foster generalization across action and object categories rather than literal string memorization.The examples show a barrier preventing a move and distinguish turning left or right as instances of turning, while linking object commands to the object’s location.Source: §3.2 · p. 14
• The Learner initially searches exhaustively through known object names, but can use look to search more efficiently and retain exhaustive search when looking fails.With obstacles, simply moving forward may fail, so the Learner must develop turning-based strategies or receive further Teacher examples.Source: §3.2 · pp. 16, 18
• Paired trials teach the Learner to decompose high-level orders into primitive action sequences, while interactive how-to requests let it follow Teacher directions and earn reward faster than random search.The proposed algorithm bank includes counting, loops, and path-finding, which the Learner could combine for novel missions with hierarchically structured subgoals.Source: §3.2 · pp. 16, 18, 19
• Variable-length demonstrations should teach the Learner that finding an apple means repeating move-and-look until it is found, rather than merely repeating a fixed action sequence.The proposed curriculum also develops counting and reusable modifiers, while building composite skills and search strategies for tasks involving objects and obstacles.Source: §3.2 · pp. 16–18
• The Learner is a bridge from natural-language instructions to a simple command-understanding Environment, and successful machines could automate programming through natural-language communication.Source: §3.2 · pp. 19–20

3.3 Interacting with the trained intelligent machine

• The envisioned machine adapts to unprogrammed tasks, with users able to shape its behavior through language and occasional explicit rewards or understood negative feedback.In the Alice-assistant example, it retrieves useful information from the Web on demand rather than storing all the knowledge needed for its tasks.Source: §3.3 · pp. 20–21
• Proposed uses range from tutoring and research to debugging and programming, while the authors frame AI as a tool for extending human capabilities and deem friendliness concerns premature given current technology.Source: §3.3 · p. 21

4.1 Types of learning

• In the translator example, communication requires learning reward responses and strategies, while later vocabulary learning can simply store word pairs using previously acquired skills.Once the vocabulary is memorized, the translator may need no further learning and can have fixed functionality.Source: §4.1 · p. 22
• As functionality becomes more open-ended, the machine must rely increasingly on general learning from few examples rather than manual programming.The authors say current statistical systems learn from large datasets but have limited adaptability; a speech recognizer cannot become a speech translator just by instruction.Source: §4.1 · pp. 22–23

4.2 Long-term memory and compositional learning skills

• Long-term memory should store facts, algorithms, and learning skills, which can compose existing structures into new persistent skills.The proposed memory lets the machine extend itself by triggering learning skills in response to its current situation.Source: §4.2 · p. 23
• With component abilities already learned, the Learner could compose a new task plan from a single example, store the resulting skill with its label, and continue extending knowledge unsupervised.The paper illustrates this with finding one apple and two bananas and bringing them home; the new skill may be labeled “prepare breakfast.”Source: §4.2 · p. 24

4.3 Computational properties of intelligent machines

• The authors argue that the machine needs a Turing-complete model to represent any algorithm in fixed length, without requiring a Turing machine specifically.They caution that weaker systems may appear to learn complex patterns by memorizing examples, but this does not demonstrate generalization from few examples.Source: §4.3 · pp. 24–25
• They hypothesize that a parallel model with task-dependent growth could support long-term memory, and that connections between new and spawning cells may create useful structure.The contribution of that topology to learning is a conjecture; cellular automata and some L-systems are mentioned as examples with desired properties.Source: §4.3 · p. 25

5 Related ideas

• Unlike Turing’s largely programmed child machine, this proposal starts with elementary skills and treats the ability to learn as the central capability to develop.The authors also distinguish their goal from passing the imitation game: they target tasks humans can perform using a computer, Internet connection, and communication.Source: §5 · p. 26
• The authors distinguish their synthetic environment from early manually programmed simulations, while acknowledging that its assumptions may misrepresent real-world phenomena and that the transition to human interaction needs empirical study.They propose first developing basic abilities in the controlled environment before moving to real-life interaction.Source: §5 · pp. 26–27
• The proposal differs from multiagent systems focused on conflict resolution and distributed problem solving among groups of mostly scripted agents, emphasizing linguistic interaction to train one agent.It also contrasts with semantic-parsing systems tuned to particular tasks and reliant on hard-wired expert knowledge, aiming instead to build skills incrementally from little prior knowledge.Source: §5 · p. 28
• The authors align their program with lifelong and continual learning, but emphasize interaction and composing skills across progressively more complex tasks rather than merely ordering training examples by difficulty.They note that NELL shares incremental refinement and information sharing across tasks, while their current focus is defining tasks rather than methods.Source: §5 · p. 29
• Unlike Atari-learning work that learns separate games disjointly, this proposal builds on prior skills; unlike the Sukhbaatar et al. sandbox, it uses linguistic interaction and progressively compositional tasks.The authors also distinguish Weston et al.’s held-out-data evaluation from their aim of minimizing the trials needed to master tasks.Source: §5 · pp. 30–31
• The authors argue that algorithmic learning methods should generalize to new tasks composed from learned algorithms, using few examples and requiring fewer examples for increasingly related skills.They say existing approaches that store data but not learned algorithms fail in this setting, and report being unaware of a current technique addressing these issues.Source: §5 · p. 31

• The proposed intelligent machine should address all aspects of intelligence within one system, with learning and communication as fundamental abilities.Source: §6 · p. 31
• A simulated environment requires the machine to acquire facts and skills through communication while learning increasingly ambitious tasks.Source: §6 · p. 31
• The proposed computational system may learn algorithmic patterns from a few examples without strong supervision and use long-term memory for data and learned skills.Source: §6 · p. 31
• The authors argue current methods are far from adequate and call for novel techniques beyond currently accepted machine-learning paradigms.Source: §6 · p. 32
• The authors describe the roadmap as only the beginning of a long journey toward AI and invite other researchers to pursue its goals.Source: §6 · p. 32