A Machine With Human-Like Memory Systems

1. Introduction

• Semantic memory concerns general world knowledge, whereas episodic memory concerns personal events reconstructed with their when-and-where context.The paper grounds this distinction in cognitive science.Source: §1 · p. 1
• The authors model an agent with both memory systems and hypothesize that encoding and storing relevant observations in either can improve answers over using only one system.Source: §1 · pp. 1–2
• The paper introduces the Room, an OpenAI Gym-compatible environment for learning to encode, store, and retrieve memories to maximize rewards.Source: §1 · p. 2
• The authors report better performance when an agent collaborates with another agent or a human than when it acts alone.Source: §1 · p. 2

2.1. The Room Environment

• The Room asks agents where an object is and provides one-at-a-time observations of people placing objects; correct answers earn +1, while incorrect answers earn 0.Room changes stochastically: people may place objects in commonsense locations with chance pcommonsense or random noncommonsense locations with chance 1 −pcommonsense, change an object’s location with chance pnew location, or change their object with chance pnew object. RDF-triple-like inputs are readable and writable by humans and machines and enable use of existing knowledge graphs such as ConceptNet.Source: §2.1 · p. 2

2.2. Episodic and Semantic Memory Systems

• Because agents see only one person at a time, they retain observations in bounded episodic and semantic memories and learn what to store, forget, or summarize.Episodic records include person-specific events and timestamps; semantic memories capture general knowledge as person-independent quadruples whose final element records strength.Source: §2.2 · p. 3
• When an episodic record is unavailable, an agent can use semantic commonsense, such as a general laptop-location fact, to answer a question.Semantic memories can summarize repeated observations and do not include people’s names.Source: §2.2 · p. 3

2.3. Hybrid Intelligence

• The environment permits agents to collaborate by using their combined memories, including recent episodic records or shared commonsense knowledge.The paper describes both human-machine and machine-machine collaboration.Source: §2.3 · p. 3

3.1. Data Collection and the Environment Hyperparameters

• The environment uses a restricted ConceptNet subset, 10 objects and names, one location per object, AtLocation questions, tuned event probabilities, and a 1,000-step limit.Source: §3.1 · p. 4

3.2. Single Agent Policies

• Four single-agent policies vary memory type and handling: episodic-only, semantic-only, both with episodic compression, and both with pretrained ConceptNet semantics.Source: §3.2 · p. 4
• Agents have equal total memory capacity, and the comparison includes uniform-random forgetting and answering policies.The dual-memory policy splits capacity between systems; the pretrained-semantic policy can allocate remaining space to episodic memory.Source: §3.2 · p. 4

3.3. Multiple Agent Policies

• Multiple-agent policies otherwise follow the single-agent policies but can use their combined memory systems to answer questions.Source: §3.3 · p. 5

4. Results

• Handcrafted forgetting and answering policies outperform random policies, while randomizing both produces the worst performance.Figure 1 compares handcrafted and random policies.Source: §4 · p. 5
• At low memory capacity, episodic-only memory performs better; as capacity increases, semantic memory helps, and the agent with pretrained semantic knowledge performs better by focusing more on episodic memory.Figure 2 reports total rewards across handcrafted policies and memory capacities after one episode with the best handcrafted policies.Source: Figure 2 · p. 5, §4 · p. 6
• With total capacity held equal, two agents answer more questions than one because they explore in different directions and cover more area.Figure 3 compares single-agent and double-agent performance; its lighter, narrower bars represent the single agent.Source: Figure 3 · p. 6, §4 · p. 6

• ACT-R and Soar emphasize theory but lack computational experiments, making comparison difficult.Source: §5 · p. 6
• Human-based experiments on episodic and semantic memory in object recall do not scale as well as the authors’ computational method.Source: §5 · p. 6
• Episodic Memory Reader studies question answering and learning what to forget, but focuses only on episodic memory and uses numeric embeddings that are difficult to interpret.Source: §5 · pp. 6–7

• Agents with both semantic and episodic memory outperform agents with only one system in the Room, and commonsense-pretrained agents outperform agents without pretraining.Source: §6 · p. 7
• Multiple collaborating agents answer questions better than one agent because they can complement each other’s memories.Source: §6 · p. 7
• The current collaboration setup includes only agents; future work proposes multiple humans and agents and agents asking humans questions when uncertain.Source: §6 · p. 7
• Future work proposes comparing reinforcement-learning agents with handcrafted agents and making the Room more human-like with images, voices, and more entities and relations.Source: §6 · p. 7
• The authors identify conflicts and differing degrees of trust among humans and agents as future collaboration challenges.Source: §6 · p. 7