Source-linked AI summary
Memory Networks
Jason Weston, Sumit Chopra, Antoine Bordes
TL;DR
Memory networks address the challenge of combining inference with a large, readable and writable long-term memory for prediction. The paper presents a trainable architecture for question answering and evaluates it on large-scale and simulated-world tasks. The results show viability on large-scale QA, multi-step reasoning in simulated stories, and correct answers with previously unseen words.
Problem
Most machine learning models lack an easy way to read and write potentially large long-term memories while combining them seamlessly with inference.
Method
Memory networks jointly train inference components with a readable and writable long-term memory, using the memory as a dynamic knowledge base for question answering.
Results
Memory networks are viable for large-scale QA, chain multiple supporting statements in simulated-world QA, and correctly answer questions containing previously unseen words.
Takeaways & Limitations
The framework provides a memory-based approach to QA that supports reasoning over stored information and generalization to unfamiliar words.
Takeaways & Limitations
The proposed forgetting procedure for full memories was not explored experimentally, and a single-dictionary linear-embedding variant performed worse.
Abstract
from arXiv · showhide
We describe a new class of learning models called memory networks. Memory networks reason with inference components combined with a long-term memory component; they learn how to use these jointly. The long-term memory can be read and written to, with the goal of using it for prediction. We investigate these models in the context of question answering (QA) where the long-term memory effectively acts as a (dynamic) knowledge base, and the output is a textual response. We evaluate them on a large-scale QA task, and a smaller, but more complex, toy task generated from a simulated world. In the latter, we show the reasoning power of such models by chaining multiple supporting sentences to answer questions that require understanding the intension of verbs.
1 INTRODUCTION
Memory networks address the difficulty of combining inference with a large, compartmentalized long-term memory. They learn to read and write this memory jointly with inference for tasks such as answering questions about supplied facts or stories.
- Most machine learning models lack an easy way to read and write parts of a potentially large long-term memory while performing inference.The paper argues that hidden states and weights in typical recurrent models are often too small and insufficiently compartmentalized for accurately remembering extended inputs.
- Memory networks combine learned inference strategies with a readable and writable memory component.The model is trained to operate effectively with the memory rather than treating memory and inference as separate mechanisms.
- The proposed framework is instantiated for text-domain question answering, where facts or stories are followed by questions about their subject.
2 MEMORY NETWORKS
A memory network combines an indexed memory with four components that encode inputs, update memories, read and infer from them, and produce responses. The framework supports learned memory operations, test-time storage, retrieval over subsets for scale, and optional forgetting when memory capacity is exhausted.
- A memory network consists of an indexed array of memory objects and four components: I, G, O, and R.I maps inputs to internal features; G updates memories; O computes output features from the input and memory; R converts outputs into responses.
- Given input x, the model represents it with I, updates each memory with G, computes o with O, and decodes the response as r = R(o).
- The components can use existing machine-learning methods, and memories are stored at test time without updating the parameters of I, G, O, or R.
- If memory becomes full, G can support generalized or revised memories, while H could overwrite the least useful entry; the latter was not experimentally explored.
- For very large memories, a slot-selection function can organize entries by entity or topic so G and O operate on a retrieved candidate subset.This reduces the need for both components to process all memories.
- In question answering, O finds relevant memories and R generates the final wording, with the response model conditioned on the retrieved memory output.
3 A MEMNN IMPLEMENTATION FOR TEXT
The text MemNN implementation stores textual inputs in memory, retrieves supporting sentences, and ranks a textual response. Extensions address word-stream segmentation, temporal reasoning, training, and scalable memory lookup.
- MemNNs are neural-network instantiations of memory networks for textual input and output.
- 3.1 BASIC MODEL: The basic model stores each input sentence in the next available memory slot, leaving older memories unchanged.At test time, memories are stored while model parameters remain fixed.
- 3.1 BASIC MODEL: For k = 2, the O module retrieves a second supporting memory using both the question and the first retrieved memory.The resulting output is the list [x, mo1, mo2].
- 3.1 BASIC MODEL: The R module converts retrieved output features into a textual response, ranging from returning a retrieved sentence to ranking a single-word answer.An RNN can instead be used for true sentence generation.
- 3.1 BASIC MODEL: In the milk example, retrieval chains “Joe left the milk” with “Joe travelled to the office” to answer “office”.The example requires combining multiple supporting sentences and interpreting the actions and temporal order in the story.
- 3.1 BASIC MODEL: A single dictionary with linear embeddings performed worse in unshown experiments, and deeper context-dependent transformations were left for future work.
- 3.2 WORD SEQUENCES AS INPUT: A learned segmentation function writes word sequences to memory when their score exceeds the margin, extending learning to the write operation.The segmenter is presented as a first proof of concept, with more sophisticated designs left open.
- 3.4 MODELING WRITE TIME: The model represents write-time relations with three binary temporal features and uses hashing to avoid scoring every memory when storage is very large.The temporal features compare the ages of the input and candidate memories; hashing restricts scoring to selected buckets.
4 RELATED WORK
The paper situates memory networks among document-retrieval, knowledge-base, neural, differentiable-memory, and alignment methods. It distinguishes its approach by jointly combining inference with readable and writable memory rather than using a two-stage strategy.
- Classical QA uses documents and information retrieval, while newer approaches construct knowledge bases and map questions to logical queries.
- Neural-network and embedding approaches had also been explored for question answering.
- Memory networks differ from recent knowledge-base methods by not applying a two-stage strategy.
- Earlier differentiable-memory models included pushdown automata, rapidly changing weights, and self-modifying network weights.
- Neural Turing Machines also use large addressable read-write memory, but their experiments limited memory to 128 locations, whereas this work considered up to 14M sentences.
- RNNSearch and handwriting-recognition systems dynamically learn alignments while predicting outputs, providing related examples of input-output alignment mechanisms.
- Table 1 reports results on the large-scale QA task of Fader et al. (2013), while Table 2 reports memory-hashing results on that task.
5 EXPERIMENTS
Experiments evaluate MemNNs on large-scale QA and simulated-world tasks, including multi-step reasoning, sentence answers, unseen words, and combined real- and simulated-world data. MemNNs achieve viable large-scale QA performance, substantial speedups with clustered hashing, and stronger reasoning and generalization than RNN and LSTM baselines on harder simulated tasks.
- 5.1 LARGE-SCALE QA: 14M statements from ClueWeb09 form the large-scale QA memory, covering diverse subject-relation-object facts.The task re-ranks candidate answers using test-set F1 and human annotations.
- 5.1 LARGE-SCALE QA: ∼80x speedups preserve similar performance with cluster-based hashing, whereas string hashing is faster but loses performance.The cluster-based method uses K = 1000 clusters.
- 5.2 SIMULATED WORLD QA: MemNNs outperform RNNs and LSTMs on simulated-world QA requiring multiple statements to infer object locations and action meanings.The simulation contains 4 characters, 3 objects, and 5 rooms, with generated movement and object-manipulation stories.
- 5.2 SIMULATED WORLD QA: MemNNs perform 2-stage inference with k = 2 on the harder actor+object task, while k = 1 MemNNs, RNNs, and LSTMs fail.Time features are necessary for before questions and difficulty levels above 1.
- 5.2.1 QA WITH PREVIOUSLY UNSEEN WORDS: MemNNs correctly answer questions containing previously unseen Lord of the Rings words by generalizing learned verbal patterns through 2-stage inference.Without unseen word modeling, they completely fail on this task.
- 5.2 SIMULATED WORLD QA: An ensemble trained on simulated and large-scale QA data can attempt both general-knowledge and story-specific dialogue questions.The system outputs the response from the two highest-scoring model choices.
6 CONCLUSIONS AND FUTURE WORK
The paper introduces memory networks and demonstrates one QA instantiation, while identifying harder reasoning tasks, richer architectures, and weaker supervision as future directions.
- 6 CONCLUSIONS AND FUTURE WORK: The paper introduces memory networks and demonstrates one instantiation for question answering.Future work includes harder QA, open-domain machine comprehension, more complex simulations, sophisticated memory management, and weakly supervised settings.
A SIMULATION DATA GENERATION
The simulation generates executable text-adventure stories and location questions from constrained actions, while exposing a controlled but limited testbed for language understanding.
- Actions: Its underlying action set includes navigation, object manipulation, giving, dropping, looking, inventory, and examination.
- Actions: Action constraints prevent invalid behavior, such as acquiring an object already held or dropping one the actor does not possess.
- Execution: A hand-built model selects random valid actions, with experiments restricted to actor-only or actor-plus-object behavior.
- Execution: The resulting action sequences become executable text stories, from which the simulator derives questions and exact answers about the underlying state.
- Language generation: The grammar adds lexical variety through synonyms, while compound sentences join statements with varied connective phrases.
- Limitations: The testbed omits coreference and compound noun phrases, is currently limited to location QA, and is explicitly not a substitute for real data.
B WORD SEQUENCE TRAINING
Word-sequence training adds learned segmentation so streams of words can be converted into memory segments using supervision from supporting facts.
- Segmentation: A learned segmenter identifies breakpoints in an unsegmented word stream and writes each detected segment to memory.
- Training: The segmenter is trained to activate for known supporting facts while rejecting other or unfinished segments.
- Training: Its parameters are represented by Wseg, a vector functioning as a linear classifier in embedding space.
C WRITE TIME FEATURE TRAINING
Write-time features let memory networks use the relative order in which story memories were written, with training modified to match sequential inference.
- Motivation: Write-time features are added because statement order matters for story questions but not for questions about fixed facts.
- Inference: At inference, each supporting memory is selected by scanning memories sequentially and retaining the current winner against the next memory.
- Features: The model scores memories with three binary relative-age features describing whether inputs and candidate memories are older than one another.
- Training: Training replaces the earlier hinge loss with terms that directly match the sequential selection algorithm, while retaining the final response-ranking term.
D WORD-SEQUENCE LEARNING CURVE EXPERIMENTS
The learning-curve experiments vary training-set size for k = 2 MemNNs with and without time features, showing strong data efficiency on the hardest simulated tasks.
- Experimental setup: MemNN test accuracy is measured with 100, 500, 1000, and 3000 training questions on the word-sequence simulation QA task.
- Results: On difficulty 5 actor and actor + object tasks, MemNNs outperform LSTMs while using 30 times fewer training examples.
E SENTENCE-LEVEL EXPERIMENTS
At the sentence level, MemNNs outperform RNNs on the simulation QA task. Solving the actor w/o before + object task requires retrieving two supporting statements and using time features.
- MemNNs outperform RNNs on the sentence-level simulation QA task.
- Inference retrieving k = 2 supporting statements and time features are necessary for the actor w/o before + object task.
- The sentence-level experiment uses presegmented statements and questions rather than a stream of words.
F MULTI-WORD ANSWER SETTING EXPERIMENTS
In the multi-word answer setting, MemNNs generate textual responses from retrieved features. MemNNs with LSTMs outperform MemNNs with RNNs, while both variants outperform conventional RNNs and LSTMs.
- MemNN response modules receive features [x, mo1, mo2] after the input, generalization, and output modules have run.
- The evaluation counts a generated answer as correct when it contains the correct location and optionally the subject or a correct pronoun.
- MemNNs with LSTMs outperform MemNNs using RNNs in multi-word answer generation.
- Both MemNN variants outperform conventional RNNs and LSTMs by some distance.