Source-linked AI summary
Compositional generalization through meta sequence-to-sequence learning
Brenden M. Lake
TL;DR
Standard seq2seq networks struggle with systematic compositional generalization, despite its importance for combining learned concepts. This paper trains memory-augmented networks across novel seq2seq episodes, using support examples to solve new queries. Meta seq2seq solves several SCAN tasks and learns variable-based mappings, while longer-sequence extrapolation remains limited.
Problem
Standard seq2seq networks fail on systematic compositionality, such as combining a newly learned concept with an existing modifier.
Method
Meta seq2seq trains memory-augmented networks across episodes of support pairs and query sequences, using external memory to provide query context.
Results
Meta seq2seq solves several SCAN compositional-learning tasks, including arbitrary variable assignments when possible input and output symbols were familiarized during meta-training.
Takeaways & Limitations
The model can operate closer to rule space and use network dynamics plus external memory to assign novel meanings to words at test time.
Takeaways & Limitations
The approach does not generalize systematically to longer output sequences and requires familiarity with possible input and output assignments during meta-training.
Abstract
from arXiv · showhide
People can learn a new concept and use it compositionally, understanding how to "blicket twice" after learning how to "blicket." In contrast, powerful sequence-to-sequence (seq2seq) neural networks fail such tests of compositionality, especially when composing new concepts together with existing concepts. In this paper, I show how memory-augmented neural networks can be trained to generalize compositionally through meta seq2seq learning. In this approach, models train on a series of seq2seq problems to acquire the compositional skills needed to solve new seq2seq problems. Meta se2seq learning solves several of the SCAN tests for compositional learning and can learn to apply implicit rules to variables.
1 Introduction
Systematic compositionality lets people combine familiar primitives into novel utterances, but standard seq2seq models fail when test instructions require systematic recombination. The paper proposes meta seq2seq learning, training memory-augmented networks across episodes to acquire compositional skills for new seq2seq problems.
- Motivation: People can produce and understand novel utterances by combining familiar primitives compositionally.Examples include applying a newly learned verb in combinations with familiar modifiers and activities.
- Problem: SCAN benchmarks compositional instruction following by combining primitives, modifiers, and conjunctions into action sequences.One example maps “walk twice and look right” to “WALK WALK RTURN LOOK.”
- Problem: Standard seq2seq models generalize well on similar train and test sets but fail catastrophically on systematic compositionality.They may fail to understand “jump twice” after learning related “run twice,” “walk twice,” and “jump” examples.
- Approach: Meta seq2seq learning trains memory-augmented networks across episodes, each containing support pairs and query sequences.The network stores support pairs in external memory and uses them to produce query outputs.
- Contribution: Meta seq2seq can implicitly learn rules operating on variables and solve several challenging SCAN tasks, but longer-sequence generalization remains unsolved.New problems are solved using network activation dynamics and external memory without post-training weight updates.
2 Related work
The paper situates meta sequence-to-sequence learning at the intersection of meta-learning, external-memory architectures, and SCAN-focused compositionality research. It distinguishes its sequence-level memory use from related approaches and notes limitations in existing SCAN improvements.
- Meta-learning: Meta-learning has been applied to few-shot image classification, sequential memory tasks, and low-resource machine translation.The paper emphasizes that few-shot visual tasks differ qualitatively from the compositional reasoning tasks studied here.
- External memory: External-memory architectures support reasoning across scenarios, but the meta seq2seq learner uses a simpler memory mechanism than the Differentiable Neural Computer.Meta seq2seq stores and reasons with entire sequences rather than calling memory with every new input symbol.
- SCAN methods: Data augmentation and syntax-based attention improve SCAN performance but remain limited in scope.Syntactic attention uses a symbol-to-symbol mapping module that may be inappropriate for many domains, and meta seq2seq is compared against it.
3 Model
The meta seq2seq model extends an RNN encoder-decoder with an external key-value memory that retrieves support-set outputs while processing each query. Meta-training optimizes this architecture over novel seq2seq episodes so it can use support examples during inference.
- Architecture: The model is a standard RNN encoder-decoder augmented with support-set context, external memory, and episodic training.The encoder sends stepwise query messages to an attention decoder while memory-derived context is infused into those messages.
- Memory construction: Support inputs become memory keys and support outputs become value vectors, while query encoder states form the memory queries.The input encoder uses final support states as keys, and the output encoder stores final support states as values.
- Memory retrieval: Attention weights retrieve weighted value combinations for each query timestep, producing memory output M and stepwise context C.The context is computed by combining each query input embedding h_t with its corresponding memory output M_t.
- Output decoder: The decoder translates context C into an output sequence using recurrent state updates and attention over C.It begins from the final context vector, repeatedly predicts output symbols, and terminates with an end-of-sequence symbol.
- Meta-training: Meta-training uses novel episodes with support and query items, optimizing query negative log-likelihood with one gradient step per episode.An auxiliary support loss can encourage the network to use memory during training.
4 Experiments
Across synthetic mutual-exclusivity and SCAN experiments, meta seq2seq learning enables compositional generalization by using episodic support examples and external memory. It achieves near-perfect performance on several tasks, but does not extrapolate systematically to longer sequences.
- 4.3 Experiment: Adding a new primitive through permutation meta-training: 99.95% correct (SD = 0.08) is achieved on the SCAN add-jump task, compared with 1.2% for the best standard seq2seq network reported previously.The model also outperforms syntactic attention, which achieves 78.4% with high variability across runs.
- 4.3 Experiment: Adding a new primitive through permutation meta-training: 99.48% correct without support loss versus 5.43% in another lesion setting shows that component importance depends on the training regime.Decoder attention remains critical: removing it reduces performance to 10.32% in one setting and 9.29% in another.
5 Discussion
The discussion presents meta seq2seq learning as a step toward compositional generalization, while emphasizing limits on genuinely novel symbols and longer sequences.
- Discussion: Meta seq2seq learners abstract instructions into templates with variables rather than matching only surface patterns.For example, “jump around right twice and walk thrice” is treated as “x around right twice and y thrice.”
- Discussion: Meta seq2seq solves several SCAN compositional-learning tasks but does not generalize systematically to longer sequences.Longer-sequence extrapolation remains a problem for neural networks, including meta seq2seq learning.
- Discussion: Meta seq2seq also offers a framework for studying how people generalize compositionally in dynamic learning environments.The paper presents this as an implication for cognitive science while noting that more work is needed.
- Discussion: The models can assign novel meanings to words at test time using network dynamics and external memory.This ability supports compositional generalization without post-training weight updates.
- Discussion: This variable-learning capability remains limited because meta-training must expose the models to all possible input and output assignments.The limitation prevents extrapolation outside the training space.
- Discussion: The approach may apply to low-resource machine translation, graph traversal, and Flash Fill-style program induction.For traditional seq2seq tasks, the paper suggests alternating standard training with meta-training.
A Appendix: Compositional generalization through meta seq2seq learning
The appendix visualizes how meta seq2seq uses memory and decoder attention on mutual-exclusivity and SCAN tasks. These visualizations show lookup and alignment patterns during compositional inference.
- Figure A.1: In the ME task, the support set and two queries illustrate the inference that “dax” maps to “blue.”The left matrix displays key-value memory attention, while the right matrix displays decoder attention.
- Figure A.1: For known query symbols, memory attention can be allocated to cells that do not contain the current symbol.This counterintuitive pattern reverses before <EOS>, when the corresponding support item is activated.
- Figure A.1: The withheld symbol “dax” produces broad, uniform attention across support items, indicating novelty.The attention pattern differs from the more specific retrieval used for familiar symbols.
- Figure A.1: Decoder attention forms a diagonal alignment between output symbols and corresponding encoded input symbols.The first decoder step is an exception because its hidden state is initialized with the final context step.
- Figure A.2: In SCAN, key-value memory retrieves responses for input primitives while decoder attention aligns execution with “run,” “right,” and “thrice.”The illustrated test episode concerns compositional execution in the “add jump” task.