Source-linked AI summary
Neural Turing Machines
Alex Graves, Greg Wayne, Ivo Danihelka
TL;DR
Neural networks have largely lacked explicit logical flow control and external memory for learning algorithmic procedures. The paper introduces a differentiable Neural Turing Machine with an attentional controller and memory bank, and finds that it learns simple algorithms and generalises beyond its training regime. Its scope is constrained by controller read-head bottlenecks and finite memory size in some tasks.
Problem
Modern machine learning has largely neglected logical flow control and external memory, despite their fundamental role in computer programs.
Method
The Neural Turing Machine couples a neural controller to a memory bank through selective read and write heads, with the entire architecture trained by gradient descent.
Results
The NTM learns simple algorithms from example data and generalises well outside its training regime across copying, repetition, and associative recall tasks.
Takeaways & Limitations
External memory provides a practical mechanism for neural networks to learn and execute simple programs from examples.
Takeaways & Limitations
Feedforward controllers are bottlenecked by the number of concurrent read and write heads, and copy performance is limited by finite memory size.
Abstract
from arXiv · showhide
We extend the capabilities of neural networks by coupling them to external memory resources, which they can interact with by attentional processes. The combined system is analogous to a Turing Machine or Von Neumann architecture but is differentiable end-to-end, allowing it to be efficiently trained with gradient descent. Preliminary results demonstrate that Neural Turing Machines can infer simple algorithms such as copying, sorting, and associative recall from input and output examples.
1 Introduction
The paper addresses neural networks’ limited use of logical flow control and external memory by augmenting recurrent networks with a large, addressable memory. It introduces the differentiable Neural Turing Machine and evaluates it on tasks requiring simple program induction and execution.
- Modern machine learning has largely neglected logical flow control and external memory despite their fundamental role in computer programs.
- RNNs can simulate arbitrary procedures in principle, but the paper argues that enriching them can simplify algorithmic tasks.
- The Neural Turing Machine adds a large, addressable memory to recurrent networks by analogy with Turing’s infinite memory tape.
- Unlike a Turing machine, an NTM is differentiable and trainable with gradient descent, providing a mechanism for learning programs.
- The experiments test an architecture intended for tasks requiring induction and execution of simple programs, using a constructed battery of problems.
2 Foundational Research
The foundational research frames neural computation through working memory, symbolic and recursive processing, recurrent state, and differentiable attention. These lines of work motivate architectures that combine neural dynamics with structured memory and program-like operations.
- 2.1 Psychology and Neuroscience: Working-memory research studies short-term information manipulation, capacity limits, and neural activity during delay periods.
- 2.1 Psychology and Neuroscience: Computational working-memory models range from biophysical accounts of persistent firing to systems solving explicit tasks, including LSTM-like architectures.
- 2.2 Cognitive Science and Linguistics: Cognitive science and linguistics used information-processing metaphors, while connectionist research challenged symbolic accounts of cognition.
- 2.2 Cognitive Science and Linguistics: Critics argued that neural networks lacked variable-binding, the assignment of particular data to particular roles in a data structure.
- 2.2 Cognitive Science and Linguistics: Recursive processing of variable-length structures is treated as a hallmark of human cognition and as important to cognitive flexibility.
- 2.3 Recurrent Neural Networks: RNNs use dynamic distributed state for context-dependent computation, while LSTM selectively stores information through gated integrators.
- 2.3 Recurrent Neural Networks: Recurrent networks process variable-length structures over multiple steps and have been applied to speech, text, handwriting, and translation.
- 2.3 Recurrent Neural Networks: Prior work includes differentiable attention and program-search methods built with recurrent neural networks.
3 Neural Turing Machines
A Neural Turing Machine combines a neural controller with an externally addressable memory bank, using differentiable selective read and write operations. Its addressing system combines content and location mechanisms to construct memory weightings, while controller choice affects computational capacity.
- Architecture: An NTM combines a neural network controller with a memory bank and uses heads to parameterise selective read and write operations.The controller also communicates with the external world through input and output vectors.
- Architecture: Differentiable, blurred read and write operations let attention focus strongly on one memory location or weakly across many locations.The focus mechanism determines how much each operation interacts with memory elements.
- Reading and writing: Read heads return vectors as convex combinations of memory rows, making the read operation differentiable with respect to memory and its weighting.The weighting vector is normalised over the memory locations.
- Reading and writing: Each write first erases selected memory components and then adds a vector, with independent erase and add components enabling fine-grained modification.The combined erase-and-add operation remains differentiable, and multiple heads can perform these operations in any order.
- Addressing Mechanisms: Addressing interpolates content weighting with the previous weighting, rotates it using a shift distribution, and sharpens the result before memory access.The interpolation gate controls the blend, while the shift weighting supports iteration and random-access jumps.
- Addressing Mechanisms: Content-based addressing matches memory locations to controller-emitted keys, while location-based addressing supports recognisable addresses when variable contents are arbitrary.The authors employ both mechanisms concurrently because location-based addressing proved essential for some forms of generalisation.
- Controller Network: A feedforward controller’s concurrent read and write heads limit the number of memory vectors it can transform simultaneously, whereas recurrent controllers can retain prior reads internally.With one read head the feedforward controller performs unary transforms; two heads permit binary vector transforms.
4 Experiments
Experiments test whether Neural Turing Machines can learn compact algorithms for copying, repetition, associative recall, dynamic prediction, and sorting, then generalise beyond training sequences. Across these tasks, external memory supports faster learning and stronger length generalisation than standard LSTMs, while exposing task-specific limits.
- Experimental setup: The experiments compare feedforward- and LSTM-controller NTMs with a standard LSTM on supervised episodic algorithmic tasks.Networks are reset between input sequences, and the tasks use binary targets.
- Copy: NTMs learned copying faster and at lower cost than LSTM alone, with a disparity suggesting qualitatively different solutions.The copy task requires storing arbitrary binary vectors and recalling them after a delimiter without intermediate input assistance.
- Copy: NTM continued copying beyond the training range while LSTM rapidly degraded beyond length 20, consistent with an induced copy algorithm.The inferred procedure writes each input vector sequentially, returns the head to the start, then reads and emits vectors sequentially.
- Repeat Copy: NTM extended copying across sequence length and repetition count, but could not reliably predict the end marker when repetitions exceeded ten.The authors attribute this boundary to representing the repeat count numerically, which does not easily generalise beyond a fixed range.
- Associative Recall: In associative recall, NTM reached near-zero cost in approximately 30,000 episodes and was nearly perfect for 12-item sequences, while LSTM did not reach zero cost after one million episodes.NTM’s memory was used as a re-writable table, with content lookup and shifted addressing supporting retrieval of subsequent items.
- Dynamic N-Grams and Priority Sort: NTM achieved a small but significant advantage over LSTM on dynamic N-gram prediction, though it did not reach the optimal cost, and both NTM variants substantially outperformed LSTM on sorting.Memory analysis suggests context-specific counting for prediction; sorting used eight parallel read and write heads for best feedforward-controller performance.
5 Conclusion
The Neural Turing Machine combines external memory with differentiable neural computation and learns simple algorithms that generalise beyond its training regime.
- The NTM is a differentiable neural-network architecture inspired by biological working memory and digital computers.
- Experiments show that the architecture can learn simple algorithms from example data.
- The learned algorithms generalise well outside the training regime.