Source-linked AI summary
Neural Algorithmic Reasoning
Petar Veličković, Charles Blundell
TL;DR
Classical algorithms rely on abstract inputs, while real-world data can be complex, lossy to abstract, or partially observable. Neural algorithmic reasoning trains networks to execute algorithms in latent spaces and connect them to natural inputs and outputs. The paper reports applications in reinforcement learning and genome assembly, while arguing this approach may extend classical algorithms to inputs beyond their designers’ preconditions.
Problem
Real-world data may be difficult to represent in the abstract form classical algorithms require, while deep learning lacks the broad generalization guarantees associated with algorithms.
Source: pp. 1, 3, 4Method
Neural algorithmic reasoning trains a processor network to imitate an algorithm in a high-dimensional latent space, then connects it to natural inputs and outputs through learned encoders and decoders.
Source: pp. 4, 6Results
Neural algorithmic reasoning has proved fruitful in reinforcement learning and genome assembly; XLVIN improves data efficiency over more traditional RL approaches and compares favorably with ATreeC.
Source: p. 6Takeaways & Limitations
The approach may support applying classical algorithms to inputs that substantially generalize the preconditions specified by their designers, while fitting algorithms directly to real-world problems.
Source: pp. 3, 6Abstracting railway networks requires scalar edge capacities, whose evaluation is partly judgment-based because no tested formula includes all relevant variations and imponderables.
Source: pp. 3–4
Abstract
from arXiv · showhide
Algorithms have been fundamental to recent global technological advances and, in particular, they have been the cornerstone of technical advances in one field rapidly being applied to another. We argue that algorithms possess fundamentally different qualities to deep learning methods, and this strongly suggests that, were deep learning methods better able to mimic algorithms, generalisation of the sort seen with algorithms would become possible with deep learning -- something far out of the reach of current machine learning methods. Furthermore, by representing elements in a continuous space of learnt algorithms, neural networks are able to adapt known algorithms more closely to real-world problems, potentially finding more efficient and pragmatic solutions than those proposed by human computer scientists. Here we present neural algorithmic reasoning -- the art of building neural networks that are able to execute algorithmic computation -- and provide our opinion on its transformative potential for running classical algorithms on inputs previously considered inaccessible to them.