Source-linked AI summary
Neural Programmer: Inducing Latent Programs with Gradient Descent
Arvind Neelakantan, Quoc V. Le, Ilya Sutskever
TL;DR
Deep neural networks struggle with crisp arithmetic and logic reasoning, limiting applications such as question answering. Neural Programmer adds differentiable operation and data-segment selection over multiple steps and learns from execution results without complete program annotations. On synthetic table comprehension, it completely solves the task or exceeds 99% accuracy on most cases, while gradient noise improves difficult training.
Problem
Deep neural networks have difficulty learning simple arithmetic and logic operations, limiting their use in question answering requiring complex reasoning.
Method
Neural Programmer uses a recurrent neural network augmented with arithmetic and logic operations, differentiably selecting operations and data segments to induce programs from weak supervision.
Results
Neural Programmer completely solves the synthetic table-comprehension task or achieves greater than 99% accuracy on most cases; random Gaussian gradient noise improves training.
Takeaways & Limitations
The model demonstrates compositional program induction with substantially less human supervision than prior approaches and broad applicability across domains, data sources, and languages.
Takeaways & Limitations
For text-entry tables, simple averaging fails to produce sufficiently question-specific column representations, requiring a two-stage text-match mechanism; most experiments use numeric-only tables.
Abstract
from arXiv · showhide
Deep neural networks have achieved impressive supervised classification performance in many tasks including image recognition, speech recognition, and sequence to sequence learning. However, this success has not been translated to applications like question answering that may involve complex arithmetic and logic reasoning. A major limitation of these models is in their inability to learn even simple arithmetic and logic operations. For example, it has been shown that neural networks fail to learn to add two binary numbers reliably. In this work, we propose Neural Programmer, an end-to-end differentiable neural network augmented with a small set of basic arithmetic and logic operations. Neural Programmer can call these augmented operations over several steps, thereby inducing compositional programs that are more complex than the built-in operations. The model learns from a weak supervision signal which is the result of execution of the correct program, hence it does not require expensive annotation of the correct program itself. The decisions of what operations to call, and what data segments to apply to are inferred by Neural Programmer. Such decisions, during training, are done in a differentiable fashion so that the entire network can be trained jointly by gradient descent. We find that training the model is difficult, but it can be greatly improved by adding random noise to the gradient. On a fairly complex synthetic table-comprehension dataset, traditional recurrent networks and attentional models perform poorly while Neural Programmer typically obtains nearly perfect accuracy.
1 INTRODUCTION
Neural Programmer combines differentiable neural selection with arithmetic and logic operations to induce compositional programs from weak supervision. On synthetic table comprehension, it substantially outperforms recurrent and attentional baselines, reaching nearly perfect accuracy.
- Motivation: Existing neural models struggle with crisp arithmetic and logic reasoning, including binary addition and table-based counting needed for question answering.Prior semantic-parsing approaches also rely on complex search, complete program annotations, or domain-specific grammars.
- Approach: Neural Programmer augments a recurrent neural network with arithmetic and logic operations selected over data segments across multiple steps.Its controller uses memory of previous actions and operation outputs to make subsequent decisions, enabling compositional programs.
- Approach: Soft selection of operations and data segments makes the model trainable end-to-end by backpropagation, while test-time decisions use hard selection.The target answer adjusts the network toward the operations and data segments that induce the correct program.
- Supervision: Weak supervision uses execution results rather than annotated correct programs, reducing the human supervision required for program induction.The designer specifies only a list of basic operations instead of additional rules guiding program search.
- Results: On a synthetic table-comprehension dataset, Neural Programmer completely solves the task or exceeds 99% accuracy on most cases, whereas LSTM and attention-based LSTM models perform poorly.Training is difficult, but injecting random Gaussian noise into the gradient greatly improves training and generalization.
2 NEURAL PROGRAMMER
Neural Programmer combines recurrent neural networks with differentiable arithmetic and logic operations to induce multi-step programs for table question answering. It selects operations and table columns probabilistically, combines their outputs, and uses execution history to support compositional reasoning.
- Architecture: Neural Programmer runs for T steps, selecting a data segment and operation at each step to induce programs of up to T operations.The implementation uses four modules: a question RNN, selector, operation list, and history RNN.
- Question module: The question module converts input tokens into a distributed question representation using a recurrent neural network.A bidirectional RNN can replace the basic RNN for longer questions, because the unidirectional model may struggle to retain the question's beginning.
- Selector: At every step, the selector assigns probability distributions over available operations and table columns using the question representation and execution history.Operation and column representations are used to produce the respective distributions.
- Outputs: The model supports scalar answers and table lookups, maintaining both output types while operations transform selected data segments.At inference, softmax selections are replaced with hardmax, and the final output is whichever answer type was updated after T steps.
- Operations: Built-in operations access outputs from earlier steps, enabling Neural Programmer to construct compositional programs beyond individual operations.Operations are designed to work with probabilistic row and column selections so the model remains differentiable; reset permits programs shorter than T steps.
- Text entries: For text-valued table entries, Neural Programmer uses a text-matching mechanism to identify rows before applying operations to numeric columns.The text-entry extension combines question-specific representations of text entries with column-name representations.
3 EXPERIMENTS
The experiments evaluate Neural Programmer on increasingly difficult synthetic table-comprehension tasks, including multiple columns, word variability, text entries, and comparisons with recurrent baselines. Neural Programmer achieves perfect or near-perfect performance in settings where LSTM models perform poorly, while gradient noise improves generalization.
- Experimental settings: The experiments progressively increase difficulty through more columns, mixed numeric and text entries, and varied wording in questions.The evaluation compares several synthetic table-comprehension variants before benchmarking against LSTM and attention models.
- Training: Neural Programmer uses four recurrent steps and is trained with mini-batch stochastic gradient descent and Adam.The model uses a batch size of 50 and hidden dimensionality d = 256; gradient noise is added during training to encourage program exploration.
- Neural Programmer results: Neural Programmer generalizes to unseen question templates, including cases where more than two-thirds of test templates are unseen during training.The authors interpret this as evidence of compositionality because unseen templates require programs absent from training.
- Optimization: Adding random noise to gradients almost always improves generalization compared with training without noise under otherwise identical five-column settings.Figure 8 compares the two training conditions with the same hyper-parameters.
- Baseline comparison: The best LSTM and attention-model accuracy is close to 80%, then drops to 30% when test numbers are scaled from [−10, 10] to [−50, 50].These baselines use three-layer sequence-to-sequence LSTMs, with multiple attention configurations and table-order variants.
- Neural Programmer results: 100% accuracy is achieved on the single-column task using 50,000 training examples.The model’s hardmax test-time operation makes its answers invariant to the scale of numbers and the input length.
4 RELATED WORK
Related work frames Neural Programmer within program induction and neural approaches to semantic parsing, grammar learning, and table question answering. The cited approaches differ in their reliance on grammars, preprocessing, or complete program supervision.
- Program induction: Program induction has been studied for semantic parsing, including natural-language interfaces that map questions to executable queries over structured databases.Prior work includes semantic parsers for table question answering and hierarchical Bayesian learning of simple programs.
- Neural approaches: Earlier neural methods learned context-free or context-sensitive grammars for small problems and simple Horn clauses in large knowledge bases.Other neural question-answering systems addressed datasets without complicated arithmetic and logic reasoning.
- Subsequent work: Neural Programmer-Interpreters use supervision of the entire program, while Neural Enquirer achieves perfect accuracy only with that supervision.These methods are presented as subsequent or related approaches to program induction and synthetic table question answering.
5 CONCLUSIONS
The paper develops Neural Programmer to perform complex arithmetic and logic reasoning with less sophisticated human supervision than prior work. Its experiments indicate learning with delayed supervision and compositionality.
- Conclusions: Neural Programmer augments a neural network with arithmetic and logic operations to induce programs through end-to-end backpropagation.The model is presented as broadly applicable across domains, data sources, and languages.
- Conclusions: The experiments indicate that Neural Programmer can learn with delayed supervision and generalize compositionally.The stated conclusion concerns the model’s ability to induce programs beyond those directly encountered during training.
APPENDIX
The appendix lists the synthetic question templates used to test aggregation, comparison, logic, arithmetic, word variability, and text matching. These templates define the task variants used in the experiments.
- Question templates: The single-column appendix contains 23 templates spanning aggregation, comparison, logic, and arithmetic questions.The categories include sum and count, greater and lesser comparisons, conjunction and disjunction, and difference operations.
- Question templates: An additional table lists eight two-column templates combining greater-than, lesser-than, and sum operations.These templates use the form “greater [number1] and lesser [number2] sum.”
- Word variability: The appendix includes a table describing multiple ways to refer to the same operation.This supports the word-variability experiment.
- Text matching: The text-match experiment includes ten templates in which a word condition on one column is combined with summation over another.The listed form is “[word] A sum B” in the two-column text-match setting.