PaperlayerView exact revision on arXiv

Source-linked AI summary

Neural Module Networks for Reasoning over Text

Nitish Gupta, Kevin Lin, Dan Roth, Sameer Singh, Matt Gardner

arXiv:1912.04971v2cs.CL

TL;DR

Compositional questions over text require multiple reasoning steps, including discrete symbolic operations, and are difficult for models to learn from QA supervision alone. This paper extends neural module networks with differentiable reasoning modules and auxiliary learning signals. It significantly outperforms state-of-the-art models on a module-covered subset of DROP, while broader coverage remains limited by questions requiring unsupported reasoning operations.

  • Problem

    Compositional questions over open-domain text require understanding question structure, extracting information, and performing symbolic reasoning, while learning neural module networks for such questions is challenging.

    Source: §1 · p. 1, §2.2 · p. 3

  • Method

    The paper extends neural module networks with probabilistic, differentiable text-and-symbol reasoning modules and adds an unsupervised extraction loss plus heuristic program and intermediate-output supervision.

    Source: §1 · p. 2

  • Results

    The model significantly outperforms state-of-the-art black-box models on a 21,800-question DROP subset selected to match its designed modules.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The reported results support using neural module networks for compositional questions requiring symbolic reasoning over natural-language text within the scope covered by the designed modules.

    Source: §1 · p. 2, §8 · p. 10

    The evaluation covers a heuristically selected DROP subset, and questions requiring nested counting are outside the defined modules; extending coverage to the full dataset remains challenging.

    Source: §1 · p. 2, §5.2 · p. 8

Abstract

from arXiv · show

Answering compositional questions that require multiple steps of reasoning against text is challenging, especially when they involve discrete, symbolic operations. Neural module networks (NMNs) learn to parse such questions as executable programs composed of learnable modules, performing well on synthetic visual QA domains. However, we find that it is challenging to learn these models for non-synthetic questions on open-domain text, where a model needs to deal with the diversity of natural language and perform a broader range of reasoning. We extend NMNs by: (a) introducing modules that reason over a paragraph of text, performing symbolic reasoning (such as arithmetic, sorting, counting) over numbers and dates in a probabilistic and differentiable manner; and (b) proposing an unsupervised auxiliary loss to help extract arguments associated with the events in text. Additionally, we show that a limited amount of heuristically-obtained question program and intermediate module output supervision provides sufficient inductive bias for accurate learning. Our proposed model significantly outperforms state-of-the-art models on a subset of the DROP dataset that poses a variety of reasoning challenges that are covered by our modules.

1 INTRODUCTION

Compositional questions over text require parsing question structure, extracting relevant information, and performing symbolic reasoning. This paper extends neural module networks to that setting, adds learning signals for difficult intermediate decisions, and evaluates the model on a module-covered DROP subset.

1 INTRODUCTION

  • Compositional text questions require understanding question semantics, extracting relevant information, and performing symbolic operations such as counting and sorting.The paper notes that strong QA performance can rely on surface lexical cues, while compositional QA remains challenging.Source: §1 · p. 1
  • The paper extends neural module networks with probabilistic, differentiable modules for paragraph-level reasoning over text and symbols, including arithmetic, sorting, comparison, and counting.These modules maintain uncertainty about intermediate decisions and support end-to-end differentiable training.Source: §1 · p. 2
  • An unsupervised auxiliary loss supplies inductive bias for information extraction, while heuristic question-program and intermediate-output supervision on 5–10% of training data supports accurate learning.The latter supervision is obtained heuristically for a small subset of the training data.Source: §1 · p. 2
  • The model significantly outperforms state-of-the-art black-box models on 21,800 DROP questions selected to match its designed modules.The subset covers a variety of reasoning challenges; questions requiring reasoning difficult to support with designed modules remain a challenge for broader DROP coverage.Source: §1 · p. 2

2 NEURAL MODULE NETWORKS

Neural module networks parse questions into executable programs whose learned modules compose basic reasoning operations over text. This section describes their components and the learning challenges caused by latent programs and intermediate decisions.

2 NEURAL MODULE NETWORKS

  • A neural module network represents a compositional question as an executable program whose learned modules perform basic reasoning tasks over the paragraph.For example, a program can find field goals, select those in a specified quarter, identify the longest, and relocate attention to who kicked it.Source: §2 · p. 2

2.1 COMPONENTS OF A NMN FOR TEXT

  • An attention-based encoder-decoder parser produces programs, and module input/output types constrain decoding to well-typed program structures.Question attention is passed to the module generated at each decoding step; contextualized question and paragraph representations provide the text inputs.Source: §2.1 · p. 3

2.2 LEARNING CHALLENGES IN NMN FOR TEXT

  • Learning is difficult because free-form questions lack fixed grammars and program supervision, while latent intermediate decisions receive feedback only through the final answer.Joint parser-executor training compounds the problem: errors in one component can provide incorrect training feedback to the other.Source: §2.2 · p. 3

3 MODULES FOR REASONING OVER TEXT

The model uses differentiable modules that propagate uncertainty while reasoning over text, numbers, and dates. Its module set covers grounding, filtering, extraction, comparison, counting, date differences, and span answers.

3 MODULES FOR REASONING OVER TEXT

  • The module inventory supports probabilistic reasoning over text and symbols, with distributions representing module inputs and outputs.Table 1 summarizes representative modules and their expected behavior.Source: §3 · p. 4, Table 1 · p. 4

3.1 DATA TYPES

  • Module data types include soft question and paragraph attentions, distributions over passage numbers and dates, count values from 0–9, time differences, and answer spans.Each type represents its underlying value as a normalized distribution over the relevant support.Source: §3.1 · p. 4

3.2 NEURAL MODULES FOR QUESTION ANSWERING

  • Grounding and filtering modules map question terms to passage tokens and select relevant text, while relocation re-attends to arguments associated with an attended span.Number and date modules extract distributions; comparison and extrema modules select spans using associated values, counting estimates attended spans, time-diff returns date-difference distributions, and span converts paragraph attention to an answer span.Source: §3.2 · pp. 4–6
  • The differentiable maximum-number operation analytically converts an expected number-token distribution into a distribution over its maximum, using a sample-set size of 3.The probability that a number is the maximum is computed from the probability mass at or below that value; the output is redistributed to corresponding passage tokens.Source: §3.2 · p. 6

4 AUXILIARY SUPERVISION

The model supplements end-task QA learning with an unsupervised objective for extracting event arguments and heuristic supervision for programs and intermediate module outputs. These signals are applied to selected modules and subsets of training questions.

4 AUXILIARY SUPERVISION

  • To address difficult joint learning from end-task QA supervision, the model adds an unsupervised extraction loss and heuristic program and intermediate-output supervision for 5–10% of questions.The auxiliary loss targets find-num, find-date, and relocate; the heuristic supervision covers question programs and selected intermediate outputs.Source: §4 · p. 6

4.1 UNSUPERVISED AUXILIARY LOSS FOR IE

  • The unsupervised loss encourages argument attention near its input mention, allowing attention mass to be distributed flexibly within a window of 10 tokens.The approach targets spurious attention to output tokens far from corresponding inputs and is applied to number, date, and relocate attention maps.Source: §4.1 · p. 6

4.2 QUESTION PARSE AND INTERMEDIATE MODULE OUTPUT SUPERVISION

  • Heuristic question-program supervision is generated by manually analyzing questions and applying patterns to 10% of the training questions.For example, supervision specifies question tokens to attend to when predicting a find module.Source: §4.2 · pp. 6–7
  • Intermediate-output supervision addresses the risk that final-answer feedback trains a module to predict only the answer-relevant instance rather than all instances needed for generalization.The example contrasts finding only the shortest goal with finding all goals before selecting the shortest.Source: §4.2 · p. 7
  • Noisy output supervision uses multi-hot targets for find-num and find-date on 5% of questions that also receive program supervision.For longest- or shortest-touchdown questions, nearby numbers to touchdown mentions are treated as find-num outputs; analogous procedures cover selected date and number question types.Source: §4.2 · p. 7

5 EXPERIMENTS

Experiments evaluate the model on a heuristically selected DROP subset spanning several reasoning types. Results show strong overall comparisons alongside uneven performance across question types.

5.1 DATASET

  • The evaluation uses 20,000 training/validation questions and 1,800 test questions (25% of DROP), with the test split drawn from validation.Source: §5.1 · p. 7
  • The heuristic subset covers date comparison and difference, number comparison and extraction, counting, and argument extraction.Source: §5.1 · p. 7

5.2 RESULTS

  • 77.4 F1 with BERT exceeds MTMSN's 76.5 F1, while the GRU model reaches 73.1 F1 versus NAQANet's 62.1 F1.Source: §5.2 · pp. 7–8, Table 2 · p. 8
  • The model outperforms MTMSN on most question types, but counting is unstable and wins only on some runs.Source: §5.2 · p. 8
  • The unsupervised auxiliary objective raises F1 from 57.3 to 73.1; BERT diverges without it, while intermediate-output supervision has a slight positive effect.Source: §5.2 · p. 8
  • With less training data, the model significantly outperforms MTMSN, especially at 10–25% of available supervision.Source: §5.2 · p. 8
  • Errors include cases requiring lookup, natural-language event ordering, or nested counts, the latter outside the defined modules.Source: §5.2 · p. 8

6 RELATED WORK

Prior semantic-parsing and neural-module approaches address compositional reasoning, but the cited work distinguishes this model by applying interpretable, compositional reasoning to natural language. Figure 2 examines the effects of auxiliary supervision and training-data size.

6 RELATED WORK

  • Earlier semantic parsers addressed compositional questions over structured knowledge bases and semi-structured tables.Recent BERT-based DROP models classify among predefined programs, whereas this model offers compositional parses and intermediate reasoning steps.Source: §6 · pp. 8–9
  • The auxiliary loss contributes significantly to performance, whereas module output supervision has little effect; without Hloss, training diverges for the BERT-based model.Figure 2 examines the effect of training-data size on model performance.Source: §6 · p. 9, Figure 2 · p. 9
  • Prior neural module networks mainly addressed synthetic domains; Jiang & Bansal (2019) applied NMNs to HotpotQA but lacked symbolic reasoning modules.The model combines reasoning over natural language with symbolic operations.Source: §6 · p. 9

7 FUTURE DIRECTIONS

A simple addition-and-subtraction extension scores below MTMSN on the full DROP validation set. The discussion identifies reasoning gaps and trade-offs that complicate extending the model beyond its covered operations.

7 FUTURE DIRECTIONS

  • 65.4 F1 on full DROP validation trails MTMSN’s 72.8 F1 after adding only an addition-and-subtraction module.The authors characterize this as a trivial extension of their model.Source: §7 · p. 9
  • Most analyzed errors reflect insufficient reasoning: span pruning, key–value representations, and implicit-argument extraction require additional capabilities.The authors note that interpretable modules are not always clear to design for some of these cases.Source: §7 · p. 9
  • Training on questions whose correct reasoning is outside the modules’ expressivity can update predicted-program modules to perform unintended reasoning, impairing their intended operations.The authors cite this training problem as a reason for focusing on only a subset of the data.Source: §7 · p. 9
  • Module-level error identification can support additional supervision and potential transfer learning, including indirect or distant supervision from other tasks.The authors contrast this route with direct transfer of reasoning in black-box models, which they describe as not straightforward.Source: §7 · pp. 9–10
  • Black-box operations can reduce interpretability and discourage use of sufficient interpretable modules; scarce supervised programs make that combination harder to train.The authors identify combining black-box operations with interpretable modules as a future challenge.Source: §7 · p. 10

8 CONCLUSION

The paper presents probabilistic, differentiable neural modules for compositional symbolic reasoning over natural-language text and finds that unsupervised auxiliary losses help learning. Extending the approach to the full range of DROP reasoning remains challenging.

8 CONCLUSION

  • Probabilistic modules propagate uncertainty through symbolic reasoning operations while remaining end-to-end differentiable.The conclusion also reports that unsupervised auxiliary losses significantly help learning.Source: §8 · p. 10
  • Even the full reasoning range required by DROP remains a significant challenge for neural module networks.The authors characterize restricted expressivity relative to black-box models as a cost of interpretability, compositionality, and improved generalizability.Source: §8 · p. 10

A.1 QUESTION AND PARAGRAPH ENCODER

The model encodes questions and paragraphs as contextualized token embeddings using either a GRU trained from scratch or fine-tuned BERT.

A.1 QUESTION AND PARAGRAPH ENCODER

  • The model represents question and paragraph tokens as contextualized embeddings Q and P, using either a GRU trained from scratch or fine-tuned BERT.The GRU is bidirectional and two-layered; its inputs concatenate fixed GloVe embeddings with learned character-CNN embeddings.Source: §A.1 · p. 12
  • For BERT, the encoder input concatenates question and context as [CLS] Question [SEP] Context [SEP], using bert-base-uncased.The question and context representations are separated from BERT’s output.Source: §A.1 · p. 12

A.2 QUESTION PARSER DECODER

The question parser uses an attention-based decoder to produce executable programs, with decoder attention also passed to modules during execution.

A.2 QUESTION PARSER DECODER

  • A single-layer, 100-dimensional LSTM decodes the question into the program.Source: §A.2 · p. 12
  • At each decoding step, dot-product attention over encoder states is normalized with softmax.Source: §A.2 · p. 12
  • The decoder starts from the question encoder’s final GRU state, or the [CLS] embedding for the BERT-based model.Source: §A.2 · p. 12
  • Training uses beam size 4 and Adam at 0.001 or the BERT optimizer at 1e−5.Source: §A.2 · p. 12

A.3 NUMBER AND DATE PARSING

The appendix describes number and date extraction, count-module pretraining, span prediction, heuristic supervision, and example executions of the model’s reasoning modules.

A.3 NUMBER AND DATE PARSING

  • Paragraph numbers are extracted from parseable tokens without unit normalization, while dates are collected with spaCy NER and normalized by an off-the-shelf date parser.Date mentions are represented as (day, month, year).Source: §A.3 · p. 12

A.4 PRE-TRAINING COUNT MODULE

  • Count-module pretraining helps; it uses synthetic noisy attention vectors with known counts and L2 loss to train the predicted count value.The synthetic targets range from 0 to 9, with sampled non-overlapping spans placed in the vectors.Source: §A.4 · pp. 12–13

A.5 SPAN MODULE

  • The span module scales paragraph attention by [1, 2, 5, 10], encodes it with a bidirectional GRU, and predicts span-start and span-end probabilities.Source: §A.5 · p. 13

A.6 AUXILIARY QUESTION PARSE SUPERVISION

  • For question-program supervision, the parser receives a multi-hot question-attention target whose loss rewards probability assigned to supervised tokens.Source: §A.6 · p. 13
  • Heuristic question patterns supervise date and number comparisons, extrema, counts, and event relocation with corresponding find, filter, and reasoning modules.Source: §A.6 · p. 13

A.7 HEURISTIC INTERMEDIATE MODULE OUTPUT SUPERVISION

  • For selected questions with program supervision, heuristic intermediate targets use nearby matched numbers or dates as outputs for find-num and find-date calls.Source: §A.7 · p. 14

A.8 EXAMPLE PREDICTIONS

  • Example predictions illustrate number and date comparisons, counting, maximum-number selection, and relocating from the selected passage span to an event.The count example predicts a distribution whose mode is 5; the comparison example selects the span grounded to 10.3 rather than 15.3.Source: §A.8 · p. 14, Figure 3 · p. 14, Figure 4 · p. 15
Loading 1912.04971v2…