PaperlayerView exact revision on arXiv

Source-linked AI summary

BoardgameQA: A Dataset for Natural Language Reasoning with Contradictory Information

Mehran Kazemi, Quan Yuan, Deepti Bhatia, Najoung Kim, Xin Xu, Vaiva Imbrasaite, Deepak Ramachandran

arXiv:2306.07934v1cs.CLcs.AIcs.LG

TL;DR

Existing natural-language reasoning evaluations generally assume consistent information, although real-world sources can conflict. BoardgameQA evaluates language models on defeasible reasoning with contradictory inputs and implicit background knowledge, finding that models perform poorly, especially in few-shot settings.

  • Problem

    Natural-language reasoning evaluations usually assume consistent, reliable knowledge, despite many applications requiring reasoning over contradictory information from different sources.

    Source: §1 · p. 1

  • Method

    The paper creates BoardgameQA, a synthetic dataset of defeasible theories and questions that require multi-hop reasoning, conflict resolution, and implicit background knowledge.

    Source: §1 · p. 2

  • Results

    Language models perform poorly on conflicting inputs, especially in few-shot settings compared with finetuning.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results highlight a gap in current language models’ capacity to reason over contradictory and incomplete information.

    Source: §1 · p. 2, §6 · p. 9

    The framework assumes each theory is defeasibly consistent, so preferences can resolve conflicts.

    Source: §3 · p. 4

Abstract

from arXiv · show

Automated reasoning with unstructured natural text is a key requirement for many potential applications of NLP and for developing robust AI systems. Recently, Language Models (LMs) have demonstrated complex reasoning capacities even without any finetuning. However, existing evaluation for automated reasoning assumes access to a consistent and coherent set of information over which models reason. When reasoning in the real-world, the available information is frequently inconsistent or contradictory, and therefore models need to be equipped with a strategy to resolve such conflicts when they arise. One widely-applicable way of resolving conflicts is to impose preferences over information sources (e.g., based on source credibility or information recency) and adopt the source with higher preference. In this paper, we formulate the problem of reasoning with contradictory information guided by preferences over sources as the classical problem of defeasible reasoning, and develop a dataset called BoardgameQA for measuring the reasoning capacity of LMs in this setting. BoardgameQA also incorporates reasoning with implicit background knowledge, to better reflect reasoning problems in downstream applications. We benchmark various LMs on BoardgameQA and the results reveal a significant gap in the reasoning capacity of state-of-the-art LMs on this problem, showing that reasoning with conflicting information does not surface out-of-the-box in LMs. While performance can be improved with finetuning, it nevertheless remains poor.

1 Introduction

BoardgameQA studies language-model reasoning over contradictory information resolved by preferences, while requiring some background knowledge to be supplied by the model. The authors report that models perform poorly on defeasible reasoning, especially few-shot, and that smaller models struggle when information is missing.

1 Introduction

  • The proposed conflict-resolution strategy prefers information from higher-ranked sources, with preferences potentially based on credibility or recency.Preferences can also express exceptions that override generic rules.Source: §1 · pp. 1–2
  • Each example provides a defeasible theory of facts, potentially conflicting rules, and rule preferences, plus a question; its reasoning difficulty can be controlled automatically.Answering requires multi-hop reasoning and conflict resolution. The formulation draws on classical defeasible reasoning [27].Source: §1 · p. 2
  • BoardgameQA withholds some knowledge needed to answer, requiring models to supply implicit background knowledge rather than relying only on provided facts.The authors argue that benchmarks supplying all relevant knowledge do not reflect common real-world reasoning scenarios.Source: §1 · p. 2
  • BoardgameQA is a synthetic dataset for multi-hop reasoning with contradictory inputs, using board-game scenarios to support natural rules and commonsense reasoning.The authors describe it as the first dataset for multi-hop reasoning with contradictory inputs.Source: §1 · p. 2
  • Language models perform poorly on conflicting-source reasoning, especially few-shot, and smaller models struggle when required information is missing from the input.The results suggest that preference understanding and defeasible reasoning do not emerge out of the box in pretrained models.Source: §1 · p. 2

2 Related Work

The related work situates BoardgameQA across text-based logical reasoning, reasoning with conflicting sources, and reasoning with incomplete information. It contrasts the dataset with prior work on consistent facts and rules, single-hop conflict reasoning, and missing-knowledge benchmarks.

Text-based logical reasoning approaches

  • Prior logical-reasoning work fine-tunes LMs to answer directly [11] [4] [38] [18], while proof generation and modular reasoning improve performance and intermediate proof chains [29, 13, 55, 56, 22, 12, 49, 23].The paper experiments with four approaches: fine-tuning without explicit reasoning steps, fine-tuning with explicit reasoning steps, prompt-tuning with chain-of-thought prompting [50], and few-shot in-context learning with chain-of-thought.Source: §2 · pp. 2–3

Text-based logical reasoning datasets

  • Existing text-based logical-reasoning datasets contain only non-contradictory facts and rules; ConditionalQA [45] is the closest comparison, using an “If X then yes, if Y then no” answer format.The paper compares a subset of datasets along three desired features, but the supplied passage does not specify the feature values.Source: §2 · p. 3

Reasoning with conflicts

  • Defeasible reasoning accepts the conclusion from the higher-preference rule in a conflict; prior natural-language work studied single-hop evidence, while this work develops a dataset for multi-hop reasoning with source preferences [26].The problem has applications including legal reasoning, and a recent survey identified it as an important future direction for LM reasoning [54].Source: §2 · p. 3

Reasoning with incomplete information

  • Unlike benchmarks that omit required information or provide no required rules, this work focuses on cases where some knowledge comes from the model and some is supplied as input [44] [5] [2] [48] [47] [15] [43] [21].Existing benchmarks include both setups: missing information that the model must supply, and cases where none of the required rules are provided as input.Source: §2 · p. 3

3 Background and Notation

The section defines logical notation for facts and rules, then formalizes defeasible theories in which preferences resolve conflicts between contrary rule conclusions. It also states consistency assumptions and distinguishes two conflict types by whether the competing rule’s body can be proved.

Background and Notation

  • A monotonic theory contains positive or negative facts and rules; adding facts preserves any fact already derived.Facts use entity–predicate–entity triples, rules have bodies and heads, and ! denotes negation.Source: §3 · p. 3
  • A defeasible theory adds pair-wise rule preferences, allowing a higher-priority rule to defeat contrary evidence from a lower-priority rule.The initial facts are internally consistent and take priority over derived facts; the theory is assumed defeasibly consistent so preferences can resolve conflicts. Many kinds of preferences can be converted into pair-wise relative preferences.Source: §3 · pp. 3–4
  • Type1 conflicts have a provable body for the higher-priority rule, whereas Type2 conflicts have a provable body for the lower-priority rule but not its opponent.In Type1, the opposing rule need not be considered for resolution; in Type2, it must be taken into account.Source: §3 · p. 4

4 The BoardgameQA Dataset

BoardgameQA measures language models’ reasoning with conflicting inputs by generating defeasible theories and questions through backward story generation [53] [22]. Questions receive proved, disproved, or unknown labels.

Dataset formulation

  • Each example pairs a defeasible theory with a question and tests whether the theory proves it, proves its negation, or establishes neither.The three labels are proved, disproved, and unknown.Source: §4 · p. 4

Entities and predicates

  • Theories represent board-game states and rules, with sampled difficulty variables and animal entities and boardgame-inspired predicates that differ across training and test.The variables support variation in properties such as proof depth and the number and types of conflicts.Source: §4 · p. 4

Rule types and question selection

  • Questions are sampled as signed entity-predicate-entity triples, and six rule templates cover quantifiers, conjunctions, and missing information.The fifth template allows a predicate or object in a rule body to fall outside the predefined vocabulary.Source: §4 · pp. 4–5

Theory generation

  • Recursive generation builds rules and sub-questions that support a target question, optionally adds an opposing derivation with source preferences, and stops when the proof reaches the target depth.At depth zero, the target question is added as a fact; the generator tracks facts, rules, and preferences as proofs.Source: §4 · pp. 5–6

Incomplete information

  • For incomplete-information examples, the generator hides intermediate knowledge that models must supply, sampling extra-reasoning cases with probability pMissInfo.The extra knowledge draws on world, commonsense, mathematical, or orthographic reasoning; some categories are withheld from training to prevent learning them there for test use.Source: §4 · p. 6

Conversion to natural language

  • Manually constructed templates convert generated facts, rules, preferences, and questions into text, with some rule templates reserved for test examples.The paper also compares example length and unique-token counts with other deductive-reasoning datasets in Figure 3.Source: §4 · p. 6

Disproved and unknown examples

  • Disproved examples negate a proved example’s question, while unknown examples perturb the theory until a defeasible solver can no longer derive the queried statement.Perturbations include changing predicates or signs, replacing a fact, or reversing a preference.Source: §4 · p. 6

5 Experiments

Experiments show that models struggle with contradictory inputs, especially at greater reasoning depths, and that correct labels do not guarantee correct proofs. Performance also varies with conflict type, information incompleteness, and distractors.

5 Experiments

  • Experiments compare multiple LM architectures and training paradigms using classification accuracy, with a roughly 33% majority-class baseline and balanced-label dataset variations.Each variation includes 1,000 training, 500 validation, and 1,000 test examples, sampled equally across labels.Source: §5 · pp. 6–7

5.1 Can LMs Reason with Contradictory Inputs?

  • Few-shot models perform poorly at all depths, while tuned models do reasonably at depth 1 but degrade substantially at depths 2–3; PaLM 540B outperforms PaLM 62B.Unlike prior monotonic-reasoning results, finetuned models do not reach near-perfect performance at higher depths.Source: Figure 4 · p. 7, §5.1 · p. 7

5.2 Does Correct Label Prediction Mean Correct Proof?

  • Models often generate wrong proofs despite correct labels; few-shot models struggle with conflict resolution, while tuned models perform better and prompt-tuned proofs are less often wrong.On depth 2, evaluation used Rule F1, Conflict F1, and manual Overall Proof Accuracy on 50 sampled examples per model.Source: §5.2 · pp. 7–8, Figure 5 · p. 8

5.4 Which Conflict Type is More Difficult to Resolve?

  • Models perform slightly better with mostly Type1 conflicts; tuned models also perform better overall when datasets favor one conflict type.Type2 conflicts require showing that at least one element in the conflicting rule’s body cannot be proved, unlike Type1 conflicts.Source: §5.4 · p. 8, Figure 7 · p. 8

5.5 Does Information Incompleteness Make Reasoning More Difficult?

  • As required background knowledge increases, finetuned-model performance decreases, while prompt-tuned and few-shot performance remains relatively unchanged.The comparison uses KnowledgeLight, KnowledgeMedium, and KnowledgeHeavy datasets, with pMissInfo set to 0.2, 0.5, and 0.8, respectively.Source: §5.5 · p. 9, Figure 8 · p. 9

5.6 Do Distractors Make Reasoning More Difficult?

  • Tuned-model performance drops substantially with more distractors, while few-shot performance declines monotonically but only marginally, consistent with [40].A small number of distractors does not substantially degrade tuned models and may help avoid learning spurious correlations.Source: §5.6 · p. 9, Figure 9 · p. 9

6 Conclusion

BoardgameQA evaluates language models’ reasoning with conflicting sources and partially provided knowledge. The tested models performed poorly, highlighting a gap in reasoning over contradictory and incomplete information.

6 Conclusion

  • Language models performed poorly on conflicting inputs, and smaller models also struggled when reasoning required additional knowledge from the model.The authors identify reasoning over contradictory and incomplete information as a gap in current language models’ capabilities.Source: §6 · p. 9
  • BoardgameQA measures language models’ reasoning with conflicting sources and includes scenarios where required knowledge is only partially provided as input.Source: §6 · p. 9
  • The authors hope BoardgameQA will guide methods to improve language-model reasoning in this setting or alternative formulations of conflict resolution that better facilitate it.Source: §6 · p. 9

A More Experimental Results and Analysis

The section examines binary classification results, prediction patterns across depths, and examples of model failures on BoardgameQA.

Binary Classification

  • Binary-dataset results showed similar overall patterns, with some improvement for T5 at depth 2.The binary version included only proved and disproved examples at depths 1, 2, and 3 to test the effect of unknown labels versus conflict resolution difficulty.Source: A More Experimental Results and Analysis · p. 14

Examples of Model Failures

  • Failure examples showed errors in conflict and preference reasoning, incomplete information, logical rules, conjunctions, distracting facts, and fidelity to provided facts and rules.Some models altered facts or rules to find a proof even when none existed.Source: A More Experimental Results and Analysis · p. 14

Confusion Matrices

  • For unknown-labeled examples, the proof-tuned model predicted unknown 268 times at depth 1 but only 2 times at depth 2.At higher depths, proof-tuned models tended to generate proved or disproved labels even when the gold label was unknown.Source: A More Experimental Results and Analysis · p. 14

B Experimental Details

Experiments used T5X on v3 TPUs, except for 540B models, which ran on v4 TPUs. Few-shot prompting and model-training settings were adapted to the dataset and method.

Implementation

Few-shot and prompt tuning

  • Few-shot prompts used one randomly selected training demonstration per label, with both types of conflict resolution represented.For three-way classification, the demonstrations covered proved, disproved, and unknown.Source: B Experimental Details · p. 14
  • Prompt tuning used prompt size 100, up to 50K steps, batch size 8, learning rate 0.1, and weight decay 0.0001.Prompt size 100 was chosen based on the best-performing setting in [25].Source: B Experimental Details · p. 14

Model training

  • T5 used batch size 8, learning rate 0.001, and up to 50K epochs; BERT used batch size 16, learning rate 4.6e −5, and 20 epochs.For experiments involving tuning, results were reported for the epoch with the best validation accuracy.Source: B Experimental Details · pp. 14–15

Experimental assumption

  • The footnote sets pConf = 1.0 for the first call to Algorithm 16 to prevent a spurious correlation that could be exploited without reasoning.Source: B Experimental Details · p. 14

C BoardgameQA Details

This section provides further details on BoardgameQA’s generation and properties, including a sample of theory and question generation from Algorithm 16.

C BoardgameQA Details

  • Figure 22 provides a sample of theory and question generation from Algorithm 16.The section introduces it as an illustration of BoardgameQA’s generation process.Source: C BoardgameQA Details · p. 16

C.1 Consistency of the Dataset

BoardgameQA constructs consistent defeasible theories by separating entities across rule branches to prevent derivations from defeating one another, then applies defeasible reasoning to verify that each question follows from the final theory.

C.1 Consistency of the Dataset

  • BoardgameQA avoids inconsistencies and loops by sampling only entities unused in other rules and subquestions, keeping facts and rules in separate branches disjoint.For a rule such as a ∧ b ⇒ c, this prevents derivations in one branch from defeating derivations in another or in later proof stages.Source: §C.1 · p. 16
  • The authors apply defeasible reasoning to the final logical theory to ensure that the question can be derived from it.Source: §C.1 · p. 16

C.2 Incomplete Information · C.3 Entities, Predicates, and Templates

BoardgameQA’s incomplete-information cases test whether models can connect facts to rule conditions using numerical reasoning, background knowledge, and linguistic entailment. The test set uses different entities, predicates, and templates to increase language complexity.

C.2 Incomplete Information

  • Age, money, and friends cases require numerical comparisons, including unit conversion for age, summation across players for money, and counts that may be split by adjective for friends.The generated facts and rules vary whether the target comparison is more than or less than.Source: §C.2 · pp. 16–18
  • Affordance, color, place, and job cases require models to connect facts with rule conditions using knowledge of object properties, color groups, city-country relations, or job-industry relations.Affordance properties and item lists are manually written; colors are grouped by properties, and place and job mappings are manually written.Source: §C.2 · pp. 17–18
  • Textual-entailment cases require recognizing that one manually written sentence implies another, while name cases use whether two players’ names start with the same letter.Examples of entailment pairs include “assassinated the mayor” and “killed the mayor.”Source: §C.2 · pp. 17–18
  • Volume and event cases require background knowledge about whether objects fit in boxes and when major world events occurred to connect facts with rule conditions.Volume cases specify object dimensions and box dimensions; event cases use movie years and rules about release before or after an event.Source: §C.2 · p. 18
  • Extra-knowledge and reasoning cases are added only at the last theory-generation step, d = 1, to avoid subquestions that could produce unnatural rules.The example of an unnatural rule is “If ... then the dog is named Paco.”Source: §C.2 · pp. 18–19

C.3 Entities, Predicates, and Templates

  • The test set uses different entities, predicates, and templates from those used elsewhere to make the language more complex.Table 3 presents the entity, predicate, and template sets used in BoardgameQA.Source: §C.3 · p. 19

D Limitations and Negative Societal Impact

BoardgameQA currently focuses on deductive entailment with binary contradictions, small prompt-contained theories, modus ponens, and preference-based conflict resolution. Future work could broaden these settings, while preference assignment requires care to avoid unfair treatment of information sources.

Limitations

  • The dataset focuses on classification of proved, disproved, or unknown answers with binary True/False contradictions, leaving non-classification questions and non-binary conflicts for future work.Examples of broader cases include identifying who a dog will attack and resolving whether it is in Canada or Australia.Source: D Limitations and Negative Societal… · p. 19
  • The work assumes facts and game rules fit in the prompt; future extensions could address prompt-length limits by retrieving relevant facts and rules.Source: D Limitations and Negative Societal… · p. 19
  • The work studies only deductive reasoning with modus ponens; future work could extend BoardgameQA to rules such as proof by contradiction and disjunction elimination [40].Source: D Limitations and Negative Societal… · p. 19
  • The paper studies preference-based conflict resolution as one practical approach, while other natural approaches remain for future work and preference assignment may require care to avoid unfair treatment of sources.Source: D Limitations and Negative Societal… · p. 20

Related papers

Loading 2306.07934v1…