Source-linked AI summary
BoardgameQA: A Dataset for Natural Language Reasoning with Contradictory Information
Mehran Kazemi, Quan Yuan, Deepti Bhatia, Najoung Kim, Xin Xu, Vaiva Imbrasaite, Deepak Ramachandran
TL;DR
Automated reasoning often assumes consistent knowledge, although real-world sources can conflict and require preference-guided resolution. The paper introduces BoardgameQA, a dataset for defeasible multi-hop reasoning with implicit background knowledge, and finds that language models reason poorly with conflicting inputs, with finetuning helping but performance remaining poor.
Problem
Existing natural-language reasoning evaluations usually assume consistent, reliable knowledge, despite real-world reasoning often involving contradictory sources and incomplete information.
Method
The paper introduces BoardgameQA, a synthetic board-game dataset that evaluates defeasible multi-hop reasoning with contradictory rules, source preferences, and partially implicit knowledge.
Results
LMs perform poorly when reasoning with conflicting inputs, especially in few-shot settings, while finetuning improves performance but does not eliminate the difficulty.
Takeaways & Limitations
Conflict resolution and defeasible reasoning do not reliably emerge in pretrained LMs, revealing a gap in reasoning capacity relevant to applications with contradictory or incomplete information.
Takeaways & Limitations
The framework assumes initial facts are internally consistent and that preferences can resolve every conflict in the theory.
Abstract
from arXiv · showhide
Automated reasoning with unstructured natural text is a key requirement for many potential applications of NLP and for developing robust AI systems. Recently, Language Models (LMs) have demonstrated complex reasoning capacities even without any finetuning. However, existing evaluation for automated reasoning assumes access to a consistent and coherent set of information over which models reason. When reasoning in the real-world, the available information is frequently inconsistent or contradictory, and therefore models need to be equipped with a strategy to resolve such conflicts when they arise. One widely-applicable way of resolving conflicts is to impose preferences over information sources (e.g., based on source credibility or information recency) and adopt the source with higher preference. In this paper, we formulate the problem of reasoning with contradictory information guided by preferences over sources as the classical problem of defeasible reasoning, and develop a dataset called BoardgameQA for measuring the reasoning capacity of LMs in this setting. BoardgameQA also incorporates reasoning with implicit background knowledge, to better reflect reasoning problems in downstream applications. We benchmark various LMs on BoardgameQA and the results reveal a significant gap in the reasoning capacity of state-of-the-art LMs on this problem, showing that reasoning with conflicting information does not surface out-of-the-box in LMs. While performance can be improved with finetuning, it nevertheless remains poor.
1 Introduction
BoardgameQA addresses natural-language reasoning when information is contradictory, preferences determine which sources prevail, and some required knowledge is implicit. The dataset benchmarks language models on defeasible, multi-hop reasoning and finds substantial weaknesses, especially for conflicting inputs.
- Existing natural-language reasoning benchmarks usually assume consistent, reliable knowledge, unlike real applications involving contradictory sources.Conflicts may arise across online sources or social media, with preferences based on credibility or recency.
- BoardgameQA formulates reasoning with preference-guided conflicting information as defeasible reasoning and generates theories with facts, contradictory rules, and rule preferences.Examples require multi-hop reasoning and conflict resolution, with automatically controllable difficulty.
- BoardgameQA withholds part of the knowledge required for answering so models must supply implicit background knowledge themselves.The board-game setting supports synthetic defeasible rules that can be stated naturally while invoking commonsense world knowledge.
- The dataset uses board-game scenarios to create natural-text examples with complex defeasible rules and, according to the authors, is the first dataset for multi-hop reasoning with contradictory inputs.Figure 2 highlights both conflict resolution and missing knowledge in a sample.
- LMs perform poorly on conflicting sources, particularly in few-shot settings, while smaller models also struggle when required information is missing.Finetuning improves performance but does not eliminate the gap in defeasible reasoning capacity.
2 Related Work
Prior work covers text-based logical reasoning, conflict handling, and incomplete-information benchmarks, but the compared logical-reasoning datasets generally use non-contradictory facts and rules. BoardgameQA combines these dimensions by targeting reasoning with conflicting and partially missing information.
- Text-based logical reasoning approaches: Prior text-based reasoning methods include direct answer generation, explicit proof generation, modular reasoning, finetuning, and few-shot approaches.The paper experiments with finetuning with or without explicit reasoning steps and prompt-tuning with chain-of-thought prompting.
- Text-based logical reasoning datasets: Existing text-based logical-reasoning datasets measure natural-language reasoning but generally contain only non-contradictory facts and rules.ConditionalQA is identified as a nearby dataset with answers conditioned on alternative cases.
- Reasoning with conflicts: Reasoning with conflicting information is studied as defeasible reasoning, with applications including legal reasoning and longstanding AI approaches for resolving conflicts.The paper positions this problem as an important direction in LM reasoning research.
- Reasoning with incomplete information: Incomplete-information benchmarks require models to supply missing knowledge, and BoardgameQA focuses on cases combining provided information with knowledge that must come from the model.Some prior datasets omit part or all of the rules needed for reasoning.
3 Background and Notation
The paper represents defeasible theories as facts, rules, and priorities that resolve conflicting derivations, extending ordinary monotonic reasoning with defeasible conclusions.
- Defeasible Theory: A defeasible theory consists of facts, rules, and pair-wise priorities that determine which conflicting rule conclusion is accepted.Rules may be defeated by contrary evidence from a higher-priority rule.
- Dataset Comparison: Table 1 compares BoardgameQA with existing logical-reasoning datasets using contradictory information, incomplete information, and automatic difficulty control as key features.The comparison includes bAbI 15, CLUTRR, FOLIO, ProofWriter, PrOntoQA-OOD, AR-LSAT, ENWN, leap-of-thought, and ConditionalQA.
- Conflict Resolution: Unlike monotonic theories, defeasible theories resolve conflicts by accepting the conclusion supported by the higher-priority rule.Initial facts are internally consistent and take priority over derived facts.
- Conflict Resolution: In the penguin example, bird knowledge derives that Tweety flies, but the higher-priority penguin rule yields the accepted conclusion that Tweety does not fly.The priority relation is r3 > r2.
- Conflict Types: Conflicts arise when rules derive incompatible conclusions, and resolution may depend on whether the competing rule is activated or blocked by its premises.The paper distinguishes two conflict types according to these activation conditions.
4 The BoardgameQA Dataset
BoardgameQA generates natural-language questions from synthetic defeasible theories with controllable reasoning difficulty, conflicts, and incomplete background knowledge.
- Task Formulation: Each example pairs a defeasible theory with a question whose label is proved, disproved, or unknown.The task requires determining whether the theory derives the question, its negation, or neither.
- Dataset Design: BoardgameQA represents game states as facts, game rules as logical rules, and questions about the game, with variables controlling depth, conflict types, and other difficulty properties.Entities and predicates are sampled from predefined sets, with different entities and predicates used across training and test.
- Theory Generation: Theory generation recursively constructs rules and sub-questions from a target question, adding one proof hop per recursive call until the requested depth is reached.The generator tracks facts, rules, preferences, and proofs while using depth as its stopping criterion.
- Conflict Generation: Conflicts are introduced probabilistically by generating a rule for the negated question and selecting Type1 or Type2 preferences that control which rule can activate.Type1 adds recursive calls for all or some competing premises, whereas Type2 omits one premise so the lower-priority rule does not activate.
- Incomplete Information: The dataset hides selected intermediate knowledge so models must supply world, commonsense, mathematical, or orthographic reasoning in addition to multi-hop deduction.Some categories are excluded from training so models cannot simply memorize the missing knowledge for test examples.
- Natural-Language Conversion: Generated facts, rules, preferences, and questions are converted into natural language with multiple templates, including templates reserved for testing.Unknown examples are created by perturbing theories until the queried statement is no longer derivable.
5 Experiments
Experiments evaluate language models on BoardgameQA across reasoning depth, proof validity, conflict types, incomplete information, and distractors. Models struggle particularly with contradictory inputs, higher reasoning depths, conflict resolution, and missing knowledge, though tuning improves several capabilities.
- 5.1 Can LMs Reason with Contradictory Inputs?: Tuned models perform reasonably at depth 1 but degrade substantially at depths 2–3, while few-shot models perform poorly across all depths.PaLM 540B outperforms PaLM 62B, suggesting larger models may have greater defeasible-reasoning capacity.
- 5.2 Does Correct Label Prediction Mean Correct Proof?: Few-shot models perform poorly on conflict resolution, whereas tuned models perform substantially better, indicating that preference understanding does not emerge reliably from simple prompting.Models often generate incorrect proofs even when predicting the label correctly; this problem is less severe for prompt-tuned models.
- 5.4 Which Conflict Type is More Difficult to Resolve?: Models perform slightly better with mostly Type1 than mostly Type2 conflicts because Type2 reasoning requires proving that at least one conflicting rule-body element cannot be established.For tuned models, concentrating on one conflict type also improves overall performance, possibly because learning one type is easier than learning both.
- 5.5 Does Information Incompleteness Make Reasoning More Difficult?: As required knowledge increases, finetuned-model performance decreases, whereas prompt-tuned and few-shot performance remains relatively unchanged.The evaluated incompleteness settings use pMissInfo values of 0.2, 0.5, and 0.8, named KnowledgeLight, KnowledgeMedium, and KnowledgeHeavy.
- 5.6 Do Distractors Make Reasoning More Difficult?: Tuned models do not substantially degrade with a small number of distractors but drop substantially with more, while few-shot performance decreases monotonically but only marginally.Distractors are facts or rules absent from the proof that do not change the label; the experiments add 0, 1, or 2 per step.
6 Conclusion
The paper introduces BoardgameQA to measure language-model reasoning with conflicting inputs and partially provided knowledge. Across dataset variations, models perform poorly on contradictory reasoning, and smaller models also struggle when external knowledge must be supplied by the model.
- 6 Conclusion: BoardgameQA measures natural-language reasoning with conflicting input sources and cases where required knowledge is only partially provided.The dataset evaluates several language-model types across variations involving conflict and incomplete information.
- 6 Conclusion: Models perform poorly when reasoning with conflicting inputs, while smaller models also perform poorly when additional knowledge from the model is required.The authors identify these results as an important gap because contradictory and incomplete information is common in real-world applications.
A More Experimental Results and Analysis
Additional analyses show that models struggle with both unknown-label prediction and the underlying defeasible reasoning process. Their errors include mishandling conflicts, incomplete information, logical rules, conjunctions, distracting paths, and unsupported proofs.
- Binary Classification: The authors created binary versions at depths 1–3 to test whether poor performance reflects unknown labels or difficulty resolving conflicts.These versions include only examples with proved and disproved labels.
- Examples of Model Failures: Model failures include misunderstanding preferences and rules, mishandling incomplete information or conjunctions, pursuing distracting proof paths, and altering facts when no proof exists.These error categories are illustrated through wrong proofs in Figures 12–21.
- Confusion Matrices: Higher-depth examples cause proof-tuned models to predict proved or disproved labels for unknown cases far more often.For depth-1 unknown examples, the prompt-tuned model predicts unknown 268 times, compared with only 2 times at depth 2.
B Experimental Details
The experiments use different TPU configurations and few-shot demonstration constraints, while supplementary analyses visualize predictions and illustrate characteristic proof errors. Dataset examples also include synthetic facts such as locations, measurements, and negated relations.
- Experimental Setup: All models ran on v3 TPUs except 540B models, which used v4 TPUs, with experiments implemented in T5X.The framework is identified as T5X [36].
- Few-shot Experiments: Few-shot prompts contain one randomly selected training example per label, while ensuring demonstrations include both conflict-resolution types.Three-way tasks therefore use one proved, one disproved, and one unknown example.
- Diagnostic Analyses: Figures 11–13 provide confusion matrices and wrong-proof examples that expose prediction patterns and logical-rule errors.Figure 11 covers model predictions, while Figures 12 and 13 show errors by PaLM models.
- Training Details: For BERT, training uses batch size 16, learning rate 4.6e−5, and 20 epochs, with results reported from the best validation epoch.These settings apply to BERT experiments.
- Dataset Examples: The dataset includes synthetic scenario facts such as a seahorse’s 27-inch basketball in Istanbul and a wolf’s negated relation to the snake’s emperor.These facts precede the game rules in the example.
C BoardgameQA Details
BoardgameQA’s dataset-generation details are illustrated through a sample theory and question produced by Algorithm 16.
- Dataset Generation: Algorithm 16 generates sample theories and questions for the BoardgameQA dataset.The sample is presented in Figure 22.
- Dataset Properties: The dataset-details section examines how BoardgameQA examples are generated and what properties they have.It introduces the section as a deeper account of generation and dataset properties.
- Illustrative Example: Figure 22 provides a concrete example of the theory-and-question generation process.The figure is specifically tied to Algorithm 16.
C.1 Consistency of the Dataset
BoardgameQA constructs consistent defeasible theories by ensuring that preferences can resolve conflicts and by separating entities across recursive rule branches.
- Consistency Definition: A defeasible theory is consistent when preferences can resolve conflicts whenever they arise.BoardgameQA aims to generate theories with this property.
- Generation Constraints: Algorithm 16 avoids inconsistencies and loops by sampling only entities unused in other rules and subquestions.This restriction helps keep recursive constructions separated.
- Branch Separation: For conjunctive rules, separate entities are used in the recursive branches so derivations in one branch cannot defeat those in another.Defeasible reasoning is also applied to the final theory to ensure the question is derivable.
C.2 Incomplete Information
BoardgameQA represents incomplete information through diverse background-knowledge cases that require models to connect textual facts with rules using commonsense, numerical, temporal, spatial, and semantic understanding.
- C.2 Incomplete Information: Incomplete-information cases require models to supply background knowledge linking facts to rule bodies across jobs, affordances, colors, money, places, textual entailment, names, and related domains.Examples include mapping jobs to industries, objects to properties, cities to countries, and one sentence to another by entailment.
- C.2 Incomplete Information: Numerical and physical cases test whether models can compare friend counts, convert ages across units, and determine whether objects fit inside boxes.Volume reasoning uses object dimensions and box dimensions, while age and friend cases require quantitative comparisons.
- C.2 Incomplete Information: Temporal cases require connecting movie-release years with manually specified world-event dates.The rule may depend on whether a movie was released before or after an event.
- C.2 Incomplete Information: The dataset adds incomplete-information cases only at the final theory-generation step, avoiding unnatural rules that would arise from recursively generating subquestions.This restriction applies when d = 1.
- C.2 Incomplete Information: BoardgameQA varies entities, predicates, and templates, including test-only language templates, to increase linguistic challenge beyond the underlying reasoning requirements.The dataset also uses multiple templates per rule type.
D Limitations and Negative Societal Impact
The paper identifies limits in BoardgameQA’s reasoning scope, prompt-size assumptions, and conflict-resolution coverage, while noting that preference assignment can create fairness concerns.
- D Limitations and Negative Societal Impact: BoardgameQA primarily evaluates deductive classification with binary contradictions, leaving non-classification questions and non-binary conflicts for future work.Examples of excluded settings include identifying who will be attacked and resolving incompatible locations such as Canada versus Australia.
- D Limitations and Negative Societal Impact: The current setup assumes all initial facts and game rules fit in the prompt, so retrieval is not evaluated when prompt length prevents including relevant information.The paper proposes extending the dataset and analysis to retrieval-based settings.
- D Limitations and Negative Societal Impact: The analysis is limited to deductive reasoning with modus ponens rather than other rule forms such as proof by contradiction or disjunction elimination.Those rule types are identified as directions for extending BoardgameQA.
- D Limitations and Negative Societal Impact: Conflict resolution is studied through one preference-based solution, and assigning source preferences requires care to avoid unfair treatment of information sources.The paper leaves other natural conflict-resolution strategies for future work.