Source-linked AI summary
Transformers as Soft Reasoners over Language
Peter Clark, Oyvind Tafjord, Kyle Richardson
TL;DR
The paper addresses whether transformers can reason over explicit rules without first converting language into formal representations. It trains transformers on synthetically generated linguistic theories and finds strong accuracy, deeper-chaining generalization, and transfer to authored and paraphrased rulebases. The authors frame this as limited soft theorem proving while acknowledging restricted theory sizes and rule expressiveness.
Problem
Formal representations have made explicit-knowledge reasoning difficult, motivating the study of reasoning over rules expressed directly in language.
Method
The paper generates logical theories and linguistic equivalents, then trains transformers to emulate deductive reasoning over facts, rules, and questions in language.
Results
The models achieve 99% accuracy, generalize to deeper chaining, transfer zero-shot to hand-authored rulebases, and score 98% on paraphrased theories.
Takeaways & Limitations
Transformers can serve as limited soft theorem provers over explicit linguistic theories, with potential benefits for explanation, correctability, and counterfactual reasoning.
Takeaways & Limitations
The demonstrations use small theories, small domains, and a limited rule language with at most one universally quantified variable.
Abstract
from arXiv · showhide
Beginning with McCarthy's Advice Taker (1959), AI has pursued the goal of providing a system with explicit, general knowledge and having the system reason over that knowledge. However, expressing the knowledge in a formal (logical or probabilistic) representation has been a major obstacle to this research. This paper investigates a modern approach to this problem where the facts and rules are provided as natural language sentences, thus bypassing a formal representation. We train transformers to reason (or emulate reasoning) over these sentences using synthetically generated data. Our models, that we call RuleTakers, provide the first empirical demonstration that this kind of soft reasoning over language is learnable, can achieve high (99%) accuracy, and generalizes to test data requiring substantially deeper chaining than seen during training (95%+ scores). We also demonstrate that the models transfer well to two hand-authored rulebases, and to rulebases paraphrased into more natural language. These findings are significant as it suggests a new role for transformers, namely as limited "soft theorem provers" operating over explicit theories in language. This in turn suggests new possibilities for explainability, correctability, and counterfactual reasoning in question-answering.
1 Introduction
The paper asks whether transformers can reason over explicitly stated rules expressed in language, avoiding formal knowledge representations. RuleTakers achieve strong accuracy, transfer across rulebases and paraphrases, and suggest a limited soft-theorem-proving role.
- Formal knowledge representation has enabled AI reasoning but remains challenging to build and reason over.
- RuleTakers train transformers to emulate reasoning over facts and rules expressed as language rather than formal representations.
- The setting uses conjunctive implications with negation under logic-program semantics and is narrower than natural language inference.
- 99% accuracy is achieved on synthetic-language rules, with up to 95% on questions requiring deeper reasoning than seen during training.
- 90%+ scores are obtained zero-shot on five of six variants of two independently authored rule-based problems.
- 98% performance is achieved on paraphrased theories, while fact-dependence identification reaches 94% F1 and is perfect for over 70% of questions.
2 Related Work
Related work includes neural systems applying explicit rules, reasoning over synthetic or naturalistic text, and neural theorem proving. This paper differs by testing whether transformers can learn the input-output behavior of systematic rule reasoning directly in language.
- Earlier datasets test single-rule application or apply general knowledge to situations, including bAbI Task 15, conditional probes, QuaRTz, and ROPES.
- Unlike many bAbI and multihop reasoning tasks, this work states the rules of inference explicitly instead of requiring models to induce them from examples.
- Prior approaches often semantically parse language into formal structures before reasoning, whereas this work reasons with language directly.
- Neural theorem-proving methods combine symbolic and neural components for stepwise reasoning, while this paper imposes no structure on the neural reasoning process.
- The task is framed as algorithm emulation for systematic rule reasoning and is related to Natural Logic and NLI, which permits broader person-typical inferences.
3 Dataset Generation
The dataset generator creates finite logical theories, derives their implications, and converts facts, rules, and proof questions into synthetic English. Datasets vary by the maximum inference depth required for answering questions.
- Each example contains a context of facts and rules, a declarative statement, and a T/F answer based on deductive entailment under a closed-world assumption.
- The generator creates a logic theory, performs forward inference, and samples true questions from implications and false questions from unproven facts.
- Five datasets target inference depths 0, up to 1, up to 2, up to 3, and up to 5, with DMax used to test deeper generalization.
- Theories represent unary attributes and binary relations, with optional negation and randomly generated facts, entities, attributes, and rules.
- Rules are constrained to at most one universally quantified variable, with later conditions and conclusions referring to that variable or previously mentioned entities.
- Inference uses finite, stratified logic programs with negation as failure; inconsistent and unstratified rulebases are discarded.
- The resulting theories and questions are rendered in synthetic English using simple templates and fluency-improving rules.
4 Experiments
Experiments show that RuleTaker models generalize beyond synthetic training depths, transfer to hand-authored and paraphrased rulebases, and can identify proof-critical sentences, while retaining blind spots.
- Synthetic reasoning: 99% accuracy is achieved on distinct synthetic test problems, with up to 97.6% on depth-5 problems after training through depth 3.Deeper training improves generalization, but higher-depth models require more training data.
- Perturbation tests: 81% of expected true-to-false prediction flips occur when a critical sentence is removed, indicating moderate robustness to theory perturbations.Overall accuracy remains largely unchanged across perturbed theories, but performance drops when critical sentences are removed.
- Hand-authored problems: The models solve five of six variants of two independently authored rule-based problems zero shot, generally scoring above 90%.The birds problems are solved almost perfectly by all but the non-reasoning model, while Electricity4 exposes an anomaly.
- Paraphrased rules: 98.8% accuracy is achieved on paraphrased rulebases when trained on paraphrased data, while the strongest synthetic-only model reaches 66.6% zero shot.The paraphrased test rewordings are distinct from training and development rewordings.
- Explanations and architectures: Critical-sentence identification reaches P=98.7, R=86.9, and F1=92.4, with perfect identification for over 70% of questions.The model identifies dependencies between context sentences and conclusions, though examples include missed critical sentences and irrelevant inclusions.
- Explanations and architectures: BERT reaches 95%+ accuracy on higher-depth DMax problems, whereas ESIM scores approximately 80%, suggesting transformers learn these tasks more easily.The comparison also notes that ESIM was not pretrained on large amounts of text.
5 Discussion and Future Work
The paper identifies promising applications for language-based deductive reasoning while emphasizing that current demonstrations remain narrow and leave major generalization challenges. Future work must expand theory expressiveness, scale, consistency handling, and the gap between rigorous deduction and natural-language inference.
- Implications: RuleTaker suggests using English-authored theories for explainable problem solving, correction of model knowledge, and counterfactual reasoning.These possibilities include retrieving general knowledge from sources such as Wikipedia and modifying theories to model alternative worlds.
- Scope boundaries: The demonstrated capabilities remain limited to small theories, small domains, and rules with at most one universally quantified variable.The authors cite fewer than 20 facts, fewer than 10 rules, fewer than 100 possible ground facts, and no explored multi-variable rules.
- Future theory language: Extending the rule language requires handling transitivity, taxonomic inheritance, disjunctive conclusions, and functional relations.Such extensions require formalizing their semantics and generating training data containing valid conclusions.
- Future theory language: Natural-language statements with less precise formal meanings require richer synthetic or human-collected training data.Examples include claims such as “Most birds fly” and “It often rains in Seattle in winter.”
- Scaling and robustness: Larger theories can exceed the transformer’s ≤512-token limit, requiring new architectures or methods for retrieving relevant rules.The authors also leave inconsistent theories as an avenue for further work.
- Natural-language inference: The approach uses precise deductive semantics, whereas natural-language inference permits unsupported but typical human inferences.The authors specifically note tensions involving knowledge gaps and negation as failure.
6 Conclusion
The paper concludes that transformers can be trained to emulate deductive reasoning over language using a general pipeline from formal behavior to synthetic linguistic examples. Within the experiments, this behavior appears robust and supports potential applications in authoring, explanation, correction, and counterfactual reasoning.
- Conclusion: The methodology characterizes desired behavior formally, synthesizes examples, generates linguistic equivalents, and trains a transformer.The conclusion presents this procedure as broadly applicable beyond the particular semantics used here.
- Conclusion: The resulting behavior appears natural and robust while remaining distinct from reasoning over the original formalization.This is the authors’ within-experiment characterization of the result.
- Implications: The framework enables English-authored theories, explainable solutions using general knowledge, correction of erroneous knowledge, and neural counterfactual reasoning.The paper gives examples involving Wikipedia knowledge, instructable systems, and altered rules about birds or plastic.
- Theory representation: The theory grammar represents facts and rules using entities, attributes, relations, polarity, and implication structure.Facts may describe attributes or binary relations, while rules map one or more facts to a conclusion.
- Theory representation: The generated vocabulary draws names, attributes, and relations from specified pools, with polarity marking whether facts are negated.Variable names are also varied through simple synonyms during language generation.
A.1 Names, Attributes, and Relations
The datasets use two theory types with distinct entity and predicate vocabularies, sampled from controlled pools. Theory size and rule-generation structure are randomized to create varied synthetic reasoning problems.
- Theory types: Type1 theories use only attributes with people as entities, whereas Type2 theories combine attributes and relations with animals as entities.Both types can be generated with negation in facts and rule conditions or conclusions.
- Vocabulary pools: Type1 names are drawn from a pool including Anne, Bob, Charlie, Dave, Erin, Fiona, Gary, and Harry.The passage lists these as the Type1 attribute-theory name pool.
- Rule generation: Rule generation applies heuristics H1–H3 incrementally to construct linguistic rules such as conditions about loving the cat and the cat’s happiness.Figure A1 illustrates where these heuristics are applied during generation.
- Vocabulary pools: Attribute vocabularies include properties such as red, blue, green, kind, nice, big, cold, young, round, rough, white, smart, quiet, and furry.The listed pools differ somewhat between Type1 and Type2 theories.
- Vocabulary pools: Type2 relational theories draw relations such as likes, chases, eats, sees, visits, and needs from a controlled pool.Names and attributes are likewise sampled from specified pools.
- Generation controls: Each theory randomly selects 2–4 names, 1–5 attributes, 0 or 1–4 relations, 1–8 rules, and 1–16 facts.Relations are absent for Type1 and sampled for Type2.
A.3 Controlling Paths through the Grammar
The grammar uses non-uniform transition probabilities to control whether generated facts use simple or relational forms and whether theories include negation.
- Fact generation selects “is” for Type1 (“Att”) theories, but chooses “is” with p=0.3 and a relation with p=0.7 for Type2 (“Rel”) theories.
- Polarity generation always selects “+” for theories without negation, while negation-enabled theories select “+” probabilistically.The supplied passage gives the probability notation for the negation-enabled case but does not include its value.
A.4 Special Heuristics for Rule Generation
Rule generation applies heuristics that favor realistic shared arguments, grounded conclusions, and fluent rules, while limiting difficult multi-variable constructions.
- The heuristics encourage shared arguments, occasionally fully grounded rules, variable-free conclusions, and avoidance of multi-variable rules.Multi-variable rules were avoided because rendering them fluently in English was challenging, though extending generation to them is identified as valuable.
- Rule construction samples one to two conditions for each rule.
- Entity-selection heuristics govern the first entity, relational second entities, and later entity choices during rule construction.The supplied passages specify that relational second entities are always names and that subsequent first-entity choices can reuse names or variables.
- An example generation sequence illustrates how these heuristics are applied.
A.5 Ensuring Reasoning Depth
The generator enforces target reasoning depth by repeatedly testing generated theories, while also using hand-authored rulebases as evaluation materials.
- Target reasoning depth is enforced by generating a theory, deriving all implications, and discarding it unless its deepest reasoning depth reaches D.
- The birds rulebase provides hand-authored rules involving birds, ostriches, abnormality, wounding, flying, and negation.
- The evaluation materials include four electricity rulebases and their scenario vocabularies.
C.1 Electricity1
The Electricity1 materials define rulebases and probabilistic scenario generators that vary circuit components, power conditions, and downstream device outcomes.
- The first rulebase derives circuit completeness from switch conditions or switch absence, then derives glowing, ringing, or playing from completeness and device presence.
- Its scenarios probabilistically include a switch, switch-on status, and one selected device: a light bulb, bell, or radio.
- The second rulebase inserts an intermediate current-running step before deriving device outcomes.
- Scenario probabilities vary across rulebases, including battery, switch, switch-on, battery-flatness, and wire-material choices.
- The third rulebase distinguishes powered from dead circuits and uses completeness and power to derive current flow and device behavior.
- The fourth rulebase additionally models battery flatness, wire conductivity, current transfer to devices, and resulting device states.
D. Counterfactuals
Because RuleTakers reason over explicitly provided knowledge, they can update conclusions when the context is changed counterfactually. The model correctly revises answers across examples involving metals, insulators, and plastics.
- D. Counterfactuals: Explicitly provided knowledge enables transformers to perform counterfactual reasoning by conditioning conclusions on modified facts and rules.The paper contrasts this capability with reasoning from latent, fixed, pretrained knowledge.
- D. Counterfactuals: Counterfactual context changes correctly reverse or preserve conclusions about whether nails conduct electricity.Changing metal to an insulator makes the answer false; changing plastic to a metal makes it true.