PaperlayerView exact revision on arXiv

Source-linked AI summary

Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning

Antonia Creswell, Murray Shanahan, Irina Higgins

arXiv:2205.09712v1cs.AIcs.CL

TL;DR

LLMs often struggle to chain multiple logical steps despite handling single-step inference relatively well. The paper introduces Selection-Inference, which alternates constrained selection and inference, and reports large gains for a 7B model together with causal reasoning traces. The framework still requires further work on selection, halting, verification, and deployment evaluation.

  • Problem

    LLMs tend to perform poorly on multi-step logical reasoning, although logically valid reasoning is important for discovery and explainability.

  • Method

    Selection-Inference alternates selecting relevant facts with inferring a new intermediate fact, chaining these modular steps using pre-trained LLMs.

  • Results

    The 7B SI model achieves 58.75% generative accuracy versus 2.94% for the vanilla baseline and 41.32% for COT, outperforming 280B baselines.

  • Takeaways & Limitations

    SI improves reasoning accuracy while producing causal, interpretable traces that expose how answers were reached for human understanding, debugging, and critique.

  • Takeaways & Limitations

    Further work is needed to improve selection, address halting, add verifiers, and carefully evaluate the system before deployment.

Abstract

from arXiv · show

Large language models (LLMs) have been shown to be capable of impressive few-shot generalisation to new tasks. However, they still tend to perform poorly on multi-step logical reasoning problems. Here we carry out a comprehensive evaluation of LLMs on 50 tasks that probe different aspects of logical reasoning. We show that language models tend to perform fairly well at single step inference or entailment tasks, but struggle to chain together multiple reasoning steps to solve more complex problems. In light of this, we propose a Selection-Inference (SI) framework that exploits pre-trained LLMs as general processing modules, and alternates between selection and inference to generate a series of interpretable, casual reasoning steps leading to the final answer. We show that a 7B parameter LLM used within the SI framework in a 5-shot generalisation setting, with no fine-tuning, yields a performance improvement of over 100% compared to an equivalent vanilla baseline on a suite of 10 logical reasoning tasks. The same model in the same setting even outperforms a significantly larger 280B parameter baseline on the same suite of tasks. Moreover, answers produced by the SI framework are accompanied by a causal natural-language-based reasoning trace, which has important implications for the safety and trustworthiness of the system.

1. Introduction

LLMs often handle simple logical inference but struggle with multi-step reasoning, motivating Selection-Inference (SI), which decomposes reasoning into modular selection and inference stages.

  • SI selects relevant facts for one inference step, then uses those facts to generate a new intermediate fact.
  • SI produces causal reasoning traces whose steps depend on preceding steps, supporting human understanding, debugging, and critique.
  • LLMs perform well on simpler single-step logical inference but struggle with harder multi-step problems.

2. Related Work

Prior approaches bridge symbolic and neural reasoning through modularity or explicit generation, but many remain limited in generality, causal dependence, or computational scope.

  • Symbolic systems provide interpretable step-by-step reasoning but require hand-crafted knowledge and are hard to scale.
  • LLM reasoning approaches either predict answers implicitly, generate all reasoning in one pass, or produce steps sequentially.
  • Explicit reasoning improves accuracy, but one-pass traces can contain unrelated or incorrect steps despite correct answers.
  • Sequential reasoning is promising for causal traces, while Tafjord et al.'s approach is restricted to True/False entailment-style questions and can be computationally expensive.

3. How Well Do Large Language Models Reason?

Across 50 tasks, LLMs handle simple entailment and implication better than multi-step reasoning, while larger models improve performance but remain weak on logic compared with other language tasks.

  • LLMs were evaluated in a 5-shot setting on 50 tasks probing simple entailment, multi-step reasoning, and scaling behavior.
  • LLM performance declines on single-step inference and becomes weaker on complex multi-step tasks across bAbI, Proof Writer, 2WikiMultiHop, and StrategyQA.
  • LLMs perform well on simple entailment and implication, including tasks with negation and generative or multiple-choice scoring.
  • Larger Gopher models perform better, but the 280B model averages only 13.6% above chance across 38 multiple-choice logic tasks.
  • Scaling is significantly worse for logic-based tasks than for the evaluated non-logic language tasks.

4. The Selection-Inference (SI) Framework

SI iteratively decomposes natural-language reasoning into constrained selection and inference steps, adding each inferred fact to the context to support later reasoning.

  • Framework assumptions: SI assumes each question has a sufficient context, possibly containing distractors, and is definitively answerable from that context.
  • Iterative pipeline: Each reasoning step selects relevant context facts and infers a new fact, which is added to the context for subsequent steps.
  • Module roles: Selection sees the context and question, whereas inference sees only the selected facts and cannot access the question.
  • Implementation: The framework uses frozen, pre-trained Gopher LLMs with prompt engineering in a 5-shot generalisation setting.
  • Selection module: To prevent fabricated selections, the selection module scores context facts and chooses the fact with the highest log-likelihood.
  • Inference and halting: The first generated inference sentence becomes a new context fact, and reasoning currently halts after a fixed number of steps.

5. Experiments and Results

The SI framework was evaluated on ten logical reasoning tasks against vanilla and chain-of-thought baselines, while also producing causal traces designed for inspection and debugging. Its reported benefits include strong task performance, human-auditable reasoning, and error recovery.

  • Evaluation setup: The 7B SI model was evaluated on 10 of the 50 logical reasoning tasks selected to cover different reasoning abilities and answerable contexts.The evaluation included bAbI Tasks 1–3 and 15–16 among other tasks.
  • Quantitative results: 58.75% generative accuracy for the 7B SI model exceeded the same model’s vanilla 2.94% and COT 41.32% baselines.All reported differences were statistically significant at p < 0.01.
  • Quantitative results: The 7B SI model also exceeded the 280B model in generative evaluation, whose vanilla and COT accuracies were 31.19% and 44.03%.The comparison used like-for-like 5-shot generative evaluation.
  • Quantitative results: In multi-choice evaluation, the vanilla 7B model outperformed the 280B model, 57.31% vs 51.45%, while remaining below the 7B SI model.The 7B SI result was obtained in the harder generative setting.
  • Interpretability: The SI framework constrains inference to selected context facts, so the final answer depends on a causal natural-language reasoning trace that humans can audit.The inference module does not directly access the question or earlier reasoning steps.
  • Interpretability: SI traces can expose irrelevant or incorrect intermediate inferences, and the additive reasoning process can recover by making a useful later inference.The paper illustrates recovery after inferring that swans are often gray when the question concerns lions.

6. Fine-tuning Language Models for Selection and Inference

The paper further fine-tunes separate Selection and Inference LLMs for the SI framework. Fine-tuning substantially improves final accuracy, while selection remains harder for deeper problems and one-shot reasoning baselines can produce disordered traces.

  • Fine-tuned SI: Fine-tuning specializes separate LLMs for selecting relevant context sentences and performing single-step inference within SI.Selection outputs sentence labels to prevent fabricating facts, while Inference operates on the selected information.
  • Inference module: >99% test accuracy was reached by the fine-tuned Inference LLM after 300 steps with batch size 16.This aligns with the finding that pretrained LLMs already perform single-step entailment well.
  • Selection module: Selection accuracy exceeded 80% across most reasoning depths but declined for deeper problems because early selections require planning.The Selection LLM was trained for 4 × 10^4 steps with batch size 16 for 50 hours on a TPU.
  • Final accuracy: 78.95% final reasoning accuracy for fine-tuned SI exceeded prompt-engineered SI at 57.93% and the prompt-engineered 7B and 280B baselines.The reported generative baseline accuracies were 0.34%/15.73% for vanilla/COT 7B and 31.58%/21.12% for vanilla/COT 280B.
  • Reasoning traces: The fine-tuned SI model produced more accurate reasoning traces than a model fine-tuned to generate all reasoning steps at once.The comparison used Jaccard Similarity between predicted and ground-truth proof steps.
  • Reasoning traces: The one-pass baseline often placed correct steps in the wrong order, added nonminimal steps, or repeated steps.These trace errors were observed despite the model predicting most reasoning steps correctly.

7. Conclusion

The paper presents Selection-Inference as a modular recursive pipeline for improving natural-language logical reasoning while producing causal, interpretable traces. It reports that prompt-engineered SI outperforms vanilla and chain-of-thought baselines, including a much larger baseline, and can improve further with fine-tuning.

  • 7. Conclusion: Selection-Inference decomposes reasoning into a modular recursive pipeline that improves reasoning accuracy and produces causal, interpretable traces.The framework draws on best practices from neurosymbolic approaches.
  • 7. Conclusion: Prompt-engineered SI significantly outperforms vanilla and chain-of-thought baselines, including 40x larger baselines, in equivalent settings.
  • 7. Conclusion: Fine-tuning can further improve SI performance when step-by-step reasoning data is available.
  • 7. Conclusion: Further work should improve selection, add halting and verification mechanisms, source relevant context, and handle ambiguous or unanswerable questions.The authors also emphasize careful evaluation before deployment in high-stakes domains.

Supplementary Information

The supplementary material describes SI demonstrations, selection and inference prompts, scoring selection, and multi-step use on bAbI reasoning tasks. It also records a limitation: the system is run for a fixed single step in some experiments without a mechanism to halt when the answer is reached.

  • C.2. Proof Writer: The selection prompt demonstrates intermediate and final reasoning steps for depth-2 problems, including cases where the first conclusion does not directly answer the question.
  • C.2. Proof Writer: The scoring selection procedure is presented as Algorithm 2.
  • D.3. bAbI: Some SI experiments run only one reasoning step, although additional steps may improve correctness without a halting mechanism.
  • D.3. bAbI: For bAbI 16, SI uses two steps: infer a rule from facts, then apply that rule to the relevant fact.This exposes which facts contributed to forming the new rule.

D.4. Proof Writer

The Proof Writer experiments use an Open World Assumption subset without Unknown examples and evaluate SI across tasks requiring different reasoning depths. The section explains how intermediate conclusions are generated and reports that SI’s advantage varies with depth and selection quality.

  • D.4. Proof Writer: The experiments use Proof Writer’s Open World Assumption data and remove Unknown examples so traces terminate in True or False conclusions.The Close World Assumption instead treats unproven statements as False, while OWA includes Unknown.
  • D.4. Proof Writer: SI is evaluated on five Proof Writer tasks requiring 1, 2, 3, or 5 reasoning steps.These tasks require intermediate conclusions that may not directly answer the final question.
  • D.4. Proof Writer: Proof Writer labels a statement True when SI derives the statement or its negation matches the target; otherwise it labels the statement False.
  • D.4. Proof Writer: SI outperforms baselines at depth zero and one, but its advantage diminishes with increasing depth partly because prompt engineering does not yield an optimal selection module.
  • D.4. Proof Writer: Additional SI steps improve performance more for depth-1 reasoning than depth-2 reasoning because depth-2 selections show insufficient variation.

E. Reasoning Traces Output by SI

The supplementary examples show SI selecting limited facts and using them to generate intermediate inferences across deductive and inductive tasks. They include both successful one-step traces and recovery from an initially unhelpful inference.

  • bAbI 15: In a bAbI 15 example, SI selects that Emily is a cat and infers that Emily is afraid of wolves.
  • bAbI 15: The traces similarly infer that Winona is afraid of mice, Gertrude is afraid of mice, and Jessica is afraid of cats from their categories.
  • bAbI 16: For bAbI 16, SI selects Julius’s rhino status and Greg’s green color to infer that Julius is green.
  • bAbI 16: In another bAbI 16 example, SI first infers an irrelevant rule about swans, then recovers by inferring that lions are often white and solves the question about Greg.

F. Fine-tuning Selection-Inference Details and Extra Results

Fine-tuned Selection and Inference modules outperform prompt-engineered and vanilla baselines, while example traces illustrate multi-step reasoning and interpretable failures.

  • Selection implementation: The Selection module predicts sentence labels or generates selection strings substituted from context, preventing it from inventing facts.Label-based selection forces the model to use contextual information.
  • Fine-tuning results: Fine-tuned Selection-Inference models significantly outperform prompt-engineered SI models and a vanilla prompt-engineered model predicting the final answer directly.The comparison is reported in Fig. 4b.
  • Depth-2 traces: Depth-2 examples show SI deriving intermediate facts sequentially from selected rules and known facts to answer the final query.One trace derives that the dog is nice before inferring that the dog eats the dog.
  • Reasoning traces: The supplementary examples include depth-3 and depth-5 reasoning traces produced with modules fine-tuned on Proof Writer.The depth-5 trace is presented as an interpretable output that exposes how the model reached its answer.
  • Trace failures: A model can repeat an incorrect selection yet still expose its failure through a causal trace, and additional steps may allow recovery.The example specifically reports failure to show that the lion visits the mouse.

G. Limitation Details

The proof-of-concept SI model has limitations involving selection reliability, prompt engineering, knowledge access, and control over reasoning duration and trace filtering.

  • Scope: The model is presented as a proof of concept with several limitations discussed by the authors.This frames the reported system as an initial demonstration rather than a complete solution.
  • Selection reliability: The Selection module is the model’s main observed failure point, but this is difficult to quantify because intermediate reasoning steps lack labelled data.The authors attribute this partly to using prompt engineering rather than fine-tuning.
  • Prompt engineering: Prompt engineering struggles with multi-step reasoning because intermediate-step patterns differ from final-step patterns and vary in argument structure.Fine-tuning for specific tasks can significantly improve current results.
  • Knowledge access: The approach assumes that a database of facts or rules is already given, whereas practical settings may require retrieving relevant knowledge from an existing knowledge base.The authors identify combining SI with retrieval approaches as future work.
  • Inference control: Although performance scales with compute time, the model lacks a reliable way to decide when to halt reasoning or filter traces and currently uses a fixed budget.The authors report that filtering reasoning steps could improve accuracy.

H. Baseline Datasets

The evaluation combines datasets spanning formal deduction, multi-hop reasoning, entailment, vague-language inference, science questions, and less-structured knowledge-based question answering, with several methodological baselines and checks.

  • Dataset sources: The study uses six publicly available sources: bAbi, BigBench, AAC, Jeopardy, Proof Writer, and 2WikiMultiHop.The authors state that dataset use followed the respective license permissions.
  • Task coverage: The 50 tasks probe formal deduction, sequenced multi-hop reasoning, implication and entailment, vague-language inference, relevance and sufficiency, and grade-school science reasoning.The task groups include Formal Fallacies, Sequence Problems, BigBench entailment tasks, Information Essentiality, Physics Questions, and Unit Interpretation.
  • Evaluation checks: Unnormalised multiple-choice scores are reported because length normalisation produced only a minimal average difference: 68.3±46.53% versus 67.92±46.68%.The difference was statistically significant with p=0.0002.
  • Evaluation checks: Dataset-bias checks found a small average deviation of 0.08 ± 8.74% from expected random performance, supporting use of the expected random baseline.The reported test result was p=5e−16.
  • Evaluation checks: Appending choices to bAbI reduced average accuracy from 44.86 ± 49.74% without choices to 37.86 ± 48.5% with choices.The reported difference had p=2e−61.
  • Baseline findings: LLMs worsen as reasoning requires more steps, struggle with yes/no questions, negation, and insufficient information, and are near chance on several formal tasks.They remain reasonable at simple implication, entailment, and induction, while structured AAC settings perform well.
  • Baseline findings: In less formal multi-hop question answering, models are close to chance without context or with unstructured context but perform better with minimal structured evidence.Jeopardy is identified as an outlier among the no-context settings.
Loading 2205.09712v1…