Source-linked AI summary
Back to the Future: Unsupervised Backprop-based Decoding for Counterfactual and Abductive Commonsense Reasoning
Lianhui Qin, Vered Shwartz, Peter West, Chandra Bhagavatula, Jena Hwang, Ronan Le Bras, Antoine Bosselut, Yejin Choi
TL;DR
Abductive and counterfactual reasoning require generation conditioned on both past and future contexts, which is difficult for language models designed for left-to-right conditioning or narrow text infilling. DELOREAN uses unsupervised forward and backward decoding with off-the-shelf language models, and outperforms unsupervised methods on both tasks in automatic metrics and human evaluation.
Problem
Generative language models typically condition only on left context or perform narrowly scoped infilling, while abductive and counterfactual reasoning require conditioning on multiple past and future contexts.
Source: §1 · p. 1Method
DELOREAN uses unsupervised forward and backward decoding with off-the-shelf left-to-right language models, backpropagating through output representations while fixing model parameters.
Source: §1 · p. 2Results
On both tasks, DELOREAN outperforms all unsupervised methods in automatic metrics and human evaluation, and its outputs are judged more coherent than supervised models.
Source: §1 · p. 2Takeaways & Limitations
Abstract
from arXiv · showhide
Abductive and counterfactual reasoning, core abilities of everyday human cognition, require reasoning about what might have happened at time t, while conditioning on multiple contexts from the relative past and future. However, simultaneous incorporation of past and future contexts using generative language models (LMs) can be challenging, as they are trained either to condition only on the past context or to perform narrowly scoped text-infilling. In this paper, we propose DeLorean, a new unsupervised decoding algorithm that can flexibly incorporate both the past and future contexts using only off-the-shelf, left-to-right language models and no supervision. The key intuition of our algorithm is incorporating the future through back-propagation, during which, we only update the internal representation of the output while fixing the model parameters. By alternating between forward and backward propagation, DeLorean can decode the output representation that reflects both the left and right contexts. We demonstrate that our approach is general and applicable to two nonmonotonic reasoning tasks: abductive text generation and counterfactual story revision, where DeLorean outperforms a range of unsupervised and some supervised methods, based on automatic and human evaluation.
1 Introduction
Abductive and counterfactual reasoning require generating plausible events from incomplete or altered contexts, but language models typically condition only on past context or support narrowly scoped infilling. DELOREAN uses unsupervised forward and backward decoding to incorporate both sides, outperforming unsupervised methods on both tasks.
1 Introduction
- Abductive reasoning generates a hypothesis between observed past and future contexts, while counterfactual revision generates an ending coherent with an altered context and similar to the original ending.The paper illustrates these as distinct generation goals.Source: Figure 1 · p. 1
- Most generative language models condition only on left context or perform narrowly scoped text-infilling, limiting their direct fit for reasoning over past and future contexts.Supervised approaches still lag humans and can rely on superficial strategies or require large-scale task- and domain-specific data.Source: §1 · p. 2
- DELOREAN alternates left-to-right inference with right-to-left backpropagation through a task-specific loss, updating output representations while fixing model parameters.An unsupervised BERT next-sentence-prediction ranking step selects a coherent output.Source: §1 · p. 2
- DELOREAN outperforms all unsupervised methods on both tasks in automatic metrics and human evaluation, and its outputs are judged more coherent than supervised models.The authors describe conditional decoding as a promising research direction.Source: §1 · p. 2
2 Background
The background frames nonmonotonic reasoning as making plausible conclusions from incomplete or hypothetical observations, and reviews two language-generation tasks. Abductive generation explains observations; counterfactual story generation revises an ending after a changed condition.
2 Background
- Nonmonotonic reasoning allows plausible conclusions from incomplete or hypothetical observations, unlike reasoning restricted to information entailed by a premise.The paper studies abductive and counterfactual reasoning as examples.Source: §2 · pp. 2–3
2.1 Abductive Reasoning
- ART frames abduction as generating the most likely explanation between two observations; its 20k instances use the first and last sentences of ROCStories.Prior pretrained-LM baselines showed both promise and limitations.Source: §2.1 · p. 3
2.2 Counterfactual Reasoning
- TIMETRAVEL frames counterfactual generation as revising a three-sentence ending after the original story’s second sentence changes, balancing the alternative context with minimal edits.Its 29,849 instances are based on fictional ROCStories narratives; prior systems tended either to deviate from the original storyline or over-copy it.Source: §2.2 · p. 4
3 The DELOREAN Approach
DELOREAN iteratively refines candidate text with forward and backward passes, then ranks candidates by coherence. The method uses a soft representation of token logits so gradients can update the generation while the language model remains fixed.
3 The DELOREAN Approach
- DELOREAN interleaves forward and backward passes for a predefined number of iterations, using the forward pass for fluent continuation from context X and the backward pass to satisfy constraint Z.It samples candidate generations and returns the one ranked most coherent.Source: §3.1 · p. 4, §3 · p. 4
3.1 Decoding Strategy
- The decoding strategy maintains soft token-logit representations, updates them by backpropagating constraint loss, and mixes backward and forward logits before sampling text.The forward pass can extend beyond the predefined backward-pass length to complete sentences.Source: §3.1 · pp. 4–5
3.2 Ranking
- The ranking step uses BERT next-sentence prediction as a proxy for coherence across candidate continuations, their internal consistency, and compatibility with right-side text when applicable.The supplied table caption identifies Table 1 as automatic evaluation on the ART test set, but does not provide result values.Source: §3.2 · p. 5, Table 1 · p. 5
4 Task 1: Abductive Reasoning
Abductive generation seeks a hypothesis Y that explains observations X and Z jointly. DELOREAN is evaluated against unsupervised and supervised baselines using automatic and human evaluation.
4 Task 1: Abductive Reasoning
- The task maps two observations to left and right contexts X and Z, with the goal of generating a hypothesis Y that maximizes P(Y | X, Z).The hypothesis is intended to explain both observations.Source: §4 · p. 5
4.1 Task Setup
- For future-context constraints, the setup maximizes the likelihood of generating Z given X followed by the current hypothesis representation, using a left-to-right LM.The setup also prepends Z to X as a future-information leak, but comparisons indicate this has a minor effect on performance.Source: §4.1 · p. 5
4.2 Experimental Setup
- Baselines included GPT-2 zero-shot prompts and supervised GPT-2 fine-tuning; each baseline also generated 20 candidates for coherence ranking.The method used GPT2-345M, with hyperparameters selected on the ART development set.Source: §4.1 · p. 5, §4.2 · p. 6
4.3 Results
- DELOREAN performed best among unsupervised systems across BLEU-4, ROUGE-L, and BERTSCORE, while ranking improved its performance and that of zero-shot baselines.The ranking component was included to help isolate the contribution of the decoding strategy.Source: §4.3 · p. 6
- 36%–215% relative improvement in Y-Z coherence over unsupervised baselines was reported in both human-evaluation setups.The evaluation also found higher X-Y coherence than supervised methods and competitive pairwise performance.Source: Table 2 · p. 6, Table 3 · p. 6, §4.3 · pp. 6–7
- A qualitative example shows the future observation “car was totaled” pointing to an accident, which DELOREAN captures in its generated hypothesis.Figure 3 presents generated hypotheses for three abductive cases.Source: Figure 3 · p. 6, §4.3 · p. 7
5 Task 2: Counterfactual Reasoning
For counterfactual story revision, DELOREAN generates endings that minimally edit the original while restoring coherence with the changed condition. Its human and automatic evaluations report advantages over unsupervised and discriminative baselines on different dimensions.
5 Task 2: Counterfactual Reasoning
- The task changes future story events after a counterfactual condition invalidates the original ending, seeking a coherent rewrite with minimal edits.The method encourages recovery of the original ending by minimizing KL divergence and ranks candidates using context coherence and internal coherence across their three sentences.Source: §5 · p. 7, §5.1 · p. 7
5.2 Experimental Setup
- Baselines included zero-shot GPT-2, GPT-2 fine-tuned on original stories with language-model or reconstruction-plus-counterfactual objectives, and a supervised model.Automatic evaluation used BERTSCORE for counterfactual coherence and BLEU-4 and ROUGE-L as measures of minimal edits.Source: §5.2 · pp. 7–8, §5.3 · p. 8
5.3 Results
- DELOREAN alone maintained a balance of coherence (1.66) and minimal edits (1.54); ranking-augmented zero-shot favored coherence (1.8), while Recon+CF copied the original ending verbatim in 84% of cases.Recon+CF's coherence score was 1.23; as β increased to emphasize minimal edits, the ranking-augmented zero-shot model's weighted performance dropped considerably.Source: §5.3 · p. 8
- In pairwise human comparisons, DELOREAN exceeded Recon+CF and Sup+Disc in coherence but trailed zero-shot re-ranked baselines; for minimal edits, the pattern reversed.The human evaluation also found that DELOREAN had the highest BERTSCORE for counterfactual coherence, while discriminative baselines achieved the highest plot fidelity.Source: §5.3 · pp. 8–9
- DELOREAN's examples rewrite endings to fit counterfactual conditions while preserving original details that remain coherent.In one example, changing Tara's shopping trip from a mall to an online order leads to a rewritten ending about a shirt being sent to her, while preserving the final sentence about wearing it.Source: §5.3 · p. 9
6 Related Work
Related work spans unsupervised generation, controllable language generation, and narrative reasoning. The paper distinguishes its open-ended reasoning setting from approaches focused on copying, discrete controls, or narrowly scoped infilling.
Unsupervised text generation
- Prior unsupervised generation often copies or selects information from source text, whereas this work targets open-ended generations that agree with and expand on prompts, making autoencoding less applicable.The cited examples include paraphrasing, translation, and summarization.Source: §6 · p. 9
Controllable language generation
- Earlier controllable generation changed discrete attributes, while PPLM uses forward and backward passes but requires trained attribute discriminators; text-infilling models fill short spans and may require extra right-to-left models or preset lengths.The paper characterizes its own approach as unsupervised and able to adjust to open-ended textual constraints.Source: §6 · p. 9
Reasoning about narratives
- Narrative reasoning research includes RocStories and its story-cloze task, along with work on event scripts, temporal commonsense, event preconditions, document events, and multiple-choice counterfactual questions.The paper places its language-based reasoning tasks among these narrative-reasoning directions.Source: §6 · p. 9
7 Conclusion
The conclusion presents DELOREAN as an unsupervised language-model approach that conditions generation on past context and future constraints through forward and backward passes. It reports effectiveness on abductive and counterfactual reasoning and says the method can be adapted to other generative reasoning tasks.
7 Conclusion
- DELOREAN generates text conditioned on past context and future constraints through forward and backward passes.The conclusion describes the approach as unsupervised.Source: §7 · p. 9
- The paper reports that DELOREAN performed substantially better than unsupervised baselines on abductive and counterfactual reasoning.Source: §7 · p. 9
- The authors state that the method can be adapted for other generative reasoning tasks.Source: §7 · p. 9
A.1 Abductive Reasoning
The abductive-reasoning setup uses a 15-token hypothesis length for the backward pass, permits longer forward generations for complete sentences, and runs 20 forward-backward iterations.
A.1 Abductive Reasoning
- The setup uses N = 15 backward-pass tokens, allows the forward pass to generate 2N tokens for complete sentences, and runs 20 forward-backward iterations.Each backward pass performs 20 gradient updates with λ = 0.0003; forward/backward logits are mixed with γ = 0.88, and greedy decoding produces one candidate per iteration.Source: §A.1 · p. 13
A.2 Counterfactual Reasoning
For counterfactual reasoning, generation is split into three sentence-level segments, with each generated sentence added to the context for the next. Eight candidates are produced per example and ranked; this generation-ranking protocol gave better performance on the counterfactual task.
A.2 Counterfactual Reasoning
- The generation-ranking protocol gave better performance on the counterfactual task.Each example produces eight candidates for ranking.Source: §A.2 · p. 13
- The counterfactual setup varies forward-backward iterations over {5, 10} and backward gradient updates over {5, 8, 10, 15}.Each configuration produces one candidate, yielding eight candidates per example.Source: §A.2 · p. 13
- Generation is split into three sentence-level segments, and each generated sentence is appended to the context for the next segment.Forward-backward passes are performed separately for each segment.Source: §A.2 · p. 13