Source-linked AI summary
Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoning
Maxwell Nye, Michael Henry Tessler, Joshua B. Tenenbaum, Brenden M. Lake
TL;DR
Neural sequence models often produce inconsistent or incoherent outputs, motivating a lightweight alternative to retraining. The paper adds a symbolic System 2 that checks neural candidates for logical consistency. The resulting approach improves coherence in language generation and performance in grounded instruction following, especially in low-data regimes.
Problem
Neural sequence models often contradict prior statements, world knowledge, or logical constraints, while targeted correction may require impractical retraining or fine-tuning.
Method
The approach uses neural models to propose candidate generations and a symbolic world model to accept, reject, or resample them without additional training or fine-tuning.
Results
The dual-system approach improves coherence and consistency in text generation and improves grounded instruction-following performance, particularly in low-data regimes.
Takeaways & Limitations
A lightweight neuro-symbolic System 2 can improve powerful neural sequence models by enforcing a small number of explicit logical constraints at inference time.
Takeaways & Limitations
The approach is most useful when a small set of mission-critical constraints can be explicitly articulated; hand-engineering becomes difficult with many or hard-to-specify constraints.
Abstract
from arXiv · showhide
Human reasoning can often be understood as an interplay between two systems: the intuitive and associative ("System 1") and the deliberative and logical ("System 2"). Neural sequence models -- which have been increasingly successful at performing complex, structured tasks -- exhibit the advantages and failure modes of System 1: they are fast and learn patterns from data, but are often inconsistent and incoherent. In this work, we seek a lightweight, training-free means of improving existing System 1-like sequence models by adding System 2-inspired logical reasoning. We explore several variations on this theme in which candidate generations from a neural sequence model are examined for logical consistency by a symbolic reasoning module, which can either accept or reject the generations. Our approach uses neural inference to mediate between the neural System 1 and the logical System 2. Results in robust story generation and grounded instruction-following show that this approach can increase the coherence and accuracy of neurally-based generations.
1 Introduction
Neural sequence models can generate fast, pattern-based text yet often violate consistency, coherence, and basic logical or world-knowledge constraints. The paper proposes a lightweight dual-system approach in which neural proposals are filtered by logical reasoning without additional problem-specific training.
- Neural models can contradict character attributes, relationships, prior statements, and basic world knowledge during generation.
- GPT-2 can produce different answers to nearly identical prompts and state mutually inconsistent facts, illustrating the difficulty of targeted fixes without retraining.
- Dual-process theories motivate combining intuitive System 1 generation with deliberative System 2 reasoning to override inconsistent responses.
- The proposed neuro-symbolic system treats neural predictions as System 1 proposals and uses a logical module to filter them for consistency and soundness without additional problem-specific training.
- Across story generation and instruction following, the approach uses neural proposals with symbolic acceptance or rejection to improve coherence and accuracy.
2 Related Work
Prior work improves neural generation through semantic parsing, generate-and-rerank methods, planning, and symbolic decoding constraints. This paper instead uses a symbolic world model to post-check candidate generations for explicit logical consistency.
- Semantic parsing has been combined with neural generation, including few- or zero-shot parsing and systems trained without supervision.
- Generate-and-rerank methods use one model to propose outputs and another to rerank them for broad criteria across generation and instruction-following tasks.
- Planning-based story systems generate events, templates, or skeletons before producing text, whereas this work uses its world model for post-checking rather than planning.
- Other approaches incorporate symbolic constraints during neural decoding through propose-and-reject inference or logical requirements on generated content.
3 Integrating System 1 and System 2
The system combines neural candidate generation, few-shot semantic parsing, and a minimal symbolic world model that rejects logically inconsistent utterances. In bAbI story generation, repeated resampling produced logically coherent stories while preserving the domain’s textual structure.
- The approach decomposes generation into neural proposals and symbolic pruning by extracting constrained world facts and checking candidates against them.
- A few-shot GPT-3 parser mediates between System 1 proposals and System 2, requiring no fine-tuning for the parsing component.
- The minimal world model tracks only selected variables, such as people, locations, and objects, to enforce constraints including unique locations and possession.Its purpose is not to verify all possible information, but to capture a few commonsense or application-critical variables.
- At generation time, inconsistent candidates are rejected and resampled until a candidate compatible with the current world-model state is found.
- The bAbI diagnostic compares neural-only and dual-system stories, including examples where the world model preserves the apple’s correct location.
- 53.1% of generated bAbI sentences were rejected, and every one of 50 stories required resampling at least one sentence to maintain coherence.The resulting stories mimicked the statistical structure of bAbI while remaining logically sound.
4 Coherent Language Generation - CLUTRR
The CLUTRR system uses a neural proposer and a logical world model to generate family-relationship stories while filtering contradictions. With up to ten re-sampling attempts, this approach improves logical consistency and human-perceived coherence over neural-only generation.
- Model specification: The CLUTRR evaluation targets complete or partial human-written family-relationship stories rather than the dataset’s original question-answering task.The goal is coherent generation with logically consistent family relations.
- Model specification: The System 1 proposer generates candidate sentences, while a constraint solver encodes family relations and rejects candidates that make the relations unsatisfiable.GPT-3 extracts logical parses, and Z3 checks constraints such as prohibiting a person from being their own child or marrying a sibling.
- Human judgments: The human evaluation compared paired continuations, with one generated by the neural model alone and the other by a dual-system model.Participants selected which continuation made the most sense; the study used four between-participant conditions and excluded participants failing quality checks.
- Human judgments: Participants preferred dual-system world-model continuations over single-system continuations in both prompt conditions: 0.84 for dataset prompts and 0.79 for model prompts.These values are proportions preferring dual-system generations, with bootstrapped 95% confidence intervals reported in the results.
5 Grounded Instruction Following
The grounded instruction-following system enforces consistency between neural action proposals and predicted target locations. Test-time symbolic execution improves evaluation accuracy, especially in low-data regimes, without additional training.
- The approach combines an LSTM action-sequence predictor, a target-location predictor, and deterministic gridworld execution to filter inconsistent actions.The target-location distribution supplies a constraint, while execution checks whether an action sequence reaches that location.
- The framework builds on shared-parameter action and location prediction, using target-location consistency as an additional correctness signal.An action sequence inconsistent with the predicted target location is treated as likely incorrect.
- The method's effect depends on how often single-system and dual-system predictions disagree, with disagreements reported in 14%–57% of cases across data regimes.Frequent disagreements allow the consistency filter to change predictions and produce a larger overall effect.
- The dual-system model improves gSCAN evaluation accuracy without additional training, particularly in low-data regimes.It samples candidate action sequences and accepts only those consistent with the predicted target location.
6 Limitations
The approach is most suitable when naturalistic generation requires only a few explicit, mission-critical logical constraints. Its current limitations include hand-engineered world knowledge, few-shot parsing capacity, and unresolved questions about learning structured knowledge.
- The method is most useful when naturalistic generation is needed and a small number of mission-critical logical constraints can be explicitly articulated.It is less suitable for creative domains or settings with many constraints because the minimal world model must be hand-engineered.
- Few-shot parsing may not scale to complex domains when examples exceed the input window or become too difficult for accurate parsing.Success is expected to be strongest when tracking only a few critical variables.
- The symbolic world model currently requires manual updates, motivating future work on automatically learning and updating structured knowledge.The authors identify neuro-symbolic learning as one possible direction.
- Symbolic judgments could supervise a learned System 2 contradiction detector, but this remains a proposed extension rather than part of the current approach.Rejected and accepted utterances could provide examples of contradictory and noncontradictory statements.
- Although pretrained models can represent some structured semantic information, the relationship between representational fidelity and generative logical coherence remains unclear.The current approach explicitly fixes logical errors during generation, while improving underlying representations is left for future work.
7 Conclusion
The paper combines neural generation with symbolic reasoning to incorporate domain knowledge and improve robustness. Across language generation and grounded instruction following, the approach improves coherence, consistency, or performance without requiring the neural model to replace its generative role.
- The paper combines neural and symbolic approaches to build more robust models that incorporate domain knowledge.The design is inspired by dual-process theories of intuitive and deliberative reasoning.
- In language generation, a minimal symbolic world model increases the coherence and consistency of neural text generation.
- In grounded instruction following, test-time consistency between predicted action sequences and goal locations improves performance, especially in low-data regimes.
- The approach is presented as applicable to other domains and forms of prior knowledge as a lightweight way to improve neural sequence models.The authors frame the work as an initial step toward more robust neuro-symbolic systems.
A Additional CRT experiments
Additional GPT-3 cognitive-reflection experiments test several variants of each question because the original tasks may have appeared in the model's training data. Responses are shown by correctness and generated at temperature 0.
- The experiments compare original CRT questions with conceptually similar variants to reduce concerns about training-set exposure.For each question type, the original and several response variants are presented together.
- Figure 8 color-codes GPT-3 responses by correctness, with green indicating correct answers and red indicating incorrect answers.Prompt text is black and response text is colored.
- All additional CRT experiments use the GPT-3 davinci model with temperature 0.
B Experimental Details
Experiments used GPT-3’s largest available davinci model, while other models were implemented in PyTorch and run on a single Nvidia GTX 1080 Ti GPU.
- GPT-3 experiments used the largest available “davinci” model, while other models were implemented in PyTorch.Testing and training ran on one Nvidia GTX 1080 Ti GPU.
- Language-generation experiments generally took less than 4 hours, whereas gSCAN experiments took less than 48 hours.
B.1 bAbI
The bAbI experiments evaluate question answering with a minimal world model that tracks entities, updates state, and rejects inconsistent candidate actions.
- Question-answering results for bAbI are reported in Table 3.
- The minimal bAbI world model tracks mentioned people, objects, and locations, updates world-state changes, and checks candidate actions for violations.
- The world model represents entities with a lookup table mapping names to Python objects belonging to Person, Location, or Obj classes.
- The full bAbI world model, including GPT-3 fact-extraction interpretation code, contains fewer than 200 lines of code.
B.2 CLUTRR
CLUTRR experiments use a fine-tuned BART-base model and symbolic family-relationship constraints to evaluate consistent story generation under different sampling budgets.
- The CLUTRR model uses BART-base fine-tuned on 24-sentence systematic-generalization stories with AdamW, learning rate 1e-5, and dropout 0.1.
- Table 5 reports CLUTRR story-generation statistics for the symbolic and neural NLI System 2 models.
- Using a sampling budget of 5 instead of 10 reduced error-free CLUTRR stories to 93.5% for dataset prompts and 88.3% for model prompts.Error-free lines were 96.0% and 94.6%, respectively, under the five-sample budget.
- The symbolic CLUTRR world model encodes gender-neutral family constraints, while parsing accepts recognized relationship terms and ignores other outputs.
B.3 gSCAN
The gSCAN experiments compare dual-system and neural-only models using exact-match accuracy, logical consistency checks, and test-time candidate search across data regimes.
- The dual-system model outperforms baselines in nearly all gSCAN test splits and is especially beneficial with less training data.
- The gSCAN architecture uses a BiLSTM instruction encoder, a CNN gridworld-state encoder, and attention-based prediction of target locations and actions.
- When greedy decoding produces an inconsistent candidate, the dual-system model performs sample-based search with a budget of 50 samples.
- The gSCAN consistency check requires the agent to pass through the target location without leaving the gridworld, rather than finish at that location.This less stringent constraint is reported as sufficient for significant accuracy gains.