BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Belief

1 Introduction

• PTLMs can answer probing questions inconsistently, even after specialized training, making their apparent beliefs difficult to pin down.The introduction frames consistent beliefs as relevant to debugging errors and encouraging rational behavior.Source: §1 · p. 1
• The system combines a weighted MaxSAT reasoner that revises clashing beliefs with feedback that uses known beliefs as context for future queries.The BeliefBank is built by deliberating over raw PTLM output.Source: §1 · p. 2
• The controlled experiments provide candidate facts and constraints between them to measure changes in belief consistency and accuracy against an original PTLM.The paper also contributes a targeted dataset for measuring consistency against given constraints.Source: §1 · p. 2
• The system avoids PTLM fine-tuning and uses an external mechanism for explicit belief reasoning, which the authors describe as supporting controllability and interpretability.The authors report that improving consistency this way improves accuracy; prior fine-tuning approaches cited here reported no or only slight accuracy gains.Source: §1 · p. 2
• A persistent BeliefBank lets a PTLM-based system deliberate over a global memory of beliefs, while constraint reasoning and feedback improve accuracy and consistency over time.The system aims to improve answers without retraining the model.Source: §1 · p. 2

2 Related work

• The paper draws on AI approaches to representing and updating beliefs—formal logic, belief revision (De Kleer, 1986; Dechter and Dechter, 1988), and uncertainty —as well as cognitive-science mental models.Source: §2 · p. 2
• The paper represents beliefs as natural-language statements rather than formal structures, layering them on top of pretrained language models.Source: §2 · p. 2
• Their reasoning component follows similar ideas applied to noisy PTLM predictions; a global persistent memory makes combining a PTLM with a constraint solver possible, despite their very different natures.The described knowledge-graph identification task builds a maximally consistent KG from noisy candidate facts, their extraction confidences, and constraints; its solution uses probabilistic soft logic (PSL).Source: §2 · p. 3
• Unlike memory components used as short-term computational working memory, BeliefBank is persistent and long-term, and treats the PTLM as part of a larger system.Unlike retrieval-augmented systems such as RAG (Lewis et al., 2020) and REALM (Guu et al., 2020), it is reflective: built from model outputs and reasoned over.Source: §2 · p. 3
• The feedback mechanism draws on Self-Talk (Shwartz et al., 2020) and dynamic prompt engineering, selecting feedback from a global BeliefBank rather than templated subqueries.The paper also frames its fixed-model system as an alternative to work that dynamically updates model parameters (Ebrahimi et al., 2021).Source: §2 · p. 3

3 Task

• The system receives a stream of questions, constraints between their truth values with weights, and a model that predicts True/False answers with confidence, optionally using prior answers as context.Source: §3 · p. 3
• The evaluation measures whether the system improves accuracy and consistency over time compared with the original PTLM.Candidate sentences may be true or false, and constraints link variabilized facts such as “X is a bird → X has wings”.Source: §3 · p. 3
• It accumulates the model’s True/False labels and may correct them with a constraint solver to improve gold-label accuracy and minimize total constraint-violation penalties.Source: §3 · p. 3

4 Approach

• The system adds a BeliefBank memory layer and manages its beliefs using constraint reasoning and feedback as questions arrive over time.Source: §4 · p. 3, §4.2 · p. 4

4.1 Definitions

• A belief is a sentence, a True/False label, and a strength; a constraint links a condition and expected conclusion with a penalty if violated.A false label represents belief in the sentence’s negation; constraints can use a shared variable to express multiple grounded rules.Source: §4.1 · pp. 3–4
• Consistency is measured as one minus the fraction of applicable constraints that are violated.A constraint is applicable when its condition is believed.Source: §4.1 · p. 4

4.2.1 Constraint Solving

• Constraint solving balances reducing violations against preserving raw model answers by treating answers as weighted constraints, with confidence scaled by a learned hyperparameter.The resulting weighted MaxSAT problem is solved with Z3; changed truth assignments update the BeliefBank.Source: §4.2.1 · p. 4

4.2.2 Feedback

• For feedback, selected prior beliefs are added to the query context; the method considers either random on-topic beliefs or the three strongest constraint-linked clashes across possible yes/no answers.If no relevant fact is found, it uses a randomly selected topic belief; three beliefs were empirically most effective.Source: §4.2.2 · pp. 4–5

5 Dataset

• The controlled dataset pairs constraints with facts over simple sentences such as “a swallow is a bird.”It was designed to support systematic experiments.Source: §5 · p. 5

5.1 Constraints

• 2,612 constraints combine ConceptNet positive implications with sibling-based mutual exclusivities from ConceptNet and WordNet.Positive implications were manually gathered from ConceptNet; sibling noun-hierarchy pairs were treated as mutually exclusive.Source: §5.1 · p. 5

5.2 Constraint Weights

• Grid search on 1,072 development facts about seven entities selects weights for forward implications, mutual exclusivities, and backward implications.The selected weights maximize F1 after the solver revises raw model beliefs.Source: §5.2 · p. 5

5.3 Facts

• 12,525 silver facts from 85 animal and plant entities are reserved for evaluation, with labels propagated from manually annotated source nodes.Because the implications are soft, propagated labels are not guaranteed to hold for every entity; there is no training phase or training data.Source: §5.3 · pp. 5–6

6 Model

• Macaw is queried with optional textual context and chooses between “yes” and “no,” with answer confidence used as the belief weight.The experiments use the T5-large version, a T5 QA model fine-tuned on approximately 400,000 QA pairs.Source: §6 · p. 6
• The model is used as a fixed black-box QA module without retraining.The paper notes that other models could equally have been used.Source: §6 · p. 6
• The experiments use the T5-large version of Macaw.Macaw was fine-tuned on approximately 400,000 QA pairs.Source: §6 · p. 6

7 Experiments

• The evaluation streams questions in batches and tracks F1 accuracy and consistency against gold labels across four configurations.Each batch represents another 10% of test questions; F1 is used because true and false labels are imbalanced.Source: §7 · p. 6, Figure 2 · p. 6

7.1 Results

• Constraint-solving reached about 95% consistency and improved accuracy by 17% over the raw model.Its global view of the BeliefBank lets it balance beliefs seen so far against provided constraints.Source: §7.1 · p. 6
• Relevant feedback improved consistency over on-topic feedback, while accuracy remained about the same and significantly above raw answers.Relevant beliefs may clash with the current answer; if they are wrong, feedback could amplify their errors.Source: §7.1 · pp. 6–7
• +18% absolute F1 over raw-model accuracy came from combining feedback and constraint-solving, the largest reported gain.The combined system's accuracy improved over time while the raw PTLM's accuracy remained constant.Source: §7.1 · pp. 6–7, Figure 3 · p. 7
• Off-topic feedback did not significantly differ from raw-model performance, while omniscient re-asking reached 74.5%, below incremental approaches.The omniscient version used raw answers from all questions as feedback; the incremental approach could feed corrected beliefs into later queries.Source: §7.1 · p. 7

7.2 Failure Analysis

• Raw-model errors were mostly false positives, and constraints could repair them but could also introduce errors through accumulated weighted rules.The raw model had about 98% recall and 54% precision; the solver's incorrect rat/cat flip illustrates sensitivity to constraint number and weights, even after tuning.Source: §7.2 · pp. 7–8
• Feedback is sensitive to question order, especially after early wrong answers, and the authors cannot yet explain why feedback improves results.Future work proposes more systematic feedback and better balancing of implicit and explicit knowledge, ideally with different confidence levels.Source: §7.2 · p. 8

8.1 Human in the Loop

• Correcting an average of six bad BeliefBank beliefs after the first 10% of questions raised final F1 and consistency by 2% absolute beyond the gains from the corrections themselves.The authors describe this as a brief, preliminary test of an interactive machine-teaching setting.Source: §8.1 · p. 8

8.2 Towards Deployment

• Real QA deployment would store declarative questions and answers, retrieve on-topic feedback, and apply constraint solving to detect clashes.The authors note that this development path begins from a constrained setting and may require textual entailment to match beliefs to constraint conditions or conclusions.Source: §8.2 · p. 8

8.3 The Broader Research Agenda

• A broader research agenda would use richer model structure, more sophisticated construction than accumulating and resolving answers, and explanations of deliberation for user interaction.These mechanisms could be symbolic or neural, but are beyond this paper’s scope.Source: §8.3 · p. 9

9 Conclusion

• A persistent global BeliefBank, constraint solver, and feedback mechanism make the overall PTLM system more coherent in accuracy and consistency.The memory layer can loosely be viewed as a mental model constructed from the PTLM’s raw answers.Source: §9 · p. 9
• The additional memory layer is characterized as a mental model built from the PTLM’s raw answers.The conclusion describes this interpretation as loose rather than a richer structured model.Source: §9 · p. 9
• The experiments used a restricted, controlled setting with a small set of relations, entities, and constraints, requiring further development for larger, more complex tasks.The authors frame the work as a first step toward globally consistent belief and continual improvement.Source: §9 · p. 9

A Selecting Constraint Rules from ConceptNet

• The authors selected 121 frequent, relatively unambiguous concepts and converted six ConceptNet relations into English constraint rules, manually filtering the rules for factual correctness.The relations were IsA, HasA, MadeOf, PartOf, HasProperty, and CapableOf.Source: A Selecting Constraint Rules from… · p. 12
• Weaker disjunctive backward rules were added to discourage the trivial solution that everything is false.For example, having a tail can weakly imply membership in one of several entity categories.Source: A Selecting Constraint Rules from… · p. 12
• Weights for forward and backward rules were set by automatic calibration.Source: A Selecting Constraint Rules from… · p. 12

• Tables 1 and 2 give the numerical data for the experimental results plotted in Figure 3.Source: B Experimental results in table form · p. 12
• Table 1 reports experimental results for accuracy (F1).Source: Table 1 · p. 13
• Table 2 reports experimental results for consistency (1 −τ).Source: Table 2 · p. 13