Source-linked AI summary
Measuring and Improving Consistency in Pretrained Language Models
Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, Yoav Goldberg
TL;DR
The paper asks whether pretrained language models preserve factual knowledge across paraphrased queries. It constructs ParaRel and evaluates masked language models with zero-shot cloze probing. The models are generally inconsistent, while continued pretraining with a novel consistency loss improves consistency and factual extraction.
Problem
The paper investigates whether factual information extracted from PLMs is invariant to paraphrasing, an important property for using them as knowledge bases.
Method
The authors create ParaRel and use zero-shot cloze evaluation across paraphrase patterns, then continue pretraining with a novel consistency loss.
Results
PLMs generally show low consistency, while the proposed method improves both consistency and the ability to extract correct facts.
Takeaways & Limitations
ParaRel provides a benchmark for tracking knowledge consistency, and consistency can be improved through continued pretraining with the proposed loss.
Takeaways & Limitations
The PLM-as-knowledge-base setup lacks standard KB abilities to return multiple answers or no answer, and the origins of observed behavior remain unexplained.
Abstract
from arXiv · showhide
Consistency of a model -- that is, the invariance of its behavior under meaning-preserving alternations in its input -- is a highly desirable property in natural language processing. In this paper we study the question: Are Pretrained Language Models (PLMs) consistent with respect to factual knowledge? To this end, we create ParaRel, a high-quality resource of cloze-style query English paraphrases. It contains a total of 328 paraphrases for 38 relations. Using ParaRel, we show that the consistency of all PLMs we experiment with is poor -- though with high variance between relations. Our analysis of the representational spaces of PLMs suggests that they have a poor structure and are currently not suitable for representing knowledge robustly. Finally, we propose a method for improving model consistency and experimentally demonstrate its effectiveness.
1 Introduction
The paper asks whether pretrained language models preserve factual knowledge across paraphrased queries. It introduces ParaRel to measure this property and finds generally poor consistency, then proposes a consistency loss that improves performance on unseen relations.
- Motivation: Consistency requires models to make the same decisions in semantically equivalent contexts despite language variability.The paper motivates consistency as a desirable property of language understanding models.
- PLMs as Knowledge Bases: PLMs are considered as knowledge bases because they can be queried in natural language rather than through a fixed schema.Equivalent-intent queries are expected to yield the same answer.
- Research Question: The paper studies whether factual information extracted from masked language models is invariant to paraphrasing using zero-shot evaluation.Zero-shot probing is used to inspect behavior acquired during pretraining without finetuning biases.
- Approach: ParaRel contains 328 manually curated paraphrases for 38 binary relations and is used to test factual consistency across BERT, BERT-whole-word-masking, RoBERTa, and ALBERT.The resource provides short textual prompts that paraphrase one another.
- Findings: All tested PLMs show poor consistency with high variance between relations, while a novel consistency loss improves BERT on unseen relations.The paper also states that fully consistent models remain a goal for future work.
2 Background
Prior work studies consistency across knowledge bases and NLP tasks, but accuracy alone does not capture contradictory predictions. Existing approaches measure inconsistency and use augmentation, regularization, memory, or other methods to improve robustness.
- Evaluation: Accuracy is insufficient for evaluating PLMs because models may make contradictory predictions even when standard task performance is strong.The paper frames consistency as a distinct performance property relevant to linguistic, commonsense, and reasoning tasks.
- Knowledge Bases: Consistency has been studied theoretically in knowledge bases through satisfiability, inconsistency detection, and measures of inconsistency.This background includes algorithms for detecting inconsistent statements and quantifying their degree.
- Question Answering: Question-answering studies find that many models are inconsistent and explore synthetic questions, data augmentation, and regularizing losses.These methods aim to create more robust or accurate QA models.
- Other Domains: Other work measures or mitigates inconsistency in procedural text, robust evaluation, natural language inference, and NLI explanations.The surveyed methods span both prediction consistency and explanation consistency.
3 Probing PLMs for Consistency
The paper defines consistency through non-contradictory predictions for quasi-paraphrased cloze queries and probes PLMs by applying multiple patterns to the same subject. The framework restricts outputs to valid candidate entities, making the task easier but preserving a strong consistency test.
- 3.1 Consistency: A model is consistent when quasi-paraphrased cloze phrases produce non-contradictory predictions across entities for N-1 relations.Quasi-paraphrases operationalize concrete, meaning-equivalent uses without requiring strict logical paraphrase.
- 3.1 Consistency: Non-contradictory predictions may include related answers such as a city and its country, whereas two different birth cities are contradictory.The distinction defines how factual conflicts are treated in evaluation.
- 3.1 Consistency: Many-to-many relations are excluded from consistency measurement because different correct answers can be consistent, while determinism is measured separately.Determinism concerns returning multiple results in the same order.
- 3.2 The Framework: For each relation, subject-object KB tuples are paired with quasi-paraphrase cloze patterns, and the model is expected to predict the same object across patterns.The framework substitutes each subject into every pattern and places a mask token in the object position.
- 3.2 The Framework: The framework restricts the output vocabulary to possible gold objects because PLMs often predict words that are not valid knowledge-base entities.This setup makes the task easier, so poor consistency strongly implies even lower consistency without restriction.
4 The PARAREL Resource
ParaRel is a manually curated benchmark of paraphrased cloze patterns for factual relations. Its construction combines existing resources, Wikipedia-derived patterns, expert additions, and annotation-based quality control, producing a high-agreement resource.
- Resource: ParaRel contains patterns for 38 T-REx relations, averaging 8.63 patterns per relation, with syntactic and lexical variation statistics reported.T-REx supplies knowledge-base triples aligned with Wikipedia abstracts.
- Construction: The resource construction begins with LAMA base patterns and augments them with candidate paraphrases from LPAQA.Automatically generated LPAQA patterns are not all correct paraphrases, motivating further curation.
- Construction: Additional patterns are manually extracted from Wikipedia sentences containing T-REx subject-object tuples and added by annotators using linguistic expertise.Two experts reviewed and corrected the patterns, discarding items without agreement.
- Human Agreement: Annotators evaluate paraphrase candidates against base patterns and include non-paraphrase controls with the same subjects and objects.Each task contains five presumed paraphrases and two controls.
- Human Agreement: 95.5% agreement on paraphrases and 98.3% on controls supports ParaRel’s high quality, further improved by inspecting disagreements.The annotations were collected from NLP graduate students.
5 Experimental Setup
The experiments evaluate four PLM families on filtered T-REx knowledge using paraphrase-based consistency measures alongside accuracy and a stricter combined metric.
- Models & Data: The study evaluates BERT, BERT whole-word-masking, RoBERTa, and ALBERT, including base and large versions where available, plus a perfectly consistent majority baseline.
- Models & Data: The evaluation retains 26,813 T-REx tuples whose objects are single tokens across all model vocabularies, separating seven N-M and 31 N-1 relations.
- Evaluation Measures: Consistency measures the percentage of non-contradictory predictions across all pattern pairs for each relation, totaling n(n −1)/2 pairs for n patterns.
- Evaluation Measures: Consistent-Acc counts an object as correct only when every pattern predicts it, combining factual accuracy with consistency more strictly than either measure alone.
6 Experiments and Results
Across extractability, knowledge, consistency, pretraining-size, and syntactic-variation analyses, PLMs show low and highly variable factual consistency, with notable differences across models and relations.
- Knowledge Extraction through Different Patterns: All PLMs successfully extract the correct object from at least one pattern, but Succ-Objs ranges from 45.8% for ALBERT-base to 65.7% for BERT-large.The authors interpret the incomplete Succ-Objs scores as evidence that much T-REx knowledge is not stored in these models.
- Consistency & Knowledge: Accuracy ranges from 29.8% for ALBERT-base to 48.7% for BERT-large whole-word-masking, while Consistent-Acc ranges from 16.4% for RoBERTa-base to 29.5% for BERT-large.
- Consistency & Knowledge: Consistency remains low overall, with the best model reaching 61.1%; BERT is more consistent than RoBERTa and ALBERT, and performance varies across relations.BERT-large scores 92% on capital-of but 44% on owned-by.
- Consistency & Knowledge: Accuracy and Consistency correlate strongly, ranging from 67.3% for RoBERTa-base to 82.1% for BERT-large.All reported correlations have small p-values ≪0.01.
- Consistency & Knowledge: Determinism follows a similar trend, measuring whether N-M relation predictions are returned in the same order; BERT-large scores 52.9% versus 44.6% for RoBERTa-base.
- Consistency & Knowledge: More training data generally improves Accuracy and Consistent-Acc, but a one-million-token RoBERTa is more consistent than models trained on ten or one-hundred million tokens.The one-million-token model has fewer parameters and lower accuracy, which the authors associate with less factual knowledge exposure.
- Do PLMs Generalize Over Syntactic Configurations?: BERT-large reaches 67.5% when only syntax differs and 78.7% when syntax and lexical choice are identical, yet PLMs still struggle with tense, word-order, and syntactic variation.
7 Analysis
The analysis examines qualitative predictions and representation spaces to understand why PLMs behave inconsistently across paraphrased factual queries. Predictions vary between correct, incorrect, consistent, and inconsistent outputs, while representations mainly cluster by pattern rather than subject.
- 7.1 Qualitative Analysis: BERT-large produces inconsistent predictions across paraphrased patterns, including cases where only one pattern gives the correct object.Other examples are consistently wrong, inconsistently wrong, or consistently factual.
- 7.1 Qualitative Analysis: Some patterns yield different predictions despite expressing the same relation, while others produce the same incorrect answer.The examples include Amsterdam versus Madagascar and Luxembourg, and repeated Renault predictions.
- 7.2 Representation Analysis: Figure 3 encodes subjects by color and patterns by shape, showing that the capital-relation representations cluster mainly by identical patterns rather than identical subjects.A knowledge-focused representation would instead cluster examples with the same subject.
- 7.2 Representation Analysis: The representation analysis encodes subjects into masked patterns and inspects the final-layer representation at the masked-token position.The resulting representations are visualized with t-SNE and also evaluated through clustering.
- 7.2 Representation Analysis: The mismatch between pattern-based representations and model behavior may reflect distances driven by behaviorally irrelevant sentence factors.This interpretation is presented as a hypothesis and aligns with similar trends reported for linguistic tasks.
8 Improving Consistency in PLMs
The paper improves PLM consistency by training on paraphrased relation patterns with a consistency loss alongside MLM training. On 25 unseen test relations, the resulting model improves consistency, accuracy, and Consistent-Acc, although downstream-task gains remain unresolved.
- 8 Improving Consistency in PLMs: The method targets the question of whether PLMs can become more consistent after pretraining, using T-REx tuples and ParaRel paraphrases.Training uses three relations and evaluates generalization to 25 unseen relations.
- 8.1 Consistency Loss Function: For each relation, the model encodes every subject-filled paraphrase and is trained to produce similar masked-token predictions across patterns.The consistency loss compares predicted distributions for pattern pairs rather than only their top-1 outputs.
- 8.1 Consistency Loss Function: The consistency objective uses two-sided KL divergence over candidate-token distributions, while filtering the vocabulary to relation-specific candidates.Candidate filtering is intended to preserve the model’s original capabilities by leaving most vocabulary items unaffected.
- 8.2 Consistency Guided MLM Continual Training: The final training objective combines the consistency loss with regular MLM loss using a weighting hyperparameter λ.The continued MLM objective is applied to text data or relation patterns, with relation-pattern MLM found to work better.
- 8.3 Improved Consistency Results: 64.0% consistency versus 58.2% for BERT-base, a 5.8-point increase, accompanied by accuracy rising from 45.6% to 47.4%.Consistent-Acc also increases by 5.9 points, and all reported improvements are statistically significant with p ≪0.01.
- 8.3 Improved Consistency Results: Removing MLM training raises consistency to 80.8% but lowers Accuracy and Consistent-Acc, suggesting MLM acts as a regularizer against catastrophic forgetting.The finetuned model without the consistency loss improves over BERT-base but remains behind the full finetuned model.
- 8.3 Improved Consistency Results: On SQuAD and paraphrased SQuAD questions, the consistency model performs on par with the baseline.The authors state that more research is required to establish downstream-task benefits.
9 Discussion
The discussion situates factual consistency within broader NLP and knowledge-base concerns, while emphasizing both the promise and limits of the study’s analysis. It also argues that consistency should be addressed in pretrained models because pretraining capabilities propagate into fine-tuned systems.
- Broader Sense of Consistency: Consistency is broader than paraphrase invariance, also encompassing behavior under changes such as negation.The paper therefore studies one specific form rather than the full concept.
- Usage of PLMs as KBs: Using PLMs as knowledge bases lacks standard capabilities to return multiple answers or no answer, and probability-threshold heuristics may not be optimal.The restriction follows because PLMs were not trained to serve as knowledge bases.
- Brittleness of Neural Models: The benchmark exposes brittleness by separating robustly encoded facts from guesses associated with heuristics or spurious pattern correlations.The authors report that PLMs are susceptible to small perturbations, with downstream fine-tuning unlikely to fully resolve the issue when equivalent examples are absent.
- Can we Expect from LMs to be Consistent?: Consistency matters across NLP tasks because models should make equivalent decisions under meaning-preserving variations, yet standard training does not explicitly target it.Pretraining objectives can favor text-specific likelihoods even when resulting generations contradict each other.
- Reasons Behind the (In)Consistency: The study’s representation-space analysis does not identify the origins of consistent or inconsistent predictions, which the authors leave for future work.It presents predictions from multiple queries and the representation space of one inspected model.
10 Conclusion
The conclusion presents PARAREL as a benchmark for measuring factual knowledge consistency and reports generally low consistency across several PLMs. It also introduces continued pretraining with a novel loss as an effective way to improve consistency and factual extraction.
- Benchmark: PARAREL contains 328 high-quality patterns covering 38 relations and is paired with T-REx data tuples as a benchmark for knowledge consistency.The resource is designed to support consistency measurement in NLP models.
- Findings: Consistency is generally low across BERT, RoBERTa, and ALBERT, although RoBERTa and ALBERT outperform BERT on other tasks.The latter two models nevertheless fall short of BERT in consistency.
- Improvement Method: Continuing pretraining with a novel loss improves both model consistency and the ability to extract correct facts.The method is described as simple and effective.
A Appendix
The appendix identifies the software libraries used to run the PLM experiments and related machine-learning analyses.
- Implementation: The experiments rely on Hugging Face’s Transformers for PLMs, Weights & Biases for experiment tracking, and sklearn for other machine-learning experiments.These tools cover model execution, logging, and auxiliary ML analyses.
B Paraphrases Analysis
The paraphrase analysis categorizes transformations in PARAREL using sampled annotated pairs, including established and newly defined types. Synonym substitution is most common, while change of aspect is rarest in the sample.
- Analysis Procedure: The analysis samples 100 annotated paraphrase pairs and labels their transformation types, allowing multiple types per pair.The categorization mainly follows Bhagat and Hovy while adding types not covered there.
- Paraphrase Types: The dataset includes synonym substitution, function-word variation, converse substitution, tense change, voice change, verb/noun conversion, external knowledge, adjective conversion, and aspect change.It also defines irrelevant addition, topicalization, apposition, and other syntactic movements.
- Paraphrase Types: The analysis additionally includes external knowledge, noun/adjective conversion, and change-of-aspect transformations.These categories capture substitutions or grammatical changes appropriate to the context.
- Paraphrase Types: Newly defined categories include irrelevant addition, topicalization, apposition, and other syntactic movements.These types cover meaning-preserving additions and several structural rearrangements.
- Observed Frequencies: Synonym substitution is the most common transformation, followed by function-word variation occurring 41 and 16 times, respectively.The counts come from the random sample used in the analysis.
- Observed Frequencies: Change of aspect is the least common transformation, occurring once in the sample.Each pair may receive multiple classifications, so transformation totals can exceed 100.