Calibrating Factual Knowledge in Pretrained Language Models

1 Introduction

• CKA detects false knowledge by testing whether a PLM scores correct facts above possible negative facts.The approach evaluates model predictions rather than requiring access to the model's black-box decision process.Source: §1 · p. 1
• CaliNet calibrates incorrect factual knowledge by adding trainable parameters while keeping the original PLM parameters fixed.It extends a specific feed-forward network with calibration memory slots and trains them on paraphrased sentences expressing correct facts.Source: §1 · pp. 1–2
• The calibrated PLM generalizes corrected knowledge to probing and question-answering tasks.Experiments report efficient calibration and remarkable generalization, alongside analysis of calibration memory slots and mechanisms.Source: §1 · p. 2
• Nearly 50% of facts randomly sampled from T-REx are stored incorrectly in PLMs.Source: §1 · p. 2
• The paper investigates calibration through calibration memory slots to explain how CaliNet works.Source: §1 · p. 2

2 Contrastive Knowledge Assessment

• CKA contrasts one positive prompt expressing the correct fact with negative prompts expressing incorrect relations.For a fact ⟨s, r, o⟩, the model should assign o higher probability under the correct relation than under an erroneous relation.Source: §2 · p. 2, Figure 2 · p. 2
• The CKA score combines the positive-fact probability with the expected probability under sampled negative relations using a smoothing factor α.Multiple erroneous relations are sampled to make the comparison more stable.Source: §2 · p. 2
• Positive prompts use LAMA templates, while manually designed negative prompts provide contradictory semantics for the same entity type.Relation replacement is preferred because relations are more limited than entities.Source: §2 · pp. 2–3
• A threshold, usually below 1.0, can identify false knowledge from the CKA score.Source: §2 · p. 3
• CKA evaluates each tail entity independently, avoiding inexhaustible-answer requirements and reducing frequency bias in factual assessment.Rank-based assessment can miss multiple valid predictions and assign high ranks because an entity frequently co-occurs with the subject.Source: §2 · pp. 2–3

3 Knowledge Calibration

• The broader calibration procedure first detects false knowledge with CKA so the model can calibrate those facts rather than relying on task-specific knowledge modification.CKA evaluates tail entities independently and reduces frequency bias because the score includes the subject in both numerator and denominator; rank-based assessment is vulnerable to inexhaustible answers and frequency bias.Source: §3 · p. 3, Table 1 · p. 4

3.1 CALINET

• CALINET calibrates detected false facts by adding a small FFN-like module that adjusts the original FFN output without modifying the pretrained parameters.The method treats key-value pairs as calibration memory slots and introduces far fewer parameters because the calibration module's intermediate dimension is much smaller.Source: §3.1 · p. 3, §3 · p. 3

3.2 Calibration Data Construction

• The calibration data uses more than five paraphrased templates per detected false triplet, masks one entity, and separates templates across training, validation, and test sets.This construction targets factual knowledge rather than fitting only one textual surface form.Source: §3.2 · pp. 3–4, Table 2 · p. 4

3.3 Model Calibration

• CALINET is trained with a masked language modeling objective while the original PLM parameters remain frozen and only calibration memory slots are optimized.Using paraphrased expressions during training is intended to preserve calibration across multiple expressions of each fact.Source: §3.3 · p. 4

4.1 False Knowledge Detection

• Nearly half of T5-base facts are false under CKA, motivating explicit detection and calibration of incorrect knowledge.Source: §4.1 · p. 5

4.2.3 Results

• CaliNet lowers perplexity on correct-fact tests, raises it on adversarial tests, and has little effect on general-test perplexity, retaining semantic understanding.Source: §4.2.3 · pp. 5–6
• 48.10% to 17.09%: CaliNet reduces the overall False Rate measured by CKA while improving Top1 prediction EM and F1.Source: §4.2.3 · p. 6

4.2.4 Scalability of Knowledge Calibration

• 60%+: CaliNet calibrates more than 60% of 5000 facts at once, although calibration EM decreases as the fact count grows.Source: §4.2.4 · p. 6, Figure 4 · p. 6
• For 1000 facts, CaliNet's calibration EM remains above 70%, whereas modifying more than 125 facts in prior batched editing had success below 70%.Source: §4.2.4 · p. 6

4.2.5 Architectures of CALINET

• 64 slots suffice for 100 facts and 256 slots are nearly sufficient for 1000 facts, supporting efficient calibration with small added modules.Source: §4.2.5 · p. 6

Position to Concatenate CALINET

• Deeper decoder layers provide stronger calibration, with the last two layers performing comparably, consistent with deeper layers storing more factual knowledge.Source: Position to Concatenate CALINET · p. 7

4.3 Calibration Generalizability

• After fine-tuning, calibrated knowledge improves performance on originally incorrect QA questions, barely affects uncalibrated questions, and approaches continued pretraining with fewer updated parameters.Source: Table 5 · p. 6, §4.3 · p. 7

5.1 Meanings of FFN Values

• The analysis interprets each value through an input-independent output-vocabulary distribution, using the output embedding matrix and calibration memory-slot index.Source: §5.1 · p. 7
• Researchers manually annotate sampled FFN and CaliNet values from their top-30 tokens into categories including person, place, organization, date, and others.Source: §5.1 · p. 7
• Nearly 80% of CaliNet values correspond to meaningful concepts used to adjust hidden states for factual calibration.Source: §5.1 · pp. 7–8

5.2 Working Mechanism of CALINET

• The mechanism traces layerwise token distributions by adding CaliNet's output to the original FFN output and residual hidden state before applying the output embedding and softmax.Source: §5.2 · p. 8
• CaliNet changes the output token distribution substantially, unlike original FFNs whose effect on top-ranked tokens is limited.Source: §5.2 · p. 8
• CaliNet raises the probability of the correct token and its synonyms, indicating generalized calibration beyond memorizing a fact's surface form.Source: §5.2 · p. 8

• Prior probing studies show that PLMs store factual knowledge, while paraphrase sensitivity, biased prompts, and answer leakage complicate assessments of what they know.Source: §6 · p. 8
• Knowledge-injection methods add structured or dense external knowledge during pretraining or through adapters and extended FFNs.Source: §6 · p. 8
• CaliNet uses paraphrased natural sentences to exploit PLMs' semantic modeling and make calibrated knowledge usable in downstream tasks.Source: §6 · p. 9
• Unlike knowledge editing approaches that modify models after fine-tuning, CaliNet calibrates factual knowledge before fine-tuning so it can generalize downstream.Source: §6 · p. 9

• CaliNet detects incorrect PLM knowledge contrastively and calibrates it at scale without updating the original model parameters.Source: §7 · p. 9
• Calibrated PLMs generalize the corrected knowledge and outperform original PLMs on downstream tasks such as open-domain question answering.Source: §7 · p. 9
• The paper further investigates calibration through neuron-level analyses of how calibration memory slots adjust predicted token distributions.Source: §7 · p. 9

Limitations and Future Work

• The approach relies on existing knowledge bases and synthetic data, while template filling remains an efficient but non-realistic test-data solution.The authors identify full-scale knowledge assessment and calibration as a long-term goal because knowledge is complicated.Source: Limitations and Future Work · p. 9
• 35.61%: CALINET reduces the False Rate on real test data, consistent with template-filled test results.Three annotators wrote 50 test facts, each with one positive and three negative sentences.Source: Limitations and Future Work · p. 9
• Prediction-based evaluation can be biased, leaving accurate assessment of factual correctness as an open research question.The paper explicitly characterizes evaluation through PLM predictions as biased.Source: Limitations and Future Work · p. 9
• The current method cannot completely calibrate all factual errors in PLMs, motivating more advanced calibration methods.Source: Limitations and Future Work · p. 9

A Implementation Details

• Experiments use HuggingFace models, default hyperparameters unless noted, learning-rate grid search, and a single A40 GPU.The learning-rate grid is {1e-2, 1e-3, . . . , 1e-4}.Source: A Implementation Details · p. 11
• Calibration uses Adafactor with a constant learning rate, batch size 512, four accumulation steps, source length 64, target length 8, and 100 warm-up steps.Source: A Implementation Details · p. 11
• Training uses 5000 steps for 100 facts and 50000 steps for 1000 facts, while WebQuestions and TriviaQA fine-tuning allows up to 4000 steps.Source: A Implementation Details · p. 11

B Negative Case of CKA

• 13.92: the average CKA score for 1,000 native-language probing facts overestimates T5 knowledge.The negative templates have broader scope than the positive template, producing a low negative score.Source: B Negative Case of CKA · p. 11