PaperlayerView exact revision on arXiv

Source-linked AI summary

ConflictBank: A Benchmark for Evaluating the Influence of Knowledge Conflicts in LLM

Zhaochen Su, Jun Zhang, Xiaoye Qu, Tong Zhu, Yanshu Li, Jiashuo Sun, Juntao Li, Min Zhang, Yu Cheng

arXiv:2408.12076v1cs.CLcs.AI

TL;DR

Knowledge conflicts in LLMs have received limited systematic study, especially conflicts within encoded knowledge and interactions among conflict forms. ConflictBank evaluates misinformation, temporal, and semantic conflicts across retrieved and embedded knowledge, with pilot experiments showing several patterns in model responses to conflicting information.

  • Problem

    Prior studies mainly examined conflicts between embedded model knowledge and retrieved context, leaving conflicts within encoded knowledge and interactions between conflict forms underexplored.

    Source: §1 · p. 2

  • Method

    ConflictBank constructs conflict claims and evidence for misinformation, temporal, and semantic scenarios, supporting studies of retrieved, embedded, and interacting conflicts.

    Source: §1 · p. 2, §2.2 · p. 3, §2.5 · p. 4

  • Results

    Models’ responses varied with evidence availability and conflict type; larger models showed lower memorization ratios, while more conflicting training data reduced default-answer ratios.

    Source: §3.2 · pp. 5–6, §3.3 · p. 7

  • Takeaways & Limitations

    The authors hope ConflictBank will support broader studies of conflict scenarios and contribute to more reliable and trustworthy language models.

    Source: §1 · p. 2

    The benchmark uses generative methods to construct conflict pairs, and the authors plan to explore other construction methods to verify its robustness.

    Source: §A.3 · p. 15

Abstract

from arXiv · show

Large language models (LLMs) have achieved impressive advancements across numerous disciplines, yet the critical issue of knowledge conflicts, a major source of hallucinations, has rarely been studied. Only a few research explored the conflicts between the inherent knowledge of LLMs and the retrieved contextual knowledge. However, a thorough assessment of knowledge conflict in LLMs is still missing. Motivated by this research gap, we present ConflictBank, the first comprehensive benchmark developed to systematically evaluate knowledge conflicts from three aspects: (i) conflicts encountered in retrieved knowledge, (ii) conflicts within the models' encoded knowledge, and (iii) the interplay between these conflict forms. Our investigation delves into four model families and twelve LLM instances, meticulously analyzing conflicts stemming from misinformation, temporal discrepancies, and semantic divergences. Based on our proposed novel construction framework, we create 7,453,853 claim-evidence pairs and 553,117 QA pairs. We present numerous findings on model scale, conflict causes, and conflict types. We hope our ConflictBank benchmark will help the community better understand model behavior in conflicts and develop more reliable LLMs.

1 Introduction

Prior work studied conflicts between a model’s embedded knowledge and retrieved context, while conflicts within encoded knowledge and interactions among conflict forms remained underexplored. ConflictBank is introduced as a benchmark for evaluating conflicts across pre-training and inference, with pilot experiments across twelve LLMs in four model series.

1 Introduction

  • Earlier studies focused on conflicts between embedded knowledge and retrieved context, leaving conflicts within encoded knowledge and interactions among conflict forms underexplored.Existing conflict datasets primarily used word substitution or language-model generation to construct conflict pairs.Source: §1 · p. 2
  • ConflictBank evaluates misinformation, temporal, and semantic conflicts across retrieved knowledge, embedded knowledge, and their interactions.The benchmark simulates conflicts encountered in pre-training and inference.Source: §1 · p. 2
  • ConflictBank contains 7,453,853 claim-evidence pairs and 553,117 QA pairs, built from 2,863,205 Wikidata claims.The paper reports pilot experiments on twelve LLMs across four model series.Source: §1 · p. 2
  • The authors release a Python package for data loading, baseline evaluation, and training, and hope the benchmark supports more reliable and trustworthy language models.The package is intended to make ConflictBank accessible for future research.Source: §1 · p. 2

2 CONFLICTBANK Benchmark

ConflictBank defines three conflict causes and constructs a benchmark by extracting Wikidata facts, turning them into conflict claims, generating varied evidence, and applying quality controls. Its evidence and QA data are intended to support evaluation of conflicts in retrieved and embedded knowledge and their interplay.

2.1 Knowledge Conflicts Causes

  • Misinformation, temporal, and semantic conflicts arise respectively from false information, changing knowledge, and ambiguity in words with multiple meanings.The paper describes these as three prevalent causes of knowledge conflict.Source: §2.1 · p. 2

2.2 Extracting Facts from Wikidata

  • Facts come from the April 2, 2024 Wikidata dump; the authors select one triple per subject-relation pair and use templates for the 100 most frequent relations.Triples and qualifiers are represented as (s, r, o, sd, od), with subject, relation, object, and descriptions of the subject and object.Source: §2.2 · pp. 2–3

2.3 Constructing Knowledge Conflict Claims

  • Conflict claims use same-type entity substitutions, with cause-specific changes for misinformation, temporal updates, and semantic ambiguity.Temporal claims add a future time span; semantic conflicts introduce an additional description for a conflicting subject.Source: §2.3 · p. 3

2.4 Generating Diverse Evidence Texts

  • LLMs generate supporting evidence in Wikipedia, book, and news styles, using entity descriptions to help produce relevant evidence.The prompts control the evidence’s textual style.Source: §2.4 · p. 3

2.5 Controlling Data Quality

  • Quality controls filter refusal-like responses, retain evidence with strong claim entailment, and remove evidence pairs classified as non-conflicting.The conflict classifier achieved over 95% accuracy in a manual evaluation of 200 random examples.Source: §2.5 · p. 4

3 Experiments

The experiments show how model responses vary with conflicting retrieved evidence, conflicts embedded in model knowledge, evidence order, and question descriptions. They also examine how model scale and conflict type relate to these responses.

3.1 Experimental Setup

  • The evaluation covers 12 LLMs across four series, uses continued pre-training to study internal conflicts, and measures reliance on parametric memory with MR.The three continued-pretraining models are QWEN1.5-4B, MISTRAL-7B, and LLAMA3-8B; lower MR indicates greater adoption of constructed conflicting knowledge.Source: §3.1 · pp. 4–5

3.2 Conflicts in Retrieved Knowledge

  • All models have MR below 50% with only contradictory external evidence, but above 50% when evidence also matches parametric memory.The authors interpret this pattern as receptivity to external evidence in the first setting and confirmation bias when evidence is mixed.Source: §3.2 · p. 5
  • MR is lower for temporal and semantic conflicts than misinformation; semantic conflicts have the lowest MR under solely contradictory evidence.For LLAMA2-70B with two pieces of evidence, temporal and semantic conflict MR values are 7.77% and 3.73%, versus 9.45% for misinformation.Source: Figure 3 · p. 5, §3.2 · p. 6
  • Within the LLaMA2 series, MR decreases as model size increases, and models tend to favor later evidence; evidence order also matters when internal conflicts are present.With conflicting memory later in the sequence, MR is 56.44% for LLaMA2-7B and 38.97% for LLaMA2-70B.Source: §3.2 · p. 6, Figure 5 · p. 6

3.3 Conflicts in Embedded Knowledge

  • Introducing conflicting training data lowers OAR, with larger proportions of conflict data producing greater declines.At a 2:1 ratio of default to conflicting data, OAR decreases by 41.5% for LLAMA2-7B and 37.6% for MISTRAL-7B.Source: §3.3 · pp. 6–7
  • Despite those internal conflicts, models retain their original performance when given default evidence, choosing the default answer regardless of the amount of conflicting data introduced.The experiment compares answering without evidence against answering with the original evidence prepended to the question.Source: Figure 6 · p. 6, §3.3 · p. 7

3.4 Interplay among the Conflicts

  • With internal conflicts, QWEN1.5-4B follows external knowledge and shows an even stronger preference for later-positioned external evidence.Its preference for later-positioned conflict evidence is reported as 77.52% versus 82.08%.Source: Figure 7 · p. 7, §3.4 · p. 8

3.5 Detailed Description Can Make the LLMs’ Objectives More Explicit

  • Adding temporal or semantic descriptions to questions lowers MR with or without internal conflicts.For LLAMA3-70B, MR drops from 67.52% to 8.31% with descriptions; with a 2:1 internal conflict ratio, it falls from 55.37% to 41.80%.Source: Figure 8 · p. 7, §3.5 · p. 8

4 Related Work

Prior work largely focuses on retrieved knowledge conflicts, while ConflictBank examines retrieved and embedded conflicts and their interactions. Existing datasets use substitution and generative methods, but address internal conflicts and complex scenarios less often.

Taxonomy of knowledge conflicts

  • ConflictBank extends work focused mainly on retrieved conflicts by investigating both retrieved and embedded conflicts and their interactions.Retrieved conflicts pit external information against model knowledge; embedded conflicts arise from discrepancies within training text.Source: §4 · pp. 8–9

The Causes of Knowledge Conflicts

  • ConflictBank covers misinformation, temporal updates, and semantic variation, broadening beyond misinformation-focused prior work.The paper identifies changing knowledge and multiple word meanings as additional real-world sources of conflict.Source: §4 · p. 8

Knowledge Conflicts Datasets

  • Earlier conflict datasets use entity-level substitution, LLM-based generation, or combinations of the two, but mainly target retrieved conflicts.Xie et al. [Xie et al., 2024] combined substitution and generation to create more coherent conflicting pairs.Source: §4 · pp. 8–9

5 Conclusion

ConflictBank is a large, diverse benchmark for studying misinformation, temporal, and semantic conflicts across retrieved, embedded, and interacting knowledge. Its pilot experiments examine LLM behavior across these scenarios, with the stated aim of supporting more trustworthy models.

5 Conclusion

  • ConflictBank comprises 553K QA pairs and 7M conflict evidence items covering misinformation, temporal updates, and semantic variations.The QA pairs support model evaluation, while the evidence supports simulating conflicts in pre-training and inference.Source: §5 · p. 9
  • The benchmark supports pilot experiments on embedded conflicts in pre-training, retrieved conflicts at inference, and their interplay.The authors aim for a detailed understanding of how models handle conflict information to guide development of more robust and trustworthy language models.Source: §5 · p. 9, §A.2 · p. 15

A.2 Motivation

  • ConflictBank is designed for equitable comparison across conflict effects, addressing existing datasets’ narrower focus on particular conflict types.The motivation is to study impacts on model behavior and reliability across varied real-world scenarios.Source: §A.2 · p. 15

A.3 Limitation

ConflictBank uses generative methods to construct conflict pairs efficiently, and the authors identify exploring other construction methods as future work. The dataset is based on public-domain Wikidata, intended for scientific research, and may contain offensive model outputs.

A.3 Limitation

  • The authors use generative methods to build conflict pairs efficiently and plan to explore other construction methods to verify dataset robustness.They note that extracting conflicts from pre-training corpora is challenging because of the data volume.Source: §A.3 · p. 15

A.4 Ethics Statement

  • The dataset is based on public-domain Wikidata, and the authors state they will release it under the same license..Source: §A.4 · p. 15
  • The dataset’s stated scope is scientific research, and model-output contexts may be offensive and do not reflect the authors’ views..Source: §A.4 · p. 15

B Dataset Details

The section presents a complete example of ConflictBank in Table 1.

B Dataset Details

B.1 Comparison of CONFLICTBANK and Prior Datasets

CONFLICTBANK is compared with prior conflict datasets, while the following subsections report processing time and data volume and describe templates for turning Wikidata facts into claims and questions.

B.1 Comparison of CONFLICTBANK and Prior Datasets

  • CONFLICTBANK is the first benchmark described here to cover three conflict causes and evaluate retrieved knowledge, embedded knowledge, and their interactions.The comparison with prior datasets is presented in Table 2.Source: §B.1 · p. 15

B.2 Running time

  • Table 3 reports the running time and data volume after each CONFLICTBANK processing step.It also specifies that retained claim-evidence pairs passing NLI checks are used to form QA pairs after conflict confirmation.Source: §B.2 · p. 15

B.3 CONFLICTBANK Templates

  • Table 4 lists templates for converting Wikidata facts into natural-language claims and questions.Source: §B.3 · p. 16

B.4 Human Evaluation

Five volunteers checked claim-evidence entailment and evidence contradictions on sampled examples; the authors report few ambiguous conflicts and conclude that their checks support dataset integrity.

B.4 Human Evaluation

  • Five volunteers each assessed 200 randomly selected examples for claim-evidence entailment and contradiction between default and conflict evidence.The two checks asked whether evidence logically supported its claim and whether default and conflict evidence contradicted each other.Source: §B.4 · p. 16
  • Only one of 200 assessed examples was judged ambiguously conflicting, and the authors retained it because the overall conflict remained unaffected.The example involved evidence mentioning Daniel Rousse’s work at both Technical University of Liberec and École de technologie supérieure.Source: §B.4 · p. 16
  • The authors conclude that the confirmation classifier and NLI model support the integrity and suitability of the conflict pairs for studying LLM knowledge conflicts.Source: §B.4 · p. 16

C Experimental Details

The experiments cover four model series, with selected models spanning parameter sizes from 0.5B to 70B.

C Experimental Details

  • The study evaluates 12 representative models across Gemma, LLaMA2, LLaMA3, and Qwen1.5, selecting sizes from 0.5B to 70B.Selected sizes are Gemma 2B and 7B; LLaMA2 7B, 13B, and 70B; LLaMA3 7B and 70B; and Qwen1.5 0.5B, 4B, 7B, 14B, and 70B.Source: §C.1 · p. 16

C.2 Implementation Details

The implementation details describe continued pretraining for three models under four conflict ratios, while the appendix lists templates and prompts for benchmark construction and evaluation.

C.2 Implementation Details

  • Three models were continued-pretrained at conflict ratios 1:0, 2:1, 1:1, and 2:3 for 4,500 steps each, covering 1.06 billion tokens.The models were Qwen1.5-4B, Mistral-7B, and LLaMA3-8B; training used a learning rate of 2e-5 and batch size 256.Source: §C.2 · p. 16

D LLM Prompts for Different Steps

  • The appendix provides prompts for generating semantic descriptions and default, misinformation, temporal, and semantic conflict evidence, plus prompts for no-evidence, conflict-evidence, and mixed-evidence evaluation.The evaluation figures use temporal conflict as an example for conflict evidence and combine temporal-conflict with default evidence for mixed evidence.Source: D LLM Prompts for Different Steps · p. 17, Figure 9 · p. 19, Table 6 · p. 21
  • Table 4 gives templates for converting Wikidata facts into natural claims and questions.Table 1 presents a complete example from CONFLICTBANK.Source: Table 1 · p. 18, Table 4 · p. 20
  • Tables 6–9 show prompts for generating evidence from default, misinformation, temporal, and semantic claims and descriptions.Source: Table 6 · p. 21, Table 7 · p. 22, Table 8 · p. 22

Related papers

Loading 2408.12076v1…