PaperlayerView exact revision on arXiv

Source-linked AI summary

Whose Facts Win? LLM Source Preferences under Knowledge Conflicts

Jakob Schuster, Vagrant Gautam, Katja Markert

arXiv:2601.03746v3cs.CL

TL;DR

Prior work examined several factors shaping how LLMs resolve conflicts between retrieved contexts, but not the information source. This paper evaluates source preferences with synthetic conflicts and sources, finding a credibility hierarchy that repetition can reverse and a fine-tuning method that reduces repetition bias.

  • Problem

    Although retrieved contexts can conflict and prior work examined relevance, parametric knowledge, frequency, and LLM-generated information, the role of information sources in LLM conflict resolution had not been studied.

    Source: §1 · p. 1

  • Method

    The authors introduce a credibility-grounded framework and evaluate 13 models using controlled multiple-choice questions with synthetic conflicts and sources to isolate source preferences.

    Source: §1 · p. 2

  • Results

    LLMs show a consistent credibility hierarchy favoring institutionally corroborated sources, but repeating low-credibility information can flip preferences; fine-tuning reduces repetition bias by up to 79.2% while retaining at least 72.5% of original preferences.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Synthetic sources isolate source effects, while real-world credibility rankings may vary with message style, topic, and source expertise.

    Source: §1 · p. 2, Limitations · p. 10

    The study uses entirely synthetic scenarios, and the authors note that source hierarchies may vary with message style, topic, and source expertise.

    Source: Limitations · p. 10

Abstract

from arXiv · show

As large language models (LLMs) are more frequently used in retrieval-augmented generation pipelines, it is increasingly relevant to study their behavior under knowledge conflicts. Thus far, the role of the source of the retrieved information has gone unexamined. We address this gap with a novel framework to investigate how source preferences affect LLM resolution of inter-context knowledge conflicts in English, motivated by interdisciplinary research on credibility. By using synthetic sources, we study preferences for different types of sources without inheriting the biases of specific real-world sources. With a comprehensive, tightly-controlled evaluation of 13 open-weight LLMs, we find that LLMs prefer institutionally-corroborated information (e.g., government or newspaper sources) over information from people and social media. However, these source preferences can be reversed by simply repeating information from less credible sources. To mitigate repetition effects and maintain consistent preferences, we propose a novel method that reduces repetition bias by up to 79.2%, while also maintaining at least 72.5% of original preferences. We release all data and code to encourage future work on credibility and source preferences in knowledge-intensive NLP.

1 Introduction

The paper studies how information sources affect LLM resolution of knowledge conflicts, a factor not examined in prior work on such conflicts. Using synthetic conflicts and sources, it finds a consistent credibility hierarchy that repetition can overturn.

1 Introduction

  • LLMs follow a consistent source-credibility hierarchy, but repeating information from low-credibility sources can flip their preferences.Figure 1 summarizes the hierarchy, repetition effect, and method’s mitigation.Source: Figure 1 · p. 1, §1 · p. 2
  • Earlier conflict studies examined relevance, alignment with parametric knowledge, information frequency, and LLM-generated information, but not the information source.LLMs are also used in retrieval-augmented generation to ground answers in external evidence, though retrieved contexts can conflict.Source: §1 · p. 1
  • The framework evaluates 13 models using synthetic conflicts and sources to isolate source preferences from pre-training knowledge and real-world source biases.Synthetic sources represent abstract source types, while the conflicts are tested in a controlled multiple-choice question-answering setting.Source: §1 · pp. 1–2
  • The proposed fine-tuning method mitigates repetition bias by up to 79.2% while maintaining at least 72.5% of original source preferences.The reported result also holds for source types not encountered in training.Source: §1 · p. 2
  • The authors frame source preferences as a research dimension with implications for RAG, question answering, and a trustworthy information ecosystem.The introduction notes that LLMs can fabricate incorrect statements when relying on parametric knowledge.Source: §1 · pp. 1–2

2 Background: Credibility

Credibility judgments involve multiple parts of communication, but this paper focuses on attitudes toward the source of a message. It examines source credibility in LLM decisions while controlling for message credibility and other evaluation challenges.

2 Background: Credibility

  • Credibility judgments can involve the source, message, medium, and recipient; this paper focuses specifically on attitudes toward the message’s originating entity.The paper distinguishes source credibility from other components of communication.Source: §2 · p. 2
  • Early credibility studies compared conflicting reports across print, television, and radio, while later work disentangled credibility dimensions and included the internet.The cited research includes Hovland and Weiss (1951), Roper (1985), Whitehead Jr. (1968), McCroskey and Young (1981), and Flanagin and Metzger (2000).Source: §2 · p. 2
  • The study investigates presumed and reputed credibility across long-studied contrasts such as newspapers, government, and social media.Presumed credibility concerns general assumptions about a source; reputed credibility concerns judgments based on third-party reports.Source: §2 · p. 2
  • Equally plausible factual conflicts hold message credibility constant, allowing the study to isolate source credibility in LLMs.Source: §2 · p. 3
  • The evaluation must control for position bias, token bias, result stability, and preferences learned during pre- or post-training.Source: §2 · p. 3

3 Data and Methodology

The section develops a synthetic-conflict and synthetic-source framework for evaluating source preferences across 13 open-weight models.

3.2 Synthetic Sources

  • Four fictional source types represent contrasts studied in credibility research, while avoiding reliance on real-world source identities.The section introduces newspaper, government, social-media-user, and person sources.Source: §3 · p. 3, §3.2 · p. 3

3.1 Plausible Knowledge Conflict Pairs

  • The evaluation uses 7,440 conflict pairs from 373 NeoQA entities and 1,903 unique counterfactually perturbed attribute values, with each variant differing from its original in one attribute value.Alternatives were manually verified, and proper-noun values with English Wikipedia articles were removed.Source: §3.1 · p. 3

3.3 Evaluation Method

  • The study evaluates 13 instruction-tuned open-weight models from four families using forced-choice questions about conflicting table-based facts.Prompts attribute the tables to source instances or provide no source, and use answer-index tokens; table order is also reversed to control position bias.Source: §3.3 · pp. 3–4
  • The source-preference metric measures how source attribution changes the probability assigned to answer A relative to an unattributed baseline.Positive scores indicate that source x increases support for A more than source y supports B.Source: §3.3 · p. 4

4 LLM Source Preferences

The section examines source-type and source-feature preferences, whether prompted credibility judgments align with behavior, and whether preferences persist when models can abstain. Models show consistent source hierarchies, while popularity, prompting, and abstention also shape conflict resolution.

4.1 Inter-Type Source Preference Behavior

  • Compared with no source available, all models prefer corroborated information; under conflicts between different source types, they show strictly transitive preferences, with 11 of 13 favoring both institutional sources over both individual sources.Inter-model rankings had average Kendall’s W of 0.74; the overall hierarchy places government above newspaper, followed by individuals.Source: §4.1 · p. 4, Figure 4 · p. 5

4.2 Intra-Type Source Preference Behavior

  • All models tend to favor more popular sources, while sociodemographic factors have only a weak effect on behavioral source preferences.Popularity is measured using newspaper circulation and social-media follower counts; most models slightly favor regional sources, academic titles, women, and older people, while username-style preferences are mixed.Source: §4.2 · pp. 5–6, Figure 5 · p. 5, Figure 6 · p. 5

4.3 Prompted Preferences vs. Behavior

  • Prompting strengthens same-direction preferences in 139 of 195 cases, but flips significant behavioral preferences in 38 cases; inter-model agreement rises from Kendall’s W 0.59 to 0.77.Behavioral evaluation remains more consistent with model use.Source: §4.3 · p. 6, Figure 7 · p. 6

4.4 Abstention Under Conflicting Contexts

  • Most models abstain on only a minority of inputs, and source preferences remain transitive when abstention is available; 11 of 13 models retain the forced-choice hierarchy.Models often show only marginal differences when attributed sources are added, despite having too little information to resolve the conflict.Source: §4.4 · p. 6

5 Credibility vs. Majority vs. Repetition

This section tests whether a majority or mere repetition can outweigh source credibility, and whether prompting models to attend to credibility and repetition restores their original preferences.

5 Credibility vs. Majority vs. Repetition

  • A two-table social-media majority flips every model’s preference toward the previously low-credibility sources (average d_SP gap 33.90), while a one-table majority leaves government preferences largely intact (gap 6.17).The two-table setup uses two identical tables attributed to different social-media sources; the one-table setup combines them under one header.Source: §5 · p. 7, Figure 8 · p. 7
  • Repeating information from one social-media source also flips preferences, despite adding no new source (average d_SP gap 30.04); repeated unattributed information likewise reverses prior preferences.When neither conflicting claim has a source, repetition bias is even stronger, and credibility takes a backseat to repetition.Source: §5 · p. 7, Figure 9 · p. 7
  • Credibility-aware prompting strengthens original source preferences and mitigates repetition bias more than a two-table majority, but usually fails to restore consistency with the original hierarchy.The prompting instruction asks models to assess source credibility and not let simple repetition influence their choice.Source: §5 · p. 8, Figure 10 · p. 8

6 Mitigating Repetition Bias

The section proposes teacher-student distillation to reduce preference changes caused by repeated information while preserving the base model’s source preferences.

6 Mitigating Repetition Bias

  • Training uses 1,500 conflict pairs from 12 seed entities, then evaluates on 361 held-out entities with previously unseen government and social-media sources.The reported setup averages seven seeds with randomly generated held-out training entities.Source: §6 · p. 8
  • The method distills a frozen base model into a LoRA student using weighted KL losses: mimic the teacher without repetition and penalize student deviations with repetition.The loss uses λ = 0.2; the teacher-student objective targets preference differences between inputs with and without repeated information.Source: §6 · p. 8
  • 79.2%: the government-versus-no-source d_SP gap fell from 45.9 to 9.55, while 72.5% of the original preference remained (29.6 to 21.5).For government-versus-social-media conflicts, repetition bias fell 78.3% (33.2 to 7.2); the method generalized to source types unseen during training.Source: §6 · p. 8, Figure 11 · p. 9

7 Related Works

Related work situates this paper across research on knowledge conflicts, credibility, and cognitive biases in NLP. The authors distinguish their focus on source credibility in inter-context conflicts from prior work in each area.

Knowledge conflicts

  • Knowledge-conflict research spans parametric-context conflicts, conflicts within parametric knowledge, and conflicts within contexts; prior studies also address repetition and position or source citation in conflict resolution.The authors identify source credibility in inter-context knowledge conflicts as unexamined to their knowledge.Source: §7 · p. 9

Credibility

  • NLP credibility research includes argument quality, rumours, fake news, and low-quality science, while QA/RAG studies examine evidence persuasiveness, message features, noisy information, citation preferences, and source selection.The paper contrasts its study of inherent judgments with work training models to incorporate credibility during generation.Source: §7 · p. 9

Biases

  • Relevant bias research includes bandwagon, illusory-truth, and authority effects, but prior work had not examined these in source credibility during inter-context conflicts.The paper also relates its work to studies of interacting biases and repetition with gender stereotypes, while noting that it does not study format biases.Source: §7 · p. 9

8 Conclusion

In synthetic experiments, source characteristics shaped how models resolved knowledge conflicts, revealing a source hierarchy that repetition could overturn. A fine-tuning method reduced repetition bias while preserving source preferences.

8 Conclusion

  • Repeated information can flip source preferences, and prompting alone cannot sufficiently mitigate this repetition bias.The proposed fine-tuning method, combined with credibility prompting, teaches repetition invariance while preserving original source preferences.Source: §8 · p. 9
  • Models favored sources with higher presumed and reputed credibility, even when allowed to abstain.Their behavior differed from human-like credibility judgments under sociodemographic variation.Source: §8 · p. 9
  • In synthetic experiments isolating LLM source preferences, a novel fine-tuning method combined with credibility prompting teaches models repetition invariance while preserving their original source preferences.Source: §8 · p. 9

Limitations

The study uses synthetic, English-language scenarios and forced-choice evaluation, so its findings have several stated boundaries. The authors identify realistic RAG settings and varied content and cultural contexts as areas where results may differ or remain untested.

Limitations

  • The source-aware mitigation strategy has not been shown to generalize to different conflicts, contexts, or tasks in realistic settings.The authors note that deduplicating a synthetic knowledge base is simpler than deduplication in realistic RAG, where source-agnostic deduplication may be inappropriate.Source: Limitations · p. 10
  • The evaluation uses forced-choice questions without step-by-step reasoning or generative answers, and alternate evaluation strategies could produce different results.Source: Limitations · p. 10
  • The tasks cover factual conflicts with equally plausible alternatives, not sentiment-based questions or conflicts between claims with differing plausibility or source authority.Tabular presentation reduces message-style effects, while interactions between other content aspects and source credibility remain for future work.Source: Limitations · p. 10
  • Experiments use English data and prompting and some U.S.-based preferences; other languages may evoke different credibility behavior and trust hierarchies.Source: Limitations · p. 10

Ethics Statement

The paper takes a descriptive view of source credibility and uses synthetic conflict pairs to study source effects. Its appendices describe how alternatives were created, account for position bias, and validate the experimental setup.

Ethics Statement

  • The authors describe credibility rather than prescribing a universal source hierarchy, noting that institutions can be captured and marginalized voices may be more credible in some contexts.They also reject treating LLMs as entities that introspect on beliefs; they frame learned credibility preferences as reflected in training data.Source: Ethics Statement · p. 10

A Creation of Conflict Pairs

  • Conflict pairs combine original NeoQA values with counterfactual alternatives that differ in one attribute value.Four alternatives are generated per entity-attribute value.Source: A Creation of Conflict Pairs · p. 16

A.1 Rescaling

  • Non-date numerical attributes are adjusted by up to ±20%, year-only dates by up to ±30 years within 1850–2025, and exact dates by up to ±365 days.Numbers with five or more digits are rounded to the third most significant decimal place.Source: §A.1 · p. 16

A.2 Sampling

  • For attributes with few plausible values, alternatives are generated with GPT-4.1 and manually filtered, then one value is sampled from the set.Source: §A.2 · p. 16

A.3 Generation

  • QWEN2.5-70B generates alternatives for attributes with many plausible values, such as professions and organizational headquarters.Source: §A.3 · p. 16

B Position Bias

  • Source preferences overcame strong position bias in some models, despite a negative correlation of -0.4 Spearman’s ρ between position bias and source preference.The experiments varied table order so models had to overcome position bias to express a source preference.Source: B Position Bias · p. 18
  • Position bias is measured as the source-independent probability of choosing answer token B, which denotes the second table, over token A.The authors retain models with strong position bias and present all possible table orders in source-preference experiments.Source: B Position Bias · p. 18

C Setup Validation; C.1 Setup

  • Setup validation tests whether models recognize synthetic source types, parse Markdown tables, follow answer-format instructions, and treat counterfactual values as plausible.The tests use source recognition prompts, single-table questions, generated answers, and source-free comparisons with original NeoQA values.Source: C Setup Validation · p. 18, §C.1 · pp. 18–19

C.2 Results

The setup checks indicate that models could recognize the synthetic sources, use the tables and answer format, and regard perturbed values as similarly plausible. The appendices also provide example prompts for studying source preferences.

C.2 Results

  • All models assigned synthetic source instances to their intended types, parsed tables, and used the context to answer in the required format.Source: §C.2 · p. 19
  • Created alternatives were about as plausible as the original NeoQA values, with small model-dependent variations.Source: §C.2 · p. 19

D Example Prompts: LLM Source Preferences

  • The example direct-preference prompt samples its question from the 20 questions used in the prompted-preference experiments.The prompt compares government and newspaper sources.Source: D Example Prompts: LLM Source… · p. 20, Table 4 · p. 20, Figure 16 · p. 20
  • Figure 17 compares attributed with unattributed information across answer-token variations; its caption reports that all models prefer attributed information.The caption says these results parallel Figure 3.Source: Figure 17 · p. 21

E Result Stability over Multiple Prompts

The induced credibility hierarchy is largely stable across alternative ranking methods and prompt variations, though lowering source emphasis creates ambiguity and weakens some preference measurements.

E Result Stability over Multiple Prompts

  • The stability tests also varied answer tokens, rephrased the instruction, and reduced source emphasis.These perturbations were designed to test whether preferences and induced hierarchies persisted across prompts.Source: E Result Stability over Multiple Prompts · p. 21

E.1 Alternative Induction of a Credibility Hierarchy

  • 10 of 13 models had the same rankings under attribution-based and match-up-based methods, with average Kendall’s τ of 0.87.The attribution-based method orders source types by their d_SP against “No source available”; the main method uses direct matchups.Source: §E.1 · p. 21

E.2 Different Answer Options

  • With answer tokens 1 and 2, 12 of 13 models produced the same ranking under both methods, with average Kendall’s τ of 0.97.The single-transferable-vote hierarchy remained identical to the main results.Source: §E.2 · pp. 21–22

E.3 Different Instruction

  • After rephrasing the instruction, 9 of 13 models produced the same ranking under both methods, with average Kendall’s τ of 0.90.The single-transferable-vote hierarchy remained identical to the main results.Source: §E.3 · p. 22

E.4 Prompt with Lower Source Focus

  • The lower-source-focus prompt removes source mentions and omits “No source available” from unattributed tables, making their provenance ambiguous.The authors therefore deem this format less reliable for evaluating preferences for corroborated information.Source: §E.4 · p. 22
  • Under lower source focus, 11 of 13 models favored both institutional sources in matchups, while 9 of 13 did so using attribution-based rankings.The attribution-based method showed weaker source effects; the two methods had average Kendall’s τ of 0.87, and their single-transferable-vote hierarchy matched the main results.Source: §E.4 · pp. 22–23

F Further Details: Intra-Type Source Conflicts

The appendix reports further source-conflict findings, compares prompting strategies for reducing repetition bias, and details the fine-tuning data and evaluation setup.

F Further Details: Intra-Type Source Conflicts

  • Replacing newspaper circulation with Article ID removes the source preference, supporting attribution of the earlier preference to source popularity.The study also tests low- and high-circulation newspapers and social-media accounts with different follower counts.Source: F Further Details: Intra-Type Source… · p. 23
  • Young people are least preferred in direct age matchups, while preferences between old and middle-aged people vary by model.The age contrasts control for gender, and the gender contrasts control for age.Source: F Further Details: Intra-Type Source… · pp. 23–24, Figure 23 · p. 24

G Full Results: Prompted Preferences

H Example Prompts: Credibility vs. Majority vs. Repetition

F Further Details: Intra-Type Source Conflicts

I Repetition Prompting for All Models

J Credibility Prompting for All Models

K Fine-tuning-based Mitigation: Details

  • Fine-tuning uses newspaper and person sources for training, then tests on unseen government and social-media sources with no training-data overlap.The data comprise 1,275 training inputs, 225 validation inputs, and conflicts drawn from the remaining 361 seed entities for testing.Source: K Fine-tuning-based Mitigation: Details · p. 30

Related papers

Loading 2601.03746v3…