PaperlayerView exact revision on arXiv

Source-linked AI summary

Task Matters: Knowledge Requirements Shape LLM Responses to Context-Memory Conflict

Kaiser Sun, Fan Bai, Mark Dredze

arXiv:2506.06485v4cs.CLcs.AI

TL;DR

Prior work has largely examined context–memory conflict in contextual question answering, leaving unclear how models behave when tasks require different kinds of knowledge. This paper holds underlying knowledge constant while varying task formulations and finds that conflict effects depend on task demands and plausibility, with context-reliance strategies helping some tasks but harming others. The findings also raise concerns about model-based evaluation and motivate task-aware balancing of context and memory.

  • Problem

    Most prior studies examine context–memory conflict in contextual question answering, leaving unclear how models behave across tasks with different knowledge requirements.

    Source: §1 · p. 2

  • Method

    The diagnostic framework holds underlying knowledge constant while varying task formulations and conflict levels across different knowledge-use requirements.

    Source: §1 · p. 2

  • Results

    Conflict has little impact on tasks requiring minimal knowledge but substantially impairs knowledge-intensive tasks, while context reiteration can help context-reliant tasks and harm tasks requiring parametric knowledge.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    The findings motivate task-aware approaches that dynamically balance contextual and parametric knowledge rather than blindly enforcing either source.

    Source: §5 · p. 9

    LLM-assisted instance generation or editing may introduce artifacts and shared linguistic priors that overestimate performance, especially when similar models are evaluated.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large language models (LLMs) draw on both contextual information and parametric memory, yet these sources can conflict. Prior studies have largely examined this issue in contextual question answering, implicitly assuming that tasks should rely on the provided context, leaving unclear how LLMs behave when tasks require different types and degrees of knowledge utilization. We address this gap with a model-agnostic diagnostic framework that holds underlying knowledge constant while introducing controlled conflicts across tasks with varying knowledge demands. Experiments on representative open-weight and proprietary LLMs show that performance degradation under conflict is driven by both task-specific knowledge reliance and conflict plausibility; that strategies such as rationales or context reiteration increase context reliance, helping context-only tasks but harming those requiring parametric knowledge; and that these effects bias model-based evaluation, calling into question the reliability of LLMs as judges. Overall, our findings reveal that context-memory conflict is inherently task-dependent and motivate task-aware approaches to balancing context and memory in LLM deployment and evaluation.

1 Introduction

Context–memory conflict is task-dependent, but prior work has largely studied contextual question answering, leaving unclear how models should behave when tasks require different kinds of knowledge. The paper introduces a diagnostic framework that holds underlying knowledge constant while varying task formulations and finds that conflict affects tasks differently.

1 Introduction

  • Prior studies focused largely on contextual question answering, leaving uncertain how conflict affects tasks with different knowledge demands.Such tasks range from extraction to integrating background knowledge with new information.Source: §1 · pp. 1–2
  • The diagnostic framework holds underlying knowledge constant while varying task formulations across four knowledge-use requirements.It tests surface extraction, contextual grounding, parametric reliance, and integration of both sources.Source: §1 · p. 2
  • Conflict minimally affects tasks requiring little or no knowledge, but significantly impairs knowledge-intensive tasks, even under explicit instructions.Context reiteration improves context-reliant tasks but degrades performance when parametric knowledge should dominate; model-based evaluations can also be biased by parametric beliefs.Source: §1 · pp. 2–3

2 Related Work

Prior studies report varied model responses to context–memory conflict, and most focus on contextual question answering. This paper varies task formulation while holding knowledge constant to examine conflict across different knowledge demands.

2 Related Work

  • Prior findings differ: models may favor parametric beliefs, update when evidence is convincing, or exhibit availability, majority, and confirmation biases.In realistic documents, models update more reliably than synthetic evaluations suggest, yet may retain an incorrect belief when it appears in context.Source: §2 · p. 3
  • Proposed mitigations include attention interventions, decoding methods, and inference-time controls that steer reliance toward context or memory.The approaches include pruning or patching attention heads, two-step interventions, proxy-model steering, and test-time attention interventions.Source: §2 · p. 3
  • Because prior work emphasizes context-heavy question answering, it leaves unresolved whether conflict has the same impact across tasks with different knowledge demands.The paper addresses this gap with controlled datasets that keep underlying knowledge constant while varying task formulation.Source: §2 · p. 3

3 Context-Memory Conflict Creation

The framework collects model-aligned beliefs, creates controlled evidence conflicts, and assigns the resulting instances to tasks with different knowledge requirements. It uses multiple prompt variations to make belief probing more conservative and validates task annotations.

3 Context-Memory Conflict Creation

  • The framework treats a model’s belief as established only when it consistently selects one answer across prompt variations and rejects conflicting alternatives.This stance-consistency filter mitigates, but does not eliminate, sycophancy; the resulting belief set is treated as a conservative lower bound.Source: §3 · pp. 3–4

3.1 Evidence Creation

  • Plausibility is willingness to consider an understood, coherent, relatively simple alternative deemed viable, assessed by common-sense alignment and logical consistency.The evidence types include no contradiction and high- or low-plausibility contradictions; original fact-checked answers provide highly plausible alternatives, while edited passages add variants.Source: §3.1 · p. 4

3.2 Task Annotation

  • Five tasks distinguish knowledge-free extraction, contextual knowledge, parametric knowledge, combined parametric-contextual knowledge, and retrieval-augmented generation.The tasks differ in the source of knowledge expected and whether the model should provide one answer or discuss both perspectives.Source: §3.2 · pp. 4–5
  • The authors changed non-extractive tasks to multiple choice because model-based judging of free-text answers was susceptible to knowledge conflict.Two LLMs reviewed task-type annotations, with a subsample checked by human annotators.Source: §3.2 · p. 5

4 Findings

Context–memory conflict has different effects across tasks, harming knowledge-intensive tasks more than knowledge-free ones. Rationales and repetition can shift context reliance, while parametric knowledge can bias model-based evaluation.

4.1 Conflict Impairs Model Performance on Knowledge-Intensive Tasks

  • Across knowledge-use tasks, conflict degrades performance but has minimal impact on knowledge-free tasks; GPT-5.2 PK accuracy falls from 89.2 (NC) to 33.7 (HPC), recovering to 71.8 (LPC).Source: §4.1 · pp. 5–6, Figure 3 · p. 6
  • Across evaluated models, RAG accuracy is higher on (NC, LPC) pairs than on (NC, HPC) pairs.Source: Table 1 · p. 6, §4.1 · p. 7

4.2 Rationales and Reiteration

  • Rationales improve CK but hurt PK, whereas context repetition improves CK without similarly distracting PK; remaining errors favor answers repeated more often in context.Source: §4.2 · pp. 7–8

4.3 Conflict leads to unreliable judge LLMs

  • κ=0.79 (GPT-4o–human) versus κ=0.90 (human–human); evaluator models can favor parametric knowledge and misjudge contextually correct answers.Source: §4.3 · pp. 8–9
  • Blindly increasing context reliance may raise prompt-injection risk.Source: §4.3 · p. 8

5 Conclusion

The conclusion frames context–memory conflict as task-dependent and presents a framework that varies task formulations while holding underlying knowledge constant. It also highlights the trade-off in increasing context reliance and motivates balancing contextual and parametric knowledge.

5 Conclusion

  • Holding underlying knowledge constant while varying task formulations provides a unified explanation for previously fragmented findings on task-dependent conflict.Source: §5 · p. 9
  • Reiteration can increase context reliance, benefiting context-driven tasks but harming tasks where parametric knowledge should dominate; parametric bias can also undermine model-based evaluation.Source: §5 · p. 9
  • The conclusion cautions against blindly enforcing context adherence or trusting only parametric knowledge, and motivates dynamically balancing both at inference time.Source: §5 · p. 9

Limitations

The diagnostic instances may not fully reflect real-world task distributions, and the study’s model coverage and treatment of knowledge limit the scope of its conclusions.

Limitations

  • LLM-assisted generation or editing may introduce biases, hallucinations, or artifacts that do not reflect real-world task distributions.Using a similar model as both editor and evaluator may also overestimate performance through shared linguistic priors.Source: Limitations · p. 9
  • The model analysis includes five open-weight models and one proprietary frontier model; broader proprietary-model coverage is left to future work.The listed models cover diverse training paradigms and roughly 20× variation in scale.Source: Limitations · p. 9
  • The unsettled, multifaceted definition of knowledge complicates reliable measurement of how much knowledge tasks involve.The paper loosely treats extractive QA as knowledge-free, while noting that all model behavior is associated with learned parameters encoding parametric knowledge.Source: Limitations · p. 9

Ethical Considerations

The paper discusses ethical risks in evaluating and deploying models under context–memory conflict, and describes how it queries models’ parametric knowledge to construct diagnostic instances.

Ethical Considerations

  • The diagnostic framework is for analysis rather than adding model capabilities, and the paper flags evaluation and deployment considerations.It examines how models resolve conflicts between contextual information and parametric knowledge.Source: Ethical Considerations · p. 9
  • Model evaluators can favor parametric knowledge over conflicting context, while indiscriminately increasing context reliance may harm parametric-knowledge tasks and increase prompt-injection vulnerability.The paper calls for task-aware evaluation and deployment that account for different knowledge requirements; its datasets use existing human-cleaned benchmarks and synthetic edits, not personal, sensitive, or human-subject data.Source: Ethical Considerations · p. 9

A Parametric Knowledge Query

  • To identify a model’s parametric answer, the query process tests both candidate answers and retains an instance only when the model judges one answer to be the sole correct option.The selected answer is assigned as the No Contradiction passage.Source: A Parametric Knowledge Query · p. 12

B Verification of Data Validity

The data-validity procedure combines model-based checks with human review of a sampled subset.

B Verification of Data Validity

  • Every diagnostic instance was checked by GPT-4o and DeepSeek-R1-Distill-Llama-70B, and retained only when both answered correctly using the gold evidence.Source: B Verification of Data Validity · p. 12
  • For each test model—OLMo, Mistral, and Qwen—annotators manually reviewed 30 randomly selected model-verified instances.They checked evidence support and gold-answer correctness for the corresponding task.Source: B Verification of Data Validity · p. 12
  • All sampled instances passed human verification, which the authors suggest likely reflects the strict preceding model-based filter.Source: B Verification of Data Validity · p. 12

C Dataset Statistics

The diagnostic datasets are model-specific, and each base question is represented under four evidence conditions.

C Dataset Statistics

D Task Examples

The appendix points to examples of each task and reports raw model performance using F1 and exact match.

D Task Examples

  • Examples of each task are shown in Figures 7 and 8.These figures accompany the task examples.Source: D Task Examples · p. 13

E Raw Performance

  • Raw performance is measured with F1 and exact match, with full model results shown in Table 4.The passage introduces the two metrics across settings.Source: E Raw Performance · p. 13

E.1 Highly Confident Instances

The highly confident-instance analysis retains samples answered perfectly in the NC condition and reports that the broader dataset’s overall trends persist.

E.1 Highly Confident Instances

  • Table 5 reports model performance on the highly confident instances.Source: Table 5 · p. 13
  • Only instances answered with 100% accuracy in the NC condition are retained, restricting analysis to fully mastered samples.The NC responses to parametric-knowledge queries are collected in binary stance format.Source: §E.1 · p. 13
  • Overall trends persist on highly confident instances, although absolute numbers vary slightly.The analysis restricts the comparison to items each model answered with 100% accuracy under NC.Source: §E.1 · p. 13, Table 5 · p. 13

F Instruction Strength

The appendix tests prompt-strength robustness and documents procedures for constructing, annotating, and validating the diagnostic materials.

F Instruction Strength

  • The instruction-strength experiment compares strong, neutral, and weak formulations using Mistral-7B and OLMo-7B.The reported average performance is in Table 6, while Table 7 shows Mistral-7B’s fine-grained performance.Source: F Instruction Strength · p. 13, Table 6 · p. 13, Table 7 · p. 14
  • Across weak, neutral, and strong prompts, relative task and evidence-type performance patterns remain consistent despite model-specific changes in absolute scores.Mistral-7B is relatively stable, while OLMo-7B degrades sharply under strong restrictive instructions; average and fine-grained results are shown in Tables 6 and 7.Source: F Instruction Strength · p. 13, Table 6 · p. 13, Table 7 · p. 14
  • Each question has four evidence types, so evidence–answer pairs total four times the final question count; GPT-5.2 per-stage drop counts are not reported.Table 3 covers instances removed during creation and filtering; GPT-5.2 filtering and verification were rerun only at the final-instance level.Source: Table 3 · p. 14

G Explanation Generation for HPCE

  • HPCE passages augment high-plausibility contradiction passages with rationales generated to explain why the HPC answer is correct instead of the NC answer.The rationale prompt asks for concise, logical, informative prose that preserves most of the original passage’s information.Source: G Explanation Generation for HPCE · p. 14

H.1 Evaluator Prompts

  • The evaluator prompt was selected for highest agreement with human annotations, and modifications to Hou et al. (2024) achieved higher Cohen’s κ.Figure 10 gives the evaluation decision tree, and Figure 11 shows the selected prompt.Source: §H.1 · p. 14

H.2 Human Annotations; J.2 Task-Annotation Prompts

  • Two NLP-expert annotators labeled 50 predictions as correct, partially correct, or incorrect; knowledge-free task annotation separately defines extractive QA as copying answer spans.The examples include a parametric-knowledge task, and GPT-4o extracted acceptable answer spans for the knowledge-free setting.Source: §H.2 · pp. 14–15, Figure 7 · p. 15, §J.2 · p. 16

J.1 Evidence Creation Prompts

  • After several rounds of prompt refinement, GPT-4o generated LPC and HPCE passages; all passages then underwent plausibility checks, with LPC judged for real-world implausibility and HPCE for high plausibility and explained conflict.Figure 14 shows the final evidence-creation prompts, Figure 15 the plausibility-check prompts, Figure 8 examples of each task, and Figure 9 an HPC and HPCE example.Source: §J.1 · p. 15, Figure 8 · p. 16, Figure 9 · p. 16

J.3 Validation Prompts

  • The final validation stage passes all instances through language models; the appendix includes prompts for evaluation, evidence plausibility, and knowledge-free annotation.Figure 10 shows annotation guidance, Figure 11 the evaluator prompt, and Figures 15–17 cover plausibility and answer validation prompts.Source: §J.3 · p. 16, Figure 10 · p. 17, Figure 11 · p. 19

Related papers

Loading 2506.06485v4…