PaperlayerView exact revision on arXiv

Source-linked AI summary

Memadapter: Counterfactual Adaptation Against Memory-induced Sycophancy

Ruqing Ning, Haibo Meng, Zhishang Xiang, Zerui Chen, Jinsong Su, Xin Wang, Qinggang Zhang

arXiv:2610.05162v1cs.AI

TL;DR

Long-term memories can preserve historical user views and contribute to sycophancy, while the appropriate influence of a memory depends on its reasoning context. MemAdapter uses counterfactual induction, context-aware reflection, and evidence-based reasoning to adapt retrieved memories; experiments across three benchmarks report improved memory-use reliability.

  • Problem

    Long-term memories can preserve users’ historical views across tasks, and reliable use requires determining how each retrieved memory should influence the current reasoning context.

    Source: §1 · p. 1, §2.3 · p. 4

  • Method

    MemAdapter examines retrieved memories counterfactually, calibrates their influence for the current task, and grounds responses in appropriate evidence.

    Source: §1 · p. 2

  • Results

    MemAdapter consistently improves overall memory-use reliability across three benchmarks and heterogeneous memory systems.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Memory use should adapt a memory’s inferential role and degree of influence to the current reasoning context, rather than only determining whether it is available.

    Source: §2.3 · p. 4

Abstract

from arXiv · show

Long-term memory enables LLM-based agents to retain and reuse information across tasks and sessions, supporting personalization and long-horizon interactions. However, persistent memories can also induce sycophancy, causing agents to over-align with users' historical beliefs even when they are inaccurate, outdated, or inconsistent with objective evidence. Existing mitigation methods assume that memory-induced sycophancy originates from biased or incorrect memories and attempt to reduce this risk by filtering such memories at different stages of the memory pipeline. However, in the real world, objective and correct memories can still induce sycophancy, and the same memory can warrant different influence across different contexts. To this end, we propose MemAdapter, a novel framework that adaptively integrates retrieved memories to support objective and reliable reasoning. Specifically, MemAdapter consists of three components: (i) Counterfactual Induction, which leverages counterfactual reasoning to uncover the potential risk of retrieved memories; (ii) Context-Aware Reflection, which calibrates the inferential influence of each retrieved memory in light of the current task via self-reflection; and (iii) Evidence-Based Reasoning, which grounds the final response in appropriate evidence while preserving the legitimate influence of memory. Extensive experiments on three benchmarks demonstrate that MemAdapter consistently improves memory reliability across diverse scenarios. Our code is available at https://github.com/DEEP-JLU/MemAdapter.

1 INTRODUCTION

Long-term memories support personalization but can also persistently bias reasoning, and their risks may depend on how they are used in the current context. MemAdapter adapts retrieved memories during reasoning, and experiments across three benchmarks report improved memory-use reliability.

1 INTRODUCTION

  • Long-term memory supports cross-task reuse but can preserve users’ historical views and undermine factuality and objective judgment.Source: §1 · p. 1
  • Existing approaches often filter or restructure memories before reasoning, but accurate, relevant memories can still contribute to biased reasoning in particular contexts.Source: §1 · pp. 1–2
  • MemAdapter adaptively integrates retrieved memories through counterfactual risk discovery, task-aware influence calibration, and evidence-grounded response generation.The framework operates after retrieval without modifying upstream memory construction or retrieval.Source: §1 · p. 2
  • MemAdapter consistently improves overall memory-use reliability across three benchmarks and heterogeneous memory systems.Source: §1 · p. 3

2 PRELIMINARY STUDY

The preliminary study shows that memory risk depends not only on stored content but also on the task context in which a memory is used. Its analyses examine performance with accurate memories and the changing role of a fixed memory across tasks.

2 PRELIMINARY STUDY

  • The study tests accurate, relevant memories and varies task context while holding a retrieved memory fixed.Source: §2 · p. 3

2.1 ACCURATE AND RELEVANT MEMORIES CAN STILL INDUCE SYCOPHANCY

  • 96.0% to 77.3%: adding accurate, highly relevant memories reduced overall accuracy, and 19.4% of initially correct predictions became incorrect after memory was introduced.Every such reversal followed the retrieved memory.Source: §2.1 · p. 3

2.2 THE SAME MEMORY WARRANTS DIFFERENT INFLUENCE ACROSS CONTEXTS

  • A cardiologist memory can tailor an explanation, but should not change a treatment choice when patient-specific evidence supports another option.Source: §2.2 · p. 3

2.3 DISCUSSION

  • Memory risk depends on interaction with downstream reasoning context, so reliable use requires separating potential risk from task-specific influence.Source: §2.3 · p. 4

3 THE FRAMEWORK OF MEMADAPTER

MemAdapter treats memory risk as context-dependent and adapts retrieved memories through counterfactual analysis, task-aware reflection, and evidence-grounded response generation.

3.1 COUNTERFACTUAL INDUCTION

  • Counterfactual Induction keeps each memory fixed while varying downstream task conditions, exposing contexts where its appropriate contribution changes.The task space includes concrete requests and evidence configurations for memory-relevant task types.Source: §3.1 · pp. 4–5
  • For each setting, it records what a memory can support and cannot justify, then groups substantive contribution changes into conditions for later calibration.The resulting characterization is task-agnostic and captures potential risks associated with each memory.Source: §3.1 · p. 5

3.2 CONTEXT-AWARE REFLECTION

  • Context-Aware Reflection combines each memory’s counterfactual boundaries with the current request and evidence to calibrate its role and degree of influence.Its task-conditioned instruction specifies what the memory may support, which response parts it may affect, and which conclusions it cannot justify.Source: §3.2 · p. 5

3.3 EVIDENCE-BASED REASONING

  • Evidence-Based Reasoning grounds the answer in appropriate evidence while preserving only the memory influence permitted by task-specific instructions.It generates an internal support trace linking response spans to sources and influence instructions, then checks those links before returning the answer.Source: §3.3 · pp. 5–6

4 EXPERIMENTS

MemAdapter improves memory-use performance across benchmarks, systems, and backbones, while gains vary by metric and system. Ablations indicate that its three stages make complementary contributions.

4.1 EXPERIMENTAL SETTINGS

  • Evaluation covers MemSyco-Bench, MemTrapBench, and PersistBench, comparing five memory systems and four post-retrieval interventions against direct generation.Compared methods receive the same retrieved candidates within each memory system; benchmark metrics follow official definitions and protocols.Source: §4.1 · p. 6, §4 · p. 6

4.2 MAIN RESULTS

  • Across three benchmarks, MemAdapter improves multiple memory-use metrics across five memory systems, though beneficial-memory failure rates rise for some systems.It improves both memory-use dimensions on MemSyco-Bench and both MemTrapBench scores under all five systems; PersistBench sycophancy failures fall across all five, while cross-domain failures fall for four.Source: §4.2 · p. 7, Table 1 · p. 7

4.3 CROSS-BACKBONE GENERALIZATION

  • Across GPT-5.6-sol and Qwen3-8B, MemAdapter improves overall memory-use performance, with gains varying by backbone, memory system, and task dimension.Personalized-memory accuracy improves across all evaluated systems under Qwen3-8B but changes are mixed across systems under GPT-5.6-sol.Source: §4.3 · p. 8, Table 2 · p. 8

4.4 ABLATION STUDY

  • Counterfactual Induction raises sample-weighted average accuracy from 70.25 to 84.26 on NaiveRAG and from 71.71 to 84.19 on A-MEM.The B variant also improves all five task-specific accuracy metrics for both systems compared with Direct Gen.Source: §4.4 · p. 9, Table 3 · p. 9
  • Adding Evidence-Based Reasoning raises sample-weighted average accuracy from 85.16 to 89.94 on NaiveRAG and from 85.68 to 90.45 on A-MEM.This compares the full MemAdapter with B + R, which includes Counterfactual Induction and Context-Aware Reflection but omits the final evidence-based stage.Source: §4.4 · p. 9, Table 3 · p. 9

5 CONCLUSION

MemAdapter adapts the use of retrieved memories without changing the upstream memory system. The paper evaluates it across varied memory systems and provides reproducibility materials.

5 CONCLUSION

  • MemAdapter addresses cases where accurate, relevant memories are used beyond what a task justifies, and their appropriate influence changes across reasoning contexts.Its post-retrieval stages derive conditional-use rules, evaluate them against the current task and evidence, and generate responses according to resolved usage scopes.Source: §5 · p. 9
  • Reproducibility materials include dataset, baseline, and implementation details, hyperparameters, prompts, performance reports, and code.The paper states that baseline models come from public repositories.Source: REPRODUCIBILITY STATEMENT · p. 10

A EXPERIMENTAL PROTOCOL AND REPRODUCIBILITY

  • A memory system constructs, maintains, and retrieves historical information; a memory-use method governs how retrieved memories are processed for an answer.MemAdapter operates after retrieval and does not modify memory construction or retrieval results.Source: A EXPERIMENTAL PROTOCOL AND… · p. 13

A.1.1 MEMORY SYSTEMS

  • The evaluation spans five memory systems with different memory representations and retrieval pipelines.Examples range from A-MEM's interconnected structured notes and Mem0's persistent semantic store to NaiveRAG's raw dialogue chunks, LightMem's staged memory design, and MemoryBank's persistent summaries.Source: §A.1.1 · p. 13

A.1.2 MEMORY-USE METHODS

The appendix describes post-retrieval intervention baselines, a controlled comparison protocol, and implementation and reproducibility settings.

A.1.2 MEMORY-USE METHODS

  • Direct Generation passes retrieved memories to the model unchanged, while Anti-Sycophancy adds a neutral-observer disclaimer and Self-ReCheck retains only memories judged relevant.Self-ReCheck selects a subset rather than assigning usage boundaries to each memory.Source: §A.1.2 · pp. 13–14
  • Dynamic Partition organizes all retrieved memories under domain headers, whereas MemGate uses a learned query-conditioned gate to admit or filter memories.MemGate retains memories whose gate scores exceed 0.07 in the described implementation.Source: §A.1.2 · p. 14

A.2 RETRIEVAL AND FAIR-COMPARISON PROTOCOL

  • Methods paired with a memory system receive the same frozen retrieved memories and otherwise share the instance, query, evidence, backbone, decoding, and evaluation settings.MemAdapter preserves the retrieved set and applies Counterfactual Induction, Context-Aware Reflection, and Evidence-Based Reasoning.Source: §A.2 · pp. 14–15

A.3 INFERENCE, EVALUATION, AND REPRODUCIBILITY SETTINGS

  • The evaluation uses three generation backbones, benchmark-specific official protocols, deterministic judges, and saved per-instance artifacts for reproducibility.DeepSeek-V4-Flash and GPT-5.6-sol use API inference; Qwen3-8B runs locally.Source: §A.3 · p. 15

B SETUP OF PRELIMINARY STUDY

The preliminary study controls for memory accuracy, relevance, and sufficiency while measuring changes in task accuracy and memory-aligned reversals. The complete-experiments section covers detailed benchmark results, ablations, and runtime comparisons.

B SETUP OF PRELIMINARY STUDY

  • The controlled evaluation retains 75 reviewed instances—25 from each benchmark—with accurate, benign, relevant memories insufficient to determine the answer.MemTrapBench safety-conflict scenarios are excluded; retained cases include ordinary reasoning involving cognitive inertia, cross-task transfer, or overgeneralization.Source: B SETUP OF PRELIMINARY STUDY · p. 15
  • The study reports No-Memory and With-Memory accuracy, their difference, and correct-to-incorrect reversals, including whether new errors align with memory.Reversal rates are normalized by the number of No-Memory-correct predictions.Source: B SETUP OF PRELIMINARY STUDY · pp. 15–16

C COMPLETE EXPERIMENTS AND RUNTIME ANALYSIS

C.1 COMPLETE RESULTS ON MEMSYCO-BENCH

On MemSyco-Bench, MemAdapter achieves the highest sample-weighted average accuracy for every evaluated memory system, with especially large gains on scope control and memory–evidence conflict. Its task-level results also show trade-offs and reduced performance disparity across memory systems.

C.1 COMPLETE RESULTS ON MEMSYCO-BENCH

  • 61.67 and 54.00 percentage points are MemAdapter’s Contextual Scope Control and Memory–Evidence Conflict gains with Mem0; the respective gains with LightMem are 57.00 and 92.67 points.MemAdapter attains the highest sample-weighted average accuracy for every evaluated memory system, and gains are especially pronounced where baseline scope control or memory–evidence conflict is weak.Source: §C.1 · p. 16, Table 5 · p. 17
  • MemAdapter consistently performs strongly on Objective Fact Judgment and Valid Memory Selection while remaining competitive on other task dimensions.The reported pattern indicates particular help with reliable memory-use decisions when retrieved memories are difficult to reconcile with current task context.Source: §C.1 · p. 17
  • MemAdapter does not lead every individual task, but achieves the highest Avg. Acc. in each memory-system block, indicating a more balanced profile across five requirements.Self-ReC. and Dyn. Part. each score 100.00 on Memory–Evidence Conflict with NaiveRAG; Anti-Syc. scores 94.67 on Contextual Scope Control with A-MEM.Source: §C.1 · p. 17, Table 5 · p. 17
  • Avg. Acc. disparity across memory systems narrows from 34.68 percentage points before intervention to 4.97 with MemAdapter.The corresponding ranges are 37.03–71.71 before intervention and 85.48–90.45 with MemAdapter; the results do not establish equal task difficulty across systems.Source: §C.1 · p. 17

C.2 COMPLETE RESULTS ON MEMTRAPBENCH

On MemTrapBench, MemAdapter has the highest average score under every memory system, with gains concentrated in Task Boundary and improvements also seen on Safety and Cognitive Bias. Scenario headroom and the strongest alternative vary across systems.

C.2 COMPLETE RESULTS ON MEMTRAPBENCH

  • 0.74–1.50 points are MemAdapter’s Task Boundary gains across all five memory systems; average-score gains range from 0.29 to 0.56 points.MemAdapter also improves Safety in every setting and Cognitive Bias in four of five, while Trauma changes are comparatively small.Source: §C.2 · pp. 17–18, Table 6 · p. 18
  • Task Boundary gains are larger where direct-generation scores leave more room, while Trauma changes are modest on already-high-scoring settings.Direct-generation Trauma scores include 4.85 for NaiveRAG and 4.88 for A-MEM; MemAdapter changes there are −0.02 and −0.01, respectively.Source: §C.2 · p. 18
  • MemAdapter exceeds Self-ReC., the second-highest average scorer in every memory-system block, by 0.07 on NaiveRAG, 0.20 on A-MEM, 0.11 on Mem0, 0.26 on MemoryBank, and 0.09 on LightMem.The margins vary by underlying memory system.Source: §C.2 · p. 18

C.3 COMPLETE RESULTS ON PERSISTBENCH

On PersistBench, MemAdapter reduces sycophancy failure rates across all three inference counts for every memory system. Cross-domain and beneficial-failure results vary more by memory system.

C.3 COMPLETE RESULTS ON PERSISTBENCH

  • Sycophancy FR@1, FR@2, and FR@3 reductions persist across all inference counts for every memory system, with the largest reductions for MemoryBank and A-MEM.PersistBench metrics are failure rates, so lower values are better.Source: Table 7 · p. 19, §C.3 · p. 19
  • Cross-domain FR improves at all three inference counts for NaiveRAG, A-MEM, Mem0, and MemoryBank, while LightMem results remain unchanged or increase slightly.Beneficial FR also varies across memory systems.Source: §C.3 · p. 19

C.4 COMPLETE ABLATION RESULTS

Across five memory systems, the full MemAdapter configuration has the strongest aggregate ablation result. Its post-retrieval reasoning adds latency, while measured runtimes remain within the reported range of existing interventions.

C.4 COMPLETE ABLATION RESULTS

  • B adds only Counterfactual Induction, B + R adds Context-Aware Reflection, and the full configuration adds Evidence-Based Reasoning.Individual task scores and intermediate-variant benefits vary across memory systems, while the full configuration has the strongest aggregate result.Source: §C.4 · pp. 19–20
  • The full MemAdapter achieves the highest sample-weighted average accuracy across all five memory systems, outperforming both partial variants.Table 8 reports the task-specific and average accuracy metrics; the full configuration adds Evidence-Based Reasoning after Boundary Induction and Reflection.Source: §C.4 · pp. 19–20, Table 8 · p. 20

C.5.1 MEASUREMENT PROTOCOL

  • Runtime is measured per successfully completed instance from the first post-retrieval model request through final response, including method calls, retries, and recorded waiting time.The protocol reports mean and P50, P95, and P99 runtimes; it excludes memory construction and retrieval, evaluation, model loading, and shutdown.Source: §C.5.1 · p. 20

C.5.2 END-TO-END RUNTIME COMPARISON

  • MemAdapter's mean runtime is 6.63–6.95 seconds per sample across three memory systems, with P99 runtime ranging from 13.94 to 16.23 seconds.The authors describe its end-to-end latency range as comparable to existing post-retrieval interventions, while noting occasional slow requests.Source: §C.5.2 · pp. 20–21, Table 9 · p. 21

D DATASET DETAILS

The benchmarks assess whether retrieved memories receive an appropriate role under the current task, rather than whether systems can simply store or retrieve information. The section also illustrates memory-use failures and context-sensitive responses through case studies.

D DATASET DETAILS

  • The three benchmarks cover decisions about using, restricting, overriding, updating, or ignoring retrieved memories, alongside preserving beneficial personalization.They frame memory use as task-dependent rather than a binary retain-or-discard choice.Source: D DATASET DETAILS · p. 21, §D.4 · p. 23

D.1 MEMSYCO-BENCH

  • MemSyco-Bench contains 1,550 instances across five scenarios, and reports scenario accuracy plus sample-size-weighted averages for its task dimensions.The scenarios cover objective fact judgment, contextual scope, memory-evidence conflict, personalized use, and valid memory selection; the latter has 350 instances and each other scenario has 300.Source: §D.1 · p. 21, §D.1.1 · p. 22

D.2 MEMTRAPBENCH

  • MemTrapBench contains 1,050 instances across four scenarios grouped as Reasoning Fixation and Belief Distortion, with higher-is-better category scores.Reasoning Fixation summarizes Task Boundary, Cognitive Bias, and Trauma; Belief Distortion corresponds to Safety.Source: §D.2 · p. 22, §D.2.1 · p. 22

D.3 PERSISTBENCH

  • PersistBench has 500 human-validated samples across Cross-Domain Leakage, Sycophancy, and Beneficial Memory Use, with lower-is-better metrics reported as percentages.Its failure rules count a sample as failed when any of three generations receives a score of at least 3 in the first two scenarios, or when the single-generation score is 2 or 3 for beneficial use.Source: §D.3 · p. 22, §D.3.1 · p. 23

D.4 MEMORY-USE REQUIREMENTS

  • The benchmarks distinguish semantic relevance from legitimate decision authority: relevant memories may still need restriction, override, replacement, or rejection.MemSyco-Bench emphasizes memory authority, MemTrapBench reasoning fixation or belief distortion, and PersistBench both harmful influence and beneficial use.Source: §D.1 · p. 21, §D.2 · p. 22, §D.3 · p. 22

E.1 CASE 1: FROM PREFERENCE-ALIGNED MEMORY TO EVIDENCE-GROUNDED DECISION

  • In the backup-service case, MemAdapter preserves the user's familiarity preference but recommends Pine Archive based on storage-cost and restoration-reliability evidence.Direct memory use instead treats preferences as decision evidence and recommends Northstar Backup.Source: E CASE STUDY · p. 23

E.2 CASE 2: FROM CONTRADICTORY MEDICAL MEMORIES TO CONTEXT-GROUNDED EMERGENCY GUIDANCE

  • In the bleeding-response case, MemAdapter excludes medical-history memories whose person-specific conditions conflict with the request's explicit context.The direct-memory baseline treats those records as facts about the injured friend and adds unsupported medical history and caveats.Source: §E.1 · p. 24

E.3 CASE 3: FROM METHODOLOGICAL BIAS TO BOUNDED PROJECT-PROCESS PERSONALIZATION

The section describes a project-process personalization case, surveys prior work on memory and retrieval, and presents algorithms for regulating post-retrieval memory use and checking final responses.

E.3 CASE 3: FROM METHODOLOGICAL BIAS TO BOUNDED PROJECT-PROCESS PERSONALIZATION

  • The Orion project case includes memories favoring rigid waterfall development and detailed weekly status reports, alongside the user's engineering leadership background.Source: §E.3 · p. 26

F RELATED WORK

  • Prior memory work spans persistent memory streams, feedback, hierarchical storage, external retrieval, memory updating, and structured or efficient organization.The related-work survey also describes query-conditioned gating, task-usefulness selection, and corrective retrieval as approaches controlling which retrieved information reaches generation.Source: F RELATED WORK · pp. 26–27
  • The paper locates its focus after retrieval: how accurate, relevant memories should contribute under the current task and available evidence without changing the upstream pipeline.The related-work discussion notes that prior approaches mainly intervene during memory construction, retrieval, filtering, or organization.Source: F RELATED WORK · p. 27

G ALGORITHM OF MEMADAPTER

  • Counterfactual Induction derives conditional-use boundaries by considering task types and settings, assessing memory contributions, and grouping equivalent contributions.These boundaries are returned for the retrieved memories.Source: G ALGORITHM OF MEMADAPTER · pp. 27–28
  • Context-Aware Reflection examines each memory under the current request, context, evidence, and other memories to produce task-specific use instructions.The procedure accounts for supported updates, qualifications, and conflicts among memories.Source: G ALGORITHM OF MEMADAPTER · p. 28
  • Evidence-Based Reasoning generates a response and support trace, then checks that each major span's source is present and supports its role.For memory-based spans, it also verifies consistency with the task-specific use instruction.Source: G ALGORITHM OF MEMADAPTER · p. 28

H PROMPT TEMPLATES

This appendix documents MemAdapter’s three-stage, comparison-method, and benchmark-judge prompts, including their roles, evidence boundaries, input–output behavior, and criteria for objective correctness and appropriate memory influence. The rubrics cover resisting harmful or out-of-scope influence while using relevant memories appropriately.

H PROMPT TEMPLATES

  • The template appendix covers MemAdapter modules, LLM-based comparison instructions, and task-specific judge prompts for MemSyco-Bench, MemTrapBench, and PersistBench.It retains role, decision-criterion, and output-behavior instructions while omitting repetitive placeholders and lengthy calibration examples.Source: H PROMPT TEMPLATES · p. 28

H.1 MEMADAPTER PROMPTS

  • MemAdapter’s prompt pipeline moves from counterfactual boundary induction to task-specific reflection and then final-answer generation under the resulting memory-use instructions.The stages respectively derive conditional-use rules, ground them in the current task, and generate the response.Source: §H.1 · p. 29

H.1.1 COUNTERFACTUAL INDUCTION PROMPT

  • Counterfactual Induction tests each memory across hypothetical task contexts, assesses its support, scope, and limits, then converts substantive differences into conditional-use boundaries.It analyzes memories without the current query or task-specific evidence and does not judge the current task.Source: §H.1.1 · p. 29

H.1.2 CONTEXT-AWARE REFLECTION PROMPT

  • Context-Aware Reflection resolves each memory’s prospective boundary for the actual task, jointly considering evidence, updates, qualifications, conflicts, and time sensitivity.It produces a task-specific influence instruction that states permitted support, scope, and limits.Source: §H.1.2 · pp. 30–31

H.1.3 EVIDENCE-BASED REASONING PROMPT

  • Evidence-Based Reasoning answers the query under the memory-use instructions, checking that major response spans have supporting sources and authorized memory use.It grounds factual claims in applicable evidence, keeps preferences from changing independently supported facts, and revises or limits unsupported claims.Source: §H.1.3 · pp. 31–32

H.2 PROMPTS FOR OTHER COMPARISON METHODS

  • Comparison methods receive the same frozen retrieved-memory candidates; Anti-Sycophancy cautions against treating memories as evidence, while Self-Recheck selects useful memory IDs.The shared inference interface supplies the current query and retrieved memories.Source: §H.2 · p. 33, §H.2.1 · p. 33, §H.2.2 · p. 33

H.2.3 DYNAMIC PARTITION PROMPT

  • Dynamic Partition assigns each memory exactly one domain, whereas MemGate uses a non-generative module to filter memories before answer generation.Dynamic Partition preserves each memory exactly once; MemGate passes the retained subset to the shared answer-generation interface.Source: §H.2.3 · p. 33, §H.2.4 · p. 33

H.3.1 MEMSYCO-BENCH JUDGE PROMPTS

  • MemSyco-Bench’s task-specific judges evaluate semantic content against supplied evidence and rubrics, using binary labels and pass criteria without speculating about hidden reasoning.The prompts cover five tasks, including whether a preference memory improperly affects an objective answer.Source: §H.3.1 · pp. 33–34
  • MemSyco-Bench separately scores objective correctness, endorsement of a preference-aligned misconception, and contamination, suppressing a pass only when correctness is met and both sycophancy scores are zero.Contamination includes treating preference as evidence or weakening a correct conclusion.Source: §H.3.1 · p. 34
  • Its scope-control rubric checks whether a preference remains applicable to the current task, audience, and other people, scoring accuracy and use outside that scope.A preference may be useful, but should not control decisions beyond its valid scope.Source: §H.3.1 · p. 34

H.3.2 MEMTRAPBENCH JUDGE PROMPT

  • MemTrapBench’s shared rubric independently scores factual and logical correctness, explicit instruction compliance, relevance and information purity, and delivery efficiency on a 0–5 scale.The template requires a justification before each score and returns JSON only.Source: §H.3.2 · p. 36
  • The Poison rubric tests whether a response rejects an adversarial false or unsafe premise in favor of objective truth and safety, while also assessing relevance.It scores safety and factual correctness and relevance on a 0–5 scale.Source: §H.3.2 · pp. 36–37
  • The Number Game rubric evaluates mathematical validity and use of the query’s required numbers, plus compliance with current constraints rather than unstated traditional restrictions.It uses the reference gold solution and scores two dimensions from 0 to 5.Source: §H.3.2 · p. 37

H.3.3 PERSISTBENCH JUDGE PROMPT

  • PersistBench’s cross-domain leakage rubric treats memory as in-domain only when necessary or materially useful, or explicitly requested, and scores irrelevant leakage from 1 to 5.Severe leakage can derail the answer or expose private-context details.Source: §H.3.3 · p. 38
  • Its sycophancy rubric accepts style, tone, or format personalization but flags adopting remembered bias in factual, moral, or epistemic claims without explicit instruction.Judges identify relevant belief-bearing memories, treat other memories as inactive noise, and assess whether the response follows evidence or adopts bias.Source: §H.3.3 · p. 38
  • The beneficial-use rubric rewards relevant memory integration when useful, ignores irrelevant or inappropriate memories, and asks judges to identify relevant memories and distractors before scoring.Its scale runs from 1 to 3, with the highest score for correct integration and appropriate personalization that materially improves the response.Source: §H.3.3 · p. 39

Related papers

Loading 2610.05162v1…