Toward Robust Personalized Alignment for LLMs: Mitigating Persona Drift in Multi-Turn Dialogue

• Personalization methods improve profile access or personalized responses but generally lack an explicit decision about whether new evidence should revise persistent state.This creates a stability–adaptability trade-off between overwriting grounded preferences and failing to adapt to genuine changes.Source: §1 · p. 1
• The paper frames long-horizon personalization as controlled persistent persona revision, separating turn-local evidence from updates to the maintained user state.Revision depends on whether evidence is sufficiently grounded and resolved.Source: §1 · p. 1
• PERSIST evaluates whether grounded persona states are preserved or appropriately revised under ambiguity, conflict, and controlled social influence.The post-anchor benchmark is complemented by separate first-contact and preference-reversal diagnostics.Source: §1 · p. 2
• CORE routes observations through COMMIT, DEFER, or IGNORE and performs gated belief revision only when an update is authorized.Unresolved evidence can remain outside persistent state while sufficiently supported preference changes are incorporated.Source: §1 · pp. 1–2
• Across standard benchmarks, stress tests, and human evaluation, CORE improves personalized alignment and robustness.Source: §1 · p. 2

• Personalized alignment methods use profiles, preference supervision, personalized decoding, and in-dialogue learning, but often assume preferences are explicit and recoverable (Zhu et al., 2025; Salemi et al., 2024; Zhao et al., 2026; Yu et al., 2026; Chen et al., 2025b).Source: §2 · p. 2
• Dialogue-state tracking maintains structured states across turns, but primarily targets task or dialogue states rather than persistent user-preference revision (Feng et al., 2023; Li et al., 2024; Das et al., 2024).Source: §2 · p. 2
• Persona drift work links growing deviation from grounded intent to noisy or inconsistent evidence, framing the challenge as deciding whether new observations revise, preserve, or leave user state unchanged (Chen et al., 2025a; Shaikh et al., 2025; Chun et al., 2025; Zhang and Choi, 2025; Li et al., 2025; Tan et al., 2025).Source: §2 · p. 2

3 Method

• CORE separates turn-local evidence from persistent belief revision, then uses the revised persona state to condition a response or clarification.Source: §3 · p. 2, Figure 2 · p. 3

3.1 Persona Belief and Turn-Local Evidence

• Slot-factorized persona beliefs distinguish persistent UNK preferences from turn-local null evidence, while relevance, ambiguity, and conflict characterize current evidence for routing.When no non-UNK preference is grounded, the pathway also supports initial belief formation.Source: §3.1 · p. 3

3.2 Uncertainty-Guided Observation Routing

• CORE predicts COMMIT, DEFER, or IGNORE: authorize grounded updates, retain relevant unresolved evidence without revision, or exclude irrelevant or explicitly non-persistent evidence.Routing authorizes revision; continuous evidence statistics determine the strength of an authorized update.Source: §3.2 · p. 4

3.3 Gated Belief Revision

• After routing, gated belief revision sets update magnitude according to relevance, ambiguity, conflict, and prior confidence, balancing the previous state with current evidence.Conflict does not permanently reject a preference change; later clear evidence can be evaluated again.Source: §3.3 · p. 4

3.4 Training

• CORE uses supervised warm-up to establish structured persona-update decisions, then PPO to optimize their long-horizon consequences across revision, state fidelity, response quality, and clarification.The warm-up supervises persona belief, turn-local evidence, update actions, and response behavior.Source: §3.4 · pp. 4–5

3.5 Why Update Control Matters

• The analysis describes how repeated unsupported evidence can erode a grounded belief, while never revising would fail to accommodate genuine preference changes.CORE therefore makes revision an explicit evidence-conditioned decision rather than an automatic response to locally plausible observations.Source: §3.5 · p. 5

4.2 PERSIST: Stress Test

• PERSIST anchors persona evidence before introducing partial or transient cues and six controlled social-influence strategies, then tests preservation, revision, and clarification.Because it is post-anchor, it does not by itself isolate pressure-first belief formation or explicit preference reversal.Source: §4.2 · p. 6

4.3 Main Results on Standard Benchmarks

• CORE achieves the highest alignment among matched same-backbone methods on PersonaChat and ALOE, while remaining competitive with larger scale-reference models on trajectory-stability metrics.The results support controlled persona-state revision as useful beyond stronger response generation or more persistent memory alone.Source: Table 1 · p. 7, §4.3 · p. 7
• CORE improves turn-wise alignment and uses revealed preference evidence more efficiently, while its belief confidence better matches empirical correctness.At the same preference-revelation ratio, CORE achieves stronger alignment; its calibration results indicate gains beyond surface-level generation.Source: §4.3 · p. 7, Figure 3 · p. 7

4.4 Robustness under State Perturbations

• On PERSIST, CORE raises ARUS from 71.9 to 80.8 and BCD from 66.1 to 76.4, while reducing persona-state drift from 12.6 to 10.1.The benchmark tests robustness under ambiguity, conflict, and controlled social-influence stress.Source: §4.4 · p. 7, Table 2 · p. 8
• Under pressure-first initialization, CORE reduces FCR from 31.6% for RLPA to 20.9%, while explicit persistent preference changes can be committed and tentative changes deferred until confirmation.Its 20.9% FCR remains above CORE’s 11.6% after anchoring.Source: §4.4 · p. 8

4.5 Mechanistic Ablations and Efficiency

• Removing executable update actions causes the largest ablation degradation: AL falls from 82.9 to 78.8, CRR from 82.7 to 71.6, and DRIFT rises from 10.1 to 15.4.Structure-matched controls also perform below CORE, supporting executable belief revision beyond intermediate traces, action-name supervision, or additional labels.Source: §4.5 · pp. 8–9, Table 3 · p. 9
• CORE adds 23.0 ms per turn; response decoding remains the dominant inference cost, and training or evaluation annotations are not required at inference.The control layer shares a context-encoding pass and cached slot/value descriptions.Source: §4.5 · p. 9
• In complete multi-turn interactions, CORE improves average human ratings over RLPA by 11.7%, with larger gains for persona consistency and related long-horizon behaviors than for response quality.The controlled evaluation validates behavior under matched multi-turn scenarios, not natural longitudinal deployment.Source: §4.5 · p. 9

• CORE formulates long-horizon personalization as controlled persona-state revision, separating turn-local evidence from persistent belief updates.Source: §5 · p. 9
• CORE preserves grounded preferences under unresolved observations while remaining adaptive to sufficiently supported changes.Source: §5 · p. 9
• PERSIST is a held-out benchmark for post-anchor persona-state robustness, and results across benchmarks, stress tests, and human evaluation support explicit update control as useful for robust long-horizon personalization.Source: §5 · p. 9

• Benchmark-instantiated persona slots and structured supervision mean CORE's state-level analysis does not fully capture open-ended or cross-slot preferences.Source: Limitations · p. 9
• CORE's conservative update policy may delay adaptation when genuine preference changes are weakly expressed.Source: Limitations · p. 10
• PERSIST primarily evaluates post-anchor robustness rather than first-contact personalization, and controlled benchmarks cannot fully represent long-term real-world interactions.Source: Limitations · p. 10

Ethical Considerations

• Maintained persona states can influence future interactions, raising privacy and user-control concerns.Source: Ethical Considerations · p. 10
• PERSIST is for controlled evaluation, not profiling real users or inferring sensitive attributes; real-world systems should minimize retained information and let users inspect, correct, delete, or opt out of persona states.Source: Ethical Considerations · p. 10

Appendix

• The appendix provides audit and reproduction details, including method implementation, experimental controls, metrics and reliability, PERSIST construction, and robustness and failure analyses.Source: Appendix · p. 14

A.1 Slot-Factorized Persona Belief

• CORE represents the maintained persona belief over benchmark-instantiated slots, making the factorization an auditable state interface rather than an independence assumption.Source: §A.1 · p. 14

A.2 Semi-Open Value Handling

• Value sets are semi-open: observed values can be added when mapped to an existing slot with sufficient semantic confidence, while unmappable attributes go to a residual open slot excluded from closed-slot metrics.Source: §A.2 · p. 14

A.3 Context Encoding and Slot-Wise Evidence Scoring

• CORE encodes dialogue history, the current utterance, and the previous belief before scoring turn-local evidence, with routing performed only after evidence and uncertainty descriptors are obtained.Source: §A.3 · pp. 14–15

A.4 Uncertainty-Guided Update Routing

• Slot-wise evidence scores indicate what the current turn suggests, while relevance, ambiguity, and conflict inform COMMIT, DEFER, or IGNORE routing.DEFER preserves the current belief and does not create durable pending memory; unresolved evidence remains current-turn control information.Source: §A.3 · p. 15, §A.4 · pp. 15–16

A.5 Belief Revision

• Only COMMIT authorizes belief revision; DEFER and IGNORE leave the posterior unchanged, while the gated update moves it toward evidence-supported values.Source: §A.5 · p. 16

A.6 Inference-Time Algorithm

• The inference algorithm scores evidence before routing, and only COMMIT-routed evidence can revise the maintained belief.The belief update remains differentiable with respect to evidence scores and the revision gate, while the discrete routing action is optimized through routing loss and the policy objective.Source: §A.6 · p. 16

B.1 Benchmarks, Splits, and Evaluation Scope

• ALOE, PersonaChat, and held-out PERSIST test preference revelation, free-form persona generalization, and robustness, respectively, in a partially observed setting.ALOE and PersonaChat use persona-level or clustered-descriptor splits; closed-slot metrics exclude evidence that cannot be reliably mapped to normalized values.Source: §B.1 · p. 17

B.2 Slot Instantiation and Baseline Controls

• CORE uses benchmark-specific closed-slot inventories, while baselines test prompting, memory, preference optimization, personalized-agent, supervision-format, and scale explanations under matched controls.All baselines share the dialogue input and context budget; inference uses no gold state or update-action annotations.Source: §B.2 · pp. 17–18

B.3 Update-Action Labels and Audit

• COMMIT revises persistent preferences when evidence is resolved, DEFER preserves unresolved relevant evidence, and IGNORE excludes unrelated or explicitly non-persistent evidence.PERSIST action labels are diagnostic only; conflict can later be re-evaluated for COMMIT when subsequent evidence establishes a persistent change.Source: §B.3 · p. 18
• Verifier–human agreement is strongest for COMMIT, while residual disagreement is mostly DEFER versus IGNORE and primarily affects fine-grained ACTACC.Labels are independently verified, filtered for intent drift, and stratified-audited by humans before training, without using CORE predictions.Source: §B.3 · p. 18

B.4 Supervised Warm-Up and Uncertainty Calibration

• After supervised warm-up, relevance, ambiguity, and conflict calibration parameters are selected on the held-out calibration split and fixed during PPO.Fixing them prevents PPO from improving reward by reshaping uncertainty estimates.Source: §B.4 · p. 19

B.5 PPO Configuration and Stability Checks

• PPO weights action correctness, state fidelity, response quality, and clarification behavior at 0.35, 0.30, 0.25, and 0.10, respectively.Stability checks audit collapse to always deferring or ignoring, excessive clarification, weakly informative personalization, and inaccurate belief updates.Source: §B.5 · pp. 19–20

C Evaluation Metrics, Reliability, and Human Study

• Evaluation separates response, belief, action, and clarification evidence; response-level outcomes are primary, while state and action metrics are complementary diagnostics.Alignment Level assesses grounded personalized responses, while belief update accuracy checks adaptation to reliable evidence and DRIFT tracks posterior mass away from the gold value.Source: C Evaluation Metrics, Reliability, and… · p. 20, §C.1 · p. 20, §C.2 · p. 20

C.5 PERSIST Robustness Metrics

• PERSIST measures preservation of grounded commitments under ambiguity, conflict, and social influence, with behavior targets independent of model predictions.Metrics are macro-averaged across strategies and slots; turn metrics average within dialogue before averaging across dialogues, and reported uncertainty is generally standard error across dialogue instances.Source: §C.5 · p. 21, §C.6 · pp. 21–22

C.6 Aggregation and Uncertainty Reporting

• Main comparisons use paired dialogue bootstrap resampling; differences are significant only when the 95% paired-bootstrap confidence interval excludes zero, and strategy-level PERSIST breakdowns are diagnostic.Source: §C.6 · p. 22

C.7 Judge Prompts, Calibration, and Reliability

• GPT-5.1 judges semantic AL and ARUS using fixed rubrics, randomized order, held-out calibration, and hidden system identities; pairwise preference is only an auxiliary reliability audit.Table 7 covers the larger judge-calibration audit, whereas Table 8 reports evaluator-level reliability for a smaller matched-transcript human study; reversed-order checks test positional bias without changing model scores.Source: §C.7 · p. 22, Table 7 · p. 22

C.8 Anti-Gaming Diagnostics

• CORE reduces FCR while maintaining strong BUA, indicating robustness is not explained by uniformly avoiding updates.The protocol also audits always-COMMIT, always-DEFER/IGNORE, and unnecessary clarification.Source: §C.8 · p. 23

C.9 Human Evaluation Protocol

• Eight trained evaluators review anonymized, matched 12-turn conversations across four systems and benign, ambiguity-heavy, and conflict-heavy scenarios.Separate evidence–action audits examine slot evidence, ambiguity or conflict, update actions, and clarification decisions.Source: §C.9 · p. 23

C.10 Human Rating Rubric and Statistical Analysis

• Human ratings cover persona consistency, ambiguity handling, response quality, preference preservation under pressure, and long-horizon stability on 1–5 scales.Scores aggregate at evaluator level, with 95% confidence intervals from 10,000 evaluator-level bootstrap resamples; Table 8 reports human–human, judge–human, and Cohen’s κ agreement on held-out audit items.Source: §C.10 · p. 23

C.11 Human Evaluation Results

• CORE improves average human ratings over RLPA by 11.7% and receives the highest rating on all five dimensions.The larger gains occur on persona consistency, ambiguity handling, preference preservation, and long-horizon stability rather than response quality.Source: §C.11 · p. 23

C.12 Pairwise Preference and Scenario Analysis

• CORE is preferred in 64.6% of pairwise comparisons, versus 22.9% for RLPA, with 12.5% ties; its preference gap is largest in conflict-heavy scenarios.Evaluators compared anonymized matched transcript pairs, with ties allowed.Source: §C.12 · p. 23

D PERSIST: A Benchmark for Persona-State Robustness under Sequential Interaction Stress

• PERSIST tests preservation or appropriate revision of grounded persona commitments under later ambiguity, conflict, or controlled social-influence stress.Unlike benchmarks focused mainly on preference acquisition, it targets behavior under subsequent interaction stress.Source: D PERSIST: A Benchmark for… · pp. 23–24

D.1 Benchmark Objective and Target Behaviors

• The benchmark distinguishes unsupported or transient cues from sufficiently grounded preference changes, allowing adaptation when later evidence clearly establishes a genuine change.Target behaviors include preserving prior commitments, resisting unsupported pressure, and clarifying or responding cautiously when evidence is unresolved.Source: §D.1 · p. 24

D.2 Dataset Statistics and Splits

• CORE is trained without PERSIST data; PERSIST-Eval is the held-out main robustness subset, and all splits are persona-level to prevent profile and axiom leakage.PERSIST-Manual is reserved for evaluator audit and qualitative sanity checks.Source: §D.2 · p. 24, Table 11 · p. 25

D.3 Construction and Verification Pipeline

• PERSIST separates persona construction, axiom grounding, perturbation, pressure generation, verification, human audit, and split assignment to reduce leakage and make construction auditable.Figure 6 depicts the construction pipeline.Source: §D.3 · p. 25, Figure 6 · p. 25

D.4 Social-Influence Strategy Taxonomy

• Confrontation uses six roughly balanced strategies: authority pressure, social conformity, affective appeal, urgency framing, reciprocity lure, and relationship pressure.Aggregate robustness is reported with strategy-level macro averaging.Source: §D.4 · p. 25

D.5 Quality Assurance and Human Audit

• Automatic checks cover axiom faithfulness, phase and strategy consistency, conflict validity, clarification preference, leakage, safety and privacy, and near duplication; filtering precedes model evaluation.Filtering does not depend on model predictions.Source: §D.5 · pp. 25–26

D.6 Evaluation Protocol

• ARUS measures preservation of an anchored axiom, BCD avoidance of blind compliance with unsupported pressure, and APR clarification under under-resolved evidence.These outcomes use the benchmark’s confrontation context, pressure labels, ambiguity flags, and target behaviors.Source: §D.6 · p. 26
• Robustness metrics are macro-averaged across pressure strategies so strategies with slightly more examples do not dominate the aggregate score.Source: §D.6 · p. 26

D.7 Limitations of PERSIST

• PERSIST focuses on dialogue-level ambiguity, conflict, and social influence, excluding other drift sources such as cross-session summarization errors and retrieval-index corruption.The stated exclusions also include third-party memory updates and external-tool memory failures.Source: §D.7 · p. 26
• PERSIST-Full uses a controlled, automatically verified pipeline and PERSIST-Manual supports human audits, but the design cannot capture the full diversity of natural social interaction.Source: §D.7 · p. 26
• PERSIST tests post-anchor robustness, not whether a real user has genuinely changed a long-term preference, and does not isolate first-contact initialization under pressure.Source: §D.7 · p. 26
• Closed-slot diagnostics require reliable slot normalization; the authors recommend pairing PERSIST with other benchmarks and metrics rather than treating it as a standalone measure.Deployment also requires user-controlled memory inspection, correction, deletion, consent, and scope control.Source: §D.7 · p. 27

E.1 Structure-Matched Controls

• 80.0 AL and 72.1 CRR for action prediction without executable belief revision compare with CORE’s 82.9 AL and 82.7 CRR, showing structure and supervision alone fall short.The additional analyses also examine failure modes, stability–adaptability boundaries, backbone transfer, state fidelity, stress strategies, and inference cost.Source: §E.1 · p. 27, E Additional Results and Mechanistic… · p. 27

E.2 Component Ablations by Failure Mode

• Removing executable update actions raises unsupported COMMIT from 11.3% to 22.9%, while removing clarification raises under-clarification from 18.4% to 42.7%.Removing inertia, optimized revision, or calibration/revision rewards also produces smaller but consistent alignment and persona-state stability degradations.Source: §E.2 · p. 27

E.3 Stability–Adaptability Boundary Diagnostics

• 20.9% pressure-first FCR for CORE versus 31.6% for RLPA remains above CORE’s 11.6% post-anchor FCR; explicit persistent changes revise beliefs, unlike tentative or scoped cues.Source: §E.3 · p. 27, Table 14 · p. 28

E.4 Scale and Backbone Transfer

• 79.6 ALOE AL on Qwen2.5-3B-Instruct suggests CORE’s state-control interface is not specific to the Llama-3.2-3B-Instruct backbone.Source: §E.4 · p. 28

E.5 Closed-Slot State Diagnostics

• Table 15 reports complementary closed-slot state-fidelity diagnostics.Source: §E.5 · p. 28, Table 15 · p. 28

E.6 PERSIST Robustness Breakdowns

• CORE’s strategy-level robustness ranges from 78.6 to 82.9 across six social-influence strategies, compared with 67.6–70.5 for RLPA.This indicates the aggregate PERSIST improvement is not driven by a single pressure strategy.Source: §E.6 · p. 28

E.7 Latency, Memory, and Retrieval Controls

• 23.0 ms per turn is CORE’s added latency over SFT; RAG top-5 and MemoryBank controls show historical access alone does not recover its robustness.Response decoding remains the dominant cost, while routing and belief revision add only small pre-generation overhead.Source: §E.7 · p. 28