Inferring Hidden User Models from the Behavior of Personalized LLM Agents

1. Introduction

• Removing source wording can reduce direct extraction targets, but retained information can still shape visible choices from which an external observer may infer it.A choice may fit several explanations, so distinguishing them requires additional tasks where those explanations predict different behavior.Source: §1 · p. 1
• UMPeek forms hypotheses from visible choices left open by a request, selects adaptive follow-up tasks, and accepts only claims that are supported and not contradicted by behavior.The method tests semantic claims without accessing or decoding the backend representation.Source: §1 · pp. 1–2
• Across four benchmarks, UMPeek’s semantic-recovery score is more than six times that of the strongest comparison attack, and it leads in every benchmark-backend setting.Matched-budget studies attribute the advantage to hypothesis-guided request selection rather than query volume alone.Source: §1 · p. 2
• The paper formalizes black-box recovery without source-record or backend access and compares UMPeek with seven attacks across four benchmarks and three backends.The evaluation also uses ablations to trace gains to task-aligned follow-ups and claim acceptance.Source: §1 · p. 2
• On every evaluated managed service, UMPeek achieves the highest recovery among the evaluated attacks for retained targets, while response-level defenses leave substantial recovery under adaptive follow-ups.Stronger stateful control reduces both recovery and personalized task utility.Source: §1 · p. 2

2. Background and Threat Model

• Memory preserves information across interactions, whereas a user model supplies user-specific information to guide personalization decisions; the concepts can overlap.A memory representation functions as a user model when the system brings it into a decision.Source: §2 · p. 2
• User models can take forms such as selected records, readable profiles, learned embeddings, or structured attributes, and may be compact or abstracted from individual events.These forms guide decisions without reproducing the source history verbatim.Source: §2 · p. 2
• When records and backend state are inaccessible, attacks designed to recover readable records lose their target, but retained information may still shape visible responses and actions.The remaining question is whether an external participant can infer retained information from choices through ordinary interaction.Source: §2 · p. 2

2.1. Personalized LLM Agent

• For a request, the system may select, transform, or compute over the broader hidden user model to assemble request-conditioned state that shapes its visible answer or tool action.Different requests can assemble different states from the same retained user model.Source: §2.1 · pp. 2–3

2.2. Attacker and Recovery Goal

• The attacker is an external participant using a finite adaptive sequence of ordinary task requests in one domain, observing only returned text and user-visible actions.The attacker cannot ask the agent to reveal its memory or modify the system, tools, or retained state.Source: §2.2 · p. 3
• The attacker needs no auxiliary information about the target and no knowledge of the model, prompts, personalization pipeline, backend representation, or source records.Public task and interface descriptions are known, and behavior may be analyzed offline with public models.Source: §2.2 · p. 3
• Recovery returns human-readable claims tied to visible choices, and a plausible claim receives no credit unless it matches the hidden state and is behaviorally attributed.Recovery may be partial; claims indistinguishable from plausible alternatives remain unrecovered.Source: §2.2 · p. 3

3. UMPeek: Hypothesis-Guided Adaptive Probing

• UMPeek starts from a concrete choice left open by the request, generates possible user-specific explanations, and uses follow-up tasks to test which explanations fit visible behavior.It observes ordinary-interface behavior and returns a semantic estimate rather than decoding backend state.Source: §3 · p. 3

3.1. Constructing Behavioral Hypotheses

• Decision dimensions bound the hypothesis space using public task and interface descriptions, and a claim template names a possible preference, constraint, or relation without fixing its value.A candidate claim is formed only when the request leaves a dimension open and the response makes a concrete choice on it.Source: §3.1 · p. 4

3.2. Adaptively Testing Behavioral Hypotheses

• Replaying a task tests whether a choice recurs, while switching tasks tests whether the same hypothesis predicts a compatible choice under changed conditions.Replay alone does not attribute a recurring choice to the hidden user model; requests are selected to distinguish unresolved claims and values.Source: §3.2 · pp. 4–5

3.3. Selective Recovery from Behavioral Evidence

• Accepted claims require a concrete visible choice, no contradictory behavior, and, for implicit claims, evidence beyond their first appearance; unresolved alternatives remain unrecovered.Omission is not evidence, and a request that fixes a value cannot support attribution of that value to personalization.Source: §3.3 · p. 5

4. Experimental Setup

• The benchmark suite includes 16,323 interactions per backend across four datasets, totaling 48,969 interactions that expose personalization through choices, task-oriented answers, and tool actions.Three held-out tasks per target test whether recovered estimates predict other personalized decisions.Source: §4 · p. 5
• Benchmark and defense evaluations connect the same agent to Mem0, Graphiti, and LangMem+LangGraph, while keeping the downstream model and black-box interface fixed.Cross-backend results measure semantic recovery rather than reconstruction of storage representations.Source: §4 · pp. 5–6
• UMPeek is compared with seven attacks under the same black-box access and evaluation-specific request cap, with outputs converted to a common semantic-claim format.The baselines span memory extraction, privacy assessment, prompt leakage, indirect injection, and attribute inference.Source: §4 · p. 6
• UMR-F1 measures semantic agreement with reference claims, while HBPS measures whether recovered claims predict behavior on three unseen tasks.The metrics capture different outcomes: predicting an option need not identify the responsible user information, and a correct claim need not affect every later task.Source: §4 · p. 6

5.1. Attack Performance

• UMPeek reaches 0.619 UMR-F1 versus 0.100 for the strongest semantic-recovery baseline, and 0.289 HBPS versus 0.147 for the strongest behavior-prediction baseline.The pooled comparison covers four benchmarks and three personalization backends.Source: Table 1 · p. 6, §5.1 · p. 6
• UMPeek retains nonzero recovery in all 12 benchmark-backend combinations, with UMR-F1 ranging from 0.467 to 0.817.Backend ordering changes across benchmarks, so the attack surface is not tied to one storage or retrieval organization.Source: Table 2 · p. 7, §5.1 · p. 7
• LoCoMo has higher UMR-F1 but lower HBPS than ETAPP across all three backends, making semantic recovery and behavioral prediction complementary outcomes.A broad estimate may recover correct claims that do not affect a particular unseen task; a less complete estimate may contain the claim driving one decision.Source: §5.1 · p. 7

5.2. Adaptive Request Selection and Ablations

• On the balanced subset, selected follow-ups raise UMR-F1 to 0.606 before acceptance yields 0.651 UMR-F1 and 0.634 claim precision.The cumulative figure tracks UMR-F1 and claim precision after each processing stage.Source: Figure 1 · p. 7, §5.2 · p. 7
• With about 15 follow-ups, full UMPeek reaches 0.651 UMR-F1 versus 0.281 for a fixed sequence; acceptance chiefly improves claim precision.Adaptive selection distinguishes competing explanations, while acceptance filters explanations lacking sufficient behavioral support.Source: §5.2 · p. 7, Table 3 · p. 8

6.1. Evaluated Systems and Protocol

• The evaluation covers three managed memory services and OpenClaw, using synthetic histories to construct personalized states before attacks.Managed-service comparisons use 20 confirmed-retained targets per service and cap each method at three requests; scores are conditional on retention, not on all source claims.Source: §6 · p. 8, §6.1 · p. 8

6.2. Cross-System Recovery

• UMPeek balances macro precision and recall at 0.500, while Imprompter reaches 0.450 recall with 0.113 precision.On Mem0, UMPeek improves both precision and recall, indicating that its selectivity does not simply trade coverage for fewer errors.Source: §6.2 · p. 8
• UMPeek achieves a macro UMR-F1 of 0.500 across managed services, versus 0.180 for the strongest comparison method.It has the highest observed UMR-F1 on each service; Figure 2 compares semantic recovery and choice prediction across eight methods on 20 retained targets per service.Source: §6.2 · p. 8, Figure 2 · p. 9
• On AWS AgentCore, UMPeek and Imprompter have nearly identical Choice F1, but UMPeek’s UMR-F1 is two and a half times higher.Confidence intervals separate UMPeek from comparison methods on AWS and Mem0 but overlap on Google; the reported gap on Google and AWS is driven mainly by precision.Source: §6.2 · pp. 8–9

6.3. Semantic Recovery in Visible Behavior

• Visible choices reveal user-level meaning in examples ranging from a travel-photography preference to a preference for calm aquatic environments.The Mem0 recovery uses wording different from the retained claim; an OpenClaw scenic-bike recommendation reveals its target preference in the first ordinary response.Source: §6.3 · p. 9

6.4. Interaction Budget and OpenClaw Boundary

• Within three requests, UMPeek has the highest observed UMR-F1 on all three managed services, with no method achieving higher recovery using the same or fewer requests.The comparison concerns use of a limited request budget, not whether UMPeek always issues the fewest requests.Source: §6.4 · p. 9
• Every selected OpenClaw target’s first interaction contains a visible choice matching the retained preference, but these retention-conditioned cases do not estimate exposure over all stored information.Thus, cross-request accumulation is not always necessary once personalization makes the relevant information visible.Source: §6.4 · p. 9

6. Real-World Validation

• Across evaluated retained targets, visible behavior supports recovery on independently implemented systems when retained information shapes agent choices.The managed-service protocol keeps retention data, reference claims, and platform state confined to evaluation while attacks use ordinary requests and visible behavior.Source: §6.1 · p. 8, §6.4 · p. 9

7. Adaptive Defenses

• PrivacyChecker checks contextual information flows, Theory-of-Mind reasons about requester use, and Stateful Counterfactual compares personalized choices with choices made without personalization.The latter withholds behavior when accumulated evidence exceeds a threshold.Source: §7 · p. 10
• Stateful Counterfactual lowers macro UMR-F1 to 0.187, but permitted personalization-dependent choices still leave evidence for recovery.It withholds decisions that change when personalization is removed.Source: §7 · pp. 10–11, Table 4 · p. 12
• The defense with the lowest macro UMR-F1 does not have the lowest HBPS or highest TaskScore on every benchmark.Semantic recovery, behavioral exposure, and task utility therefore require separate evaluation.Source: §7 · p. 11, Table 4 · p. 12
• No evaluated Stateful Counterfactual threshold both eliminates recovery and matches undefended task performance; less restrictive release raises TaskScore on all four benchmarks and UMR-F1 on three.The trade-off reflects decisions that can both improve task results and reveal the user information behind them.Source: §7 · p. 11, Figure 6 · p. 12
• After 16 follow-ups, recovery reaches about 65% of the undefended level under PrivacyChecker and 90% under the Theory-of-Mind defense.Initial-response-only evaluation would overstate their resistance to an adaptive attacker.Source: §7 · p. 11
• Removing cross-request tracking barely changes recovery or task performance, whereas removing counterfactual comparison raises UMR-F1 and lowers TaskScore on every benchmark.The ablation identifies counterfactual comparison, rather than cross-request tracking, as the active component on this subset.Source: §7 · pp. 11–12, Table 5 · p. 13
• A defense must limit how confidently sequences of released choices reveal retained information while preserving choices needed for the task.Screening isolated responses can miss information that becomes informative after another task rules out a competing explanation.Source: §7 · pp. 12–13

8. Related Work

• Persistent personalization uses profiles, embeddings, compressed contexts, and structured memories; key-fact summarization can reduce direct canary extraction, while UMPeek targets retained meaning without history or derived-state access.The paper connects persistent user state and direct extraction with behavioral inference and agent privacy controls.Source: §8 · p. 13
• Prior attacks recover training examples, memories, retrieved content, membership, or hidden prompts, while UMPeek infers semantic user information from ordinary visible behavior without backend access or tool control.Other approaches also target exposed embeddings and internal states or manipulate tool actions.Source: §8 · p. 13
• User-inference work uses predictions, activity, recommendations, text, public traces, or interaction dynamics; UMPeek adaptively tests explanations of inaccessible state through task-dependent choices.Privacy audits and controls address available private data, whereas UMPeek studies information implied by personalization-dependent choices, including information never stated.Source: §8 · p. 13

9. Conclusion

• UMPeek identifies a behavioral privacy surface distinct from direct access to records or representations, testing how retained information affects decisions rather than decoding stored state.The conclusion reports this mechanism across benchmarks, local backends, services, a personal agent, and evaluated inference-time defenses.Source: §9 · p. 13
• Reducing the behavioral effect of retained information can lower recovery but also remove useful personalization, creating a need for interaction-level controls.The stated aim is to limit what repeated choices reveal while preserving their adaptation.Source: §9 · p. 14

Appendix A. Ethical Considerations

• UMPeek has dual-use potential to infer real users’ information from choices that do not state it directly; live-service testing may also consume resources or alter personalized state.The authors judged controlled study warranted because existing evaluations can miss this channel and its measurement may help develop defenses.Source: Appendix A · p. 18
• The study recruited no participants or accessed real-user accounts; real-world validation used synthetic histories and offline evaluation used released benchmark data.Managed-service tests used dedicated author-controlled resources and isolated synthetic identities.Source: Appendix A · p. 18
• Released artifacts exclude account identifiers, credentials, private platform state, raw external-service records, and third-party source data, but do not eliminate dual-use potential.Third-party datasets, credentials, provider resources, and internal service state are also excluded from the accompanying artifacts.Source: Appendix A · p. 18

• PersonaMem-v2 contributes 5,000 multiple-choice interactions from 200 personas, and PersonaLens contributes 5,000 task-oriented interactions from 79 users [47].Source: Appendix B · pp. 18–19
• ETAPP pairs 150 instructions with 32 profiles to form 4,800 cases, while LoCoMo contributes 1,523 question-answer pairs from 20 speakers across 10 conversations.Source: Appendix B · p. 19
• UMR precision and recall use one-to-one claim matches against recovered estimates and reference claims; UMR-F1 is their harmonic mean.Empty estimates receive zero precision, and reported UMR-F1 scores average target-level values across 48,941 evaluations with at least one reference claim.Source: Appendix B · p. 19
• Held-out behavior prediction uses recovered claims to predict unseen-task behavior, with scoring tailored to choices, rankings, tool calls, or free-form answers.Each unseen task is selected so user-specific information relevant to the target interaction can also affect its reference behavior.Source: Appendix B · p. 19

• Only targets whose personalized state contains information semantically equivalent to the reference claim enter the behavior comparison, and attack methods do not receive the retention result.Source: Appendix C · p. 19
• Google Memory Bank, AWS AgentCore Memory, and Mem0 Platform each contribute 20 retained targets, tested with all eight methods under a three-request cap per target.Google and Mem0 use Gemini 2.5 Flash-Lite, while AWS uses Amazon Nova 2 Lite; results were collected in August 2026.Source: Appendix C · p. 19
• The OpenClaw evaluation tests 25 retained claims, selecting one claim per persona from 28 claims retained across 25 of 50 synthetic-persona agents.These targets are excluded from ranking the eight methods in the managed-service comparison.Source: Appendix C · p. 19

• The main comparison samples 128 targets per benchmark-backend pair, allows an initial interaction plus at most 16 follow-ups, and holds state, procedure, and request cap fixed across conditions.UMPeek selects later requests using only behavior released by the defense.Source: Appendix D · p. 19
• TaskScore applies each benchmark’s original task metric to the evaluated personalized tasks, [47],,.Source: Appendix D · p. 19
• The threshold study reuses 32 targets per benchmark-backend pair at thresholds 0.5, 1.0, and 1.5; ablations use the same subset at 1.0.Uncertainty estimates cluster outcomes by benchmark target across the three local backends, and macro scores are unweighted means over four benchmarks.Source: Appendix D · p. 19