PaperlayerView exact revision on arXiv

Source-linked AI summary

When Agents Learn to Be You: Benchmarking Privacy Leakage, Impersonation Risk, and Defenses in Persona Skills

Yongli Xiang, Zhifang Zhang, Bojun Yang, Ziming Hong, Lei Feng, Miao Xu, Tongliang Liu

arXiv:2608.03700v1cs.CRcs.CLcs.CY

TL;DR

Persona skills create a distinct safety gap by compiling scattered personal signals into portable, executable artifacts. AntiSkillBench evaluates leakage, impersonation, and defenses across the pipeline, finding persistent risks and limited, protocol-dependent protection.

  • Problem

    Existing benchmarks largely overlook safety risks specific to persona skills, which distill distributed personal signals into compact, persistent, executable artifacts.

  • Method

    AntiSkillBench evaluates persona-skill risks and defenses using 7,500 interaction traces, three distillation baselines, and four online or post-hoc interventions.

  • Results

    Private and behavioral signals persist across model backbones and distillation strategies, while defenses provide limited, protocol-dependent protection against leakage and impersonation.

  • Takeaways & Limitations

    Securing persona skills requires defenses spanning artifacts, downstream behavior, and distillation to address privacy and impersonation risks.

  • Takeaways & Limitations

    The benchmark instantiates personal traces as dialogue traces between users and assistants, limiting its direct coverage of other trace forms.

Abstract

from arXiv · show

Persona skills distill personal interaction histories into portable and executable artifacts for downstream agents. While enabling flexible personalization, this process concentrates fragmented personal signals, amplifies their impact through reuse, and challenges defenses designed for individual records or retrieval-based memory. To systematically investigate the safety of the persona-skill pipeline, we introduce AntiSkillBench, an end-to-end benchmark for evaluating risks and defenses across the persona-skill pipeline. It comprises: (i) a dataset of 7,500 persona-grounded dialogue traces, constructed from 50 behaviorally rich profiles spanning diverse task scenarios; (ii) an evaluation suite that measures skill-level privacy leakage and agent-level attribute disclosure and behavioral impersonation across three skill-distillation strategies; and (iii) a defense evaluation covering four configurations across online and post-hoc interventions, including active risk suppression and passive provenance protection. Experiments across three frontier agents show that persona-skill risks persist across agent backbones and distillation protocols, extending from explicit attributes to communication styles and personality traits. Existing defenses exhibit limited and distillation-dependent effectiveness, failing to generalize across risk and distillation strategies. These results highlight AntiSkillBench as a challenging benchmark for developing privacy-preserving and authenticity-aware persona skills.

1 Introduction

Persona skills compile distributed personal signals into persistent, portable, executable artifacts, creating safety risks that existing benchmarks largely overlook. AntiSkillBench addresses this gap with end-to-end evaluation of privacy leakage, impersonation, and defenses across the persona-skill pipeline.

  • Motivation: Persona skills encode identity-centered knowledge as persistent, portable, executable rules for person-specific assistance and user-proxy interactions.Unlike parameter-bound or retrieval-based personalization, they package personal signals into reusable artifacts.
  • Safety gap: Existing benchmarks largely overlook risks created when personal signals distributed across multiple traces are distilled into compact, persistent, executable artifacts.Prior evaluations typically use raw interaction histories or lightly processed memory records.
  • Benchmark: AntiSkillBench combines 7,500 user-grounded interaction traces, three skill-distillation baselines, a comprehensive risk suite, and four defense methods.Its defenses span online/post-hoc intervention and active/passive settings.
  • Evaluation: AntiSkillBench evaluates skill-level privacy leakage and agent-level impersonation across recovered personal information and reproduced preferences, responses, and decisions in unseen contexts.The dataset enriches 50 profiles with Big Five personality traits and fine-grained communication styles.
  • Findings: Across three frontier agentic models, leakage and impersonation persist across backbones and distillation protocols, extending beyond explicit attributes to behavioral signals.The results characterize these risks as persistent across implementations and report an asymmetry between active mitigation and passive backdoor defense.

2 Related Work

Prior work has established reusable agent skills and persona-grounded artifacts, while existing benchmarks primarily evaluate privacy or impersonation in non-skill-based settings. This work builds on these foundations to examine safety risks introduced by persona-skill distillation and downstream use.

  • Agent and Persona Skills: Agent skills package reusable knowledge, instructions, and workflows that agents select for particular tasks or domains.This paradigm builds on LLM-agent systems coupling reasoning with environmental actions.
  • Agent and Persona Skills: Persona skills distill traces of target individuals into reusable artifacts encoding person-grounded knowledge and behavioral regularities for downstream agents.COLLEAGUE.SKILL studies skill generation across public-figure, colleague, and private-relationship settings.
  • Related Benchmarks: Existing benchmarks study related non-skill-based risks, including contextual secret leakage, sensitive-span extraction, typed pseudonymization, and individual-level impersonation.PrivacyBench retrieves private records into multi-turn interactions, MemPrivacy evaluates cloud-side memory processing, and IMPersona studies impersonation through prompting, fine-tuning, and retrieval.

3 Preliminaries: Persona Skill Pipeline

The persona skill pipeline collects person-grounded traces, distills them into a reusable artifact summarizing personal information and behavioral regularities, and equips an agent to generate personalized responses. In this work, traces are instantiated as general dialogue traces between a user and an assistant.

  • 3 Preliminaries: Persona Skill Pipeline: Persona skills preserve personal context, expertise, and interaction patterns so downstream agents can provide person-specific assistance across conversations.The target user’s person-grounded information may include demographic attributes, cognitive patterns, and communication style.
  • 3 Preliminaries: Persona Skill Pipeline: A skill distillation function processes collected traces into a reusable persona skill that summarizes the user’s person-grounded information and behavioral regularities.The resulting artifact is denoted as the person-grounded skill s_u.
  • 3 Preliminaries: Persona Skill Pipeline: Personal traces can be dialogue histories, interviews, documents, messages, or multimedia-derived transcripts, and each trace may reveal part of the user’s person-grounded information.The work instantiates traces as general dialogue traces between a user and an assistant.
  • 3 Preliminaries: Persona Skill Pipeline: When equipped to an agent, the persona skill shapes the output for an input query or task, producing a personalized response that reflects the user.Skill distillation and skill-equipped generation together constitute the persona skill pipeline.

4 AntiSkillBench

AntiSkillBench evaluates privacy leakage and impersonation risks across the persona-skill pipeline using persona-grounded dialogue traces, complementary metrics, and trace-side defenses. It covers both sensitive information encoded in skills and target-user-like behavior produced by equipped agents.

  • Dataset Construction: AntiSkillBench contains 50 user roles paired with 50 questions and expanded into three-round dialogues, yielding 2,500 traces and 7,500 persona-grounded user turns.The traces are designed to be task-plausible and person-revealing, including richer reasoning and interaction patterns.
  • Threat Model: The benchmark separates skill-level privacy leakage, where skills encode private information, from agent-level impersonation, where equipped agents generate target-user-like responses.Evaluated signals include explicit identifiers, background, preferences, personality, decision patterns, and communication style.
  • Evaluation Metrics: Skill Coverage measures preserved person-grounded information, while Field QA accuracy measures direct attribute disclosure and VocabGain measures target-user-like communication.Higher Skill Coverage indicates more severe skill-level leakage, and higher VocabGain indicates stronger impersonation capability.
  • Evaluation Metrics: VocabGain evaluation compares skill-equipped, no-persona, and oracle-profile agents across category-specific scenarios using user- and category-specific lexical and discourse markers.The score normalizes the skill-equipped agent’s improvement over the no-persona baseline by the oracle gap.
  • Defense Settings: Defense evaluation restricts intervention to user traces before distillation and covers active defenses that reduce or mislead identity inference and passive defenses that provide provenance signals.Privacy Sanitization alters expression forms while preserving task intent, whereas Semantic-level Backdoor Injection embeds semantic watermarks for tracing unauthorized distillation.

5 Experiments

Experiments on AntiSkillBench evaluate privacy leakage, impersonation risk, and defense effectiveness across 50 user roles, three agent backbones, and three distillation protocols. Results show substantial leakage and actionable impersonation, while active and passive defenses provide limited or distillation-dependent protection.

  • Experimental setup: The main evaluation covers all 50 user roles with 50 dialogues per user, while ablations vary dialogue quantity.The benchmark examines how available dialogue volume affects privacy leakage, impersonation risk, and defense effectiveness.
  • Experimental setup: Three distillation protocols are compared over the same canonical user history: Direct Distill, three-stage Distill, and Colleague Distill.Direct Distill is a one-step full-history baseline; three-stage Distill introduces structured intermediates.
  • Privacy leakage: For GPT 5.4, overall Skill Coverage reaches 66.2 for three-stage, 63.6 for Direct Distill, and 55.2 for Colleague Distill.Leakage is strongest for background, personality, and communication style, rather than being limited to explicit demographics.
  • Impersonation risk: GPT 5.4 with three-stage Distill reaches 56.0 overall QA Acc, including 75.9 for communication traits, while VocabGain is positive across all four dimensions.The corresponding VocabGain values are 22.3 for demographics, 37.1 for background, 17.6 for personality, and 87.7 for communication.
  • Defense effectiveness: Online PS reduces communication VocabGain from 87.3 to 6.5 and communication QA Acc from 75.5 to 54.8 under Direct Distill, but personality Skill Coverage remains 70.0.Post-hoc ADV yields smaller reductions, while passive-defense effectiveness varies substantially across distillation methods.

6 Conclusion

The paper identifies persona skills as an emerging safety surface because portable, executable artifacts can preserve private attributes and behavioral signals that enable downstream disclosure and impersonation. It introduces AntiSkillBench to evaluate these exposures and defenses across the persona-skill pipeline.

  • Safety surface: Persona skills can preserve private attributes and behavioral signals from user traces, enabling downstream disclosure and impersonation.The risk arises when interaction histories are distilled into portable, executable artifacts for personalized agent systems.
  • Benchmark: AntiSkillBench combines controlled persona-grounded traces with evaluations of skill-level exposure and agent-level exploitation.The benchmark is designed to examine both what skills expose and how downstream agents exploit those exposures.
  • Defenses: AntiSkillBench evaluates defenses spanning online and post-hoc interventions.Its defense coverage is part of an end-to-end evaluation of the persona-skill pipeline.

Overview of the Appendices

The appendices document AntiSkillBench’s construction, illustrate distilled persona skills, and provide implementation details with qualitative examples.

  • Appendix A: Appendix A details dataset construction, including structured user records, representative samples, and prompts for persona-grounded question generation and multi-turn simulation.
  • Appendix B: Appendix B presents distilled persona skills showing how user traces become reusable artifacts while retaining personal information.
  • Appendix C: Appendix C provides implementation details and qualitative examples.

A Dataset Construction Details · A.1 Dataset Sample · A.2 Generation Prompts

AntiSkillBench constructs persona-grounded dialogue traces from structured user records that encode explicit attributes, background, personality, and communication patterns. Its sample traces span distinct scenarios, surface multiple persona dimensions, reveal reasoning and language-style cues across turns, and use separate prompts to regulate generation and simulation.

  • A Dataset Construction Details: Each target user is represented by a structured record capturing explicit profile attributes, background, personality, and communication patterns for question generation and dialogue simulation.The record is used consistently to produce task-plausible and person-representative traces.
  • A Dataset Construction Details: The dataset construction process combines structured persona information, scenario-specific requests, profile-reflecting prompts, and multi-turn simulations to produce person-representative traces.These components jointly connect explicit and implicit persona signals with interaction behavior.
  • A Dataset Construction Details: The records cover demographics, experience and education, Big Five traits, and detailed language-style dimensions such as rhythm, tone, rhetoric, humor, and argument strategy.Communication features also include sentence style, vocabulary, interaction style, certainty, reference style, and other cues.
  • A.1 Dataset Sample: The sample separates general, tool-design, and mathematical-calculation questions while keeping scenario coverage distinct from persona grounding.General questions address open-ended assistant use, tool-design questions request concrete functionality, and mathematical questions embed calculations in domain-specific contexts.
  • A.1 Dataset Sample: Five sampled prompts reflect occupation, age, birthplace, education, financial life stage, and communication style, exposing demographic, background, personality, preference, and linguistic signals.The examples are selected from one user’s 50 generated prompts.
  • A.1 Dataset Sample: A three-round trace shows multi-turn interaction revealing comparative critical reasoning, rejection of abstraction, premise-to-execution refinement, and scene-level operational criteria.The progression exposes analytical reasoning patterns alongside language-style cues that are harder to observe from a single prompt.
  • A.2 Generation Prompts: Separate prompts govern initial question generation and multi-turn user simulation, with reflection rules constraining coverage, scenario balance, and language-style consistency.Question generation targets diverse, realistic persona-grounded requests across professional life, personal life, and internal thoughts.

B Sample Distilled Skill · C Defense Implementation Details

The distilled skill removes original dialogue turns while preserving diverse person-grounded information for reuse. The excerpt retains expertise, habits, preferences, communication patterns, cultural register, and qualified background inferences.

  • B Sample Distilled Skill: The three-stage protocol compresses user traces into a reusable skill artifact without retaining the original dialogue turns.Figure 3 presents an excerpt illustrating this transformation.
  • B Sample Distilled Skill: The distilled skill preserves the user’s domain expertise as person-grounded information.
  • B Sample Distilled Skill: It retains interaction habits that characterize how the user engages with tasks.
  • B Sample Distilled Skill: It captures preferred output formats, enabling reuse of the user’s formatting preferences.
  • B Sample Distilled Skill: The artifact preserves communication style and cultural register as personal signals.
  • B Sample Distilled Skill: It also includes confidence-qualified background inferences about the person.

C.1 Privacy Sanitization Implementation … D.2 Field QA Evaluation

The paper implements three defenses with distinct online or post-hoc transformations, then evaluates distilled skills through static coverage and agent-level field question answering. These procedures target privacy leakage, impersonation-relevant attributes, behavioral triggers, and retained persona information.

  • C.1 Privacy Sanitization Implementation: Privacy Sanitization intervenes online, sanitizing the first request and regenerating later user messages and assistant responses from the defended dialogue state.The sanitizer preserves task operations, explicit constraints, and necessary facts while removing or generalizing unnecessary identity and stylistic cues.
  • C.2 Adversarial Obfuscation Implementation: Adversarial Obfuscation appends a follow-up user turn and simulated assistant response containing coherent, incorrect decoy values for exposed attributes.Validation checks that injected attributes conflict with the reference profile, remain globally consistent, and appear as factual claims.
  • C.3 Semantic-level Backdoor Injection Implementation: Semantic-level Backdoor Injection links a narrow verification trigger to a rare parenthetical mood-tag behavior selected by deterministic hashing.Online injection begins at the second user turn, whereas post-hoc injection appends one verification turn and one simulated assistant response after the final response.
  • C.4 Qualitative Examples: Qualitative examples illustrate compact trace transformations while omitting the full defender prompts.Table 6 provides representative examples of the transformations.
  • D Evaluation Details: The evaluation details assess whether persona information is preserved in distilled skill artifacts and whether agents equipped with those skills answer attribute-level questions as target users.Skill coverage uses judge questions against reference properties, while Field QA uses a fixed questionnaire judged against ground-truth fields.
  • D.1 Skill Coverage Evaluation: Skill coverage measures each evaluated property through binary LLM-judge decisions comparing the skill document with the reference user property.Each judgment includes a short rationale for auditing.
  • D.1 Skill Coverage Evaluation: The number and scope of skill-coverage questions depend on property structure, including five Big Five dimensions and nine language-style aspects.Simple properties use one question each; the listed language-style aspects include sentence style, vocabulary, tone, rhythm, and certainty level.
  • D.2 Field QA Evaluation: Field QA evaluates whether a persona-skilled agent answers a fixed cross-user questionnaire consistently with each target user’s ground-truth fields.Answers are judged against the corresponding fields in Pu.

D.3 VocabMatch Evaluation Details … E More Experiments

The paper details semantic VocabMatch scoring with category-level normalization safeguards, then evaluates injected behaviors through triggered interactions and static skill-artifact detection. These evaluations distinguish intended conversational activation from mere persistence and include false-positive baselines from undefended histories.

  • D.3 VocabMatch Evaluation Details: VocabMatch builds compact, user-specific marker sets from persona descriptions, grouped by the persona categories used in the main evaluation.Markers represent phrases likely to appear incidentally in the target user’s real requests.
  • D.3 VocabMatch Evaluation Details: VocabMatch credits semantically close variants by embedding-based soft matching over 1-4 word n-grams rather than exact string matches.The supplied scoring description defines query chunks and marker hits using embedding similarity.
  • D.3 VocabMatch Evaluation Details: τ = 0.75; categories with non-positive oracle gaps or target expectation below 0.1 are skipped for normalization and treated as zero in overall VocabGain.This aggregation rule avoids upward bias from selectively averaging only categories with valid normalization signals.
  • D.4 ASR Evaluation Details: Behavioral ASR tests whether a distilled skill reproduces injected behavior during simulated target-user conversations and under the intended trigger.The setup uses an assistant that responds helpfully while leaving room for a natural follow-up across three fixed topics.
  • Behavioral ASR: A behavioral turn is triggered only by explicitly requesting verification of a specific number, calculation, or exact claim, while the target detector checks the ending mood or mental-state tag.Both decisions are semantic rather than exact phrase matches.
  • Static ASR: Static ASR measures whether injected semantic behavior appears in the distilled skill artifact, using binary semantic judgments over complete skill documents.For each user-canary pair, an LLM judge answers the detector question, allowing wording different from injected examples.
  • Static ASR: Static ASR applies the same detectors to undefended distilled skills and reports activation rates from undefended histories as a false-positive baseline.It captures watermark survival in the artifact, whereas behavioral ASR tests whether activation remains tied to intended conversational context.

E.1 Ablation Analysis · E.2 Defense Effectiveness on Claude Haiku 4.5 · F API Services and Estimated Cost

The ablations show that dialogue exposure, sanitization strength, and backdoor coverage shape persona leakage and defense behavior, while Claude Haiku 4.5 retains strong backdoor detectability despite active defenses. The paper also reports hosted-API execution details and approximate per-character costs based on observed usage.

  • E.1 Ablation Analysis: Full dialogue access raises Skill Coverage from 50.0 to 63.3 and QA Acc from 50.0 to 56.4, showing recoverable persona information even in limited traces.QA Acc peaks at 60% usage, while Skill Coverage is highest with full dialogue access.
  • E.1 Ablation Analysis: Full sanitization produces the largest Skill Coverage reduction at -14.1%, while QA Acc declines by 7.2% and changes less consistently.Skill Coverage decreases at every evaluated sanitization strength relative to the undefended baseline.
  • E.1 Ablation Analysis: At 20% backdoor coverage, ASR reaches at least 80.0% and remains predominantly 80.0-100.0% thereafter for both online and post-hoc injection.The ablation attributes persistence to repeated representation of the trigger-target association during skill distillation.
  • E.2 Defense Effectiveness on Claude Haiku 4.5: Claude Haiku 4.5 shows the same active-defense pattern as GPT 5.4: Online PS reduces leakage more consistently than Post-hoc ADV, while personality information remains resistant.Unlike GPT 5.4, Claude retains high ASR-S across all three distillation protocols under passive defenses.
  • E.2 Defense Effectiveness on Claude Haiku 4.5: Online PS lowers overall Skill Coverage to 49.6, 48.8, and 47.4 across three-stage, Direct, and Colleague Distill, respectively.It also lowers overall QA Acc to 44.9, 42.0, and 40.0, with the strongest reductions in communication signals.
  • E.2 Defense Effectiveness on Claude Haiku 4.5: Online SBD reaches 98.0 ASR-S under three-stage and Direct Distill and 94.0 under Colleague Distill, while Post-hoc SBD reaches 100.0, 94.0, and 90.0.The backdoor signal survives Claude-based distillation regardless of online or post-hoc introduction.
  • F API Services and Estimated Cost: Agentic inferences used hosted official CLIs for gpt-5.4-medium, claude-haiku-4-5, and gemini-3.6-flash-medium between May and June 2026, without fixed generation seeds.Each configuration ran once for each of the 50 characters.
Loading 2608.03700v1…