Source-linked AI summary
SkillJack: Persistent Skill Backdoors in Self-Evolving Agents
Zonghao Ying, Xiangfan Wu, Huiyu Wu, Xing Zheng, Huangsheng Cheng, Xiaorong Shi, Jing Guo
TL;DR
Self-evolving agents can transform poisoned experiences into persistent skills, extending poisoning beyond retrieved context. SkillJack exploits this experience-to-skill pipeline, reducing detection from 98.5% to 11.4% while retaining attack effectiveness and persistence after source deletion.
Problem
Existing poisoning research focuses on retrieved memories or context, leaving the security implications of automatically converting poisoned experiences into reusable skills insufficiently studied.
Method
SkillJack uses transformation-resilient payloads designed to preserve policy-relevant actions while producing benign-looking skills through automated extraction.
Results
Detection fell from 98.5% for raw trajectories to 11.4% for extracted skills, while attack success reached 56.2% in SkillX, 89.2% in Anything2Skill, and persistence after deletion was 80.0%.
Takeaways & Limitations
Skill evolution is a distinct attack surface, so clearing poisoned records alone may not remove derived malicious skills.
Takeaways & Limitations
Results use one model and a policy-violation proxy rather than live external-service execution, so figures are routing-level rather than deployment-level.
Abstract
from arXiv · showhide
Self-evolving agents increasingly convert interaction histories into reusable skills that persist beyond individual tasks. While prior work studies memory and retrieval poisoning, such attacks only affect agents when poisoned records are retrieved as context. We uncover a new and more fundamental risk: poisoned experiences can be transformed by the agent itself into durable behavioral artifacts. We present \textbf{SkillJack}, the first attack that exploits the experience-to-skill pipeline of self-evolving agents. Instead of directly manipulating runtime context, SkillJack hijacks the agent's own learning process to implant malicious behaviors into its reusable skill repertoire. We identify three key properties of this transformation: \emph{sanitization whitewashing}, where malicious intent is obscured during skill extraction; \emph{cross-layer promotion}, where transient experiences become persistent capabilities; and \emph{persistence isolation}, where the attack survives removal of its original source records. We evaluate SkillJack on two representative systems, SkillX and Anything2Skill, using a shared dataset of 150 trajectories across four policy-risk categories. Results show that skill extraction substantially reduces attack detectability: in SkillX, safety detection drops from 98.5\% for poisoned trajectories to 11.4\% for extracted skills, while Anything2Skill shows a similar effect. Meanwhile, the implanted skills remain effective, achieving attack success rates of 56.2\% and 89.2\% on the two systems, respectively. Furthermore, 80.0\% of skill-mediated attacks persist after deleting the original poisoned records, and some skills unintentionally activate on benign queries. Our findings reveal skill evolution as a new attack surface and motivate provenance-aware skill lifecycle protection. Our code is available at https://github.com/Tencent/AI-Infra-Guard/research/skilljack.
1. Introduction
SkillJack identifies experience-to-skill automation as a distinct attack surface in self-evolving agents, where poisoned experiences become durable, reusable skills rather than merely affecting retrieved context. The study characterizes sanitization whitewashing, cross-layer promotion, and persistence isolation, and evaluates these properties in SkillX and Anything2Skill.
- Motivation: Experience-to-skill automation can transform an interaction record into a reusable skill that shapes future behavior, making experience a persistent security concern.Unlike transient context, extracted skills can become transferable procedures used in later tasks.
- New attack surface: SkillJack targets a distinct risk from conventional memory poisoning by hijacking how self-evolving agents learn from poisoned experience.Prior attacks typically influence behavior when poisoned records are later retrieved as memory or context.
- Security properties: 98.5% to 11.4%: extraction reduces LLM-judge detection from raw poisoned trajectories to extracted skills, demonstrating sanitization whitewashing.The attack obscures malicious intent during skill extraction.
- Security properties: 56.2% in SkillX and 89.2% in Anything2Skill: derived skills trigger the intended violation on attack-oriented tasks, demonstrating cross-layer promotion.Experience-layer inputs become reusable artifacts that can be selected for later tasks without rereading the original records.
- Evaluation: SkillJack is evaluated in SkillX and Anything2Skill using the same 150-trajectory dataset spanning four policy-risk categories.The evaluation measures extraction effects, downstream skill behavior, persistence after source deletion, collateral routing, and candidate defenses.
2. Overview
Experience-to-skill pipelines transform historical experience into persistent, reusable skills that are routed to future tasks. SkillJack targets this transformation by influencing the experience layer without directly modifying the skill library or extraction and routing components.
- Experience-to-Skill Pipelines: Experience-to-skill pipelines compile trajectories, interaction logs, or documents into reusable skills and route those skills to future tasks.Implementations differ in how they load, transform, persist, and route experience.
- Evaluated Systems: SkillX and Anything2Skill are the two representative experience-reuse systems evaluated in this report.They reuse experience through reusable instructions, workflows, or skills.
- Attacker Model: The attacker can inject a poisoned experience record into the learning corpus through indirect content injection, shared experience pools, or compromised trajectory datasets.Direct write access to the skill library is not assumed.
- Attacker Model: The attacker cannot modify the skill library or extraction code, control routing deterministically, or alter unchanged extraction and routing components.The attacker only needs to influence the experience layer before the system extracts a reusable skill into a separately stored artifact.
3. Method
SkillJack models experience-to-skill attacks as a four-stage pipeline in which poisoned experiences become routed, persistent skills that evade detection and induce unintended behavior. Its methodology uses narrative-framed paired records to test whether extraction preserves policy-relevant actions while reducing detectability.
- Pipeline formalization: The pipeline loads and transforms an experience item into a structured skill, persists it in the skill library, and routes relevant skills to new queries.Formally, Π = (𝜋𝐿, 𝜋𝑇, 𝜋𝑃, 𝜋𝑅), with skills represented as (name, description, code).
- Threat model: The attacker injects a poisoned experience whose derived skill evades detection, is routed to a target task, and contributes to unintended behavior.The objective is defined over the derived skill 𝑠ₚ = Π(𝑒ₚ), detector D, router 𝜋R, and malicious-behavior set B.
- Attack properties: The method studies sanitization whitewashing, cross-layer promotion, and persistence isolation as distinct transformations of poisoned experiences into durable attack artifacts.Whitewashing lowers pD(Π(𝑒ₚ)) relative to pD(𝑒ₚ); promotion enables later routing without rereading the source; isolation allows derived skills to remain after source deletion.
- Payload construction: The attack design hypothesizes that extraction better preserves action sequences embedded in plausible task narratives than sequences expressed in overtly adversarial language.The obstacle is that compression and abstraction may drop actions or rewrite them into suspicious forms, so naive malicious trajectories can be filtered or detected.
- Evaluation design: Paired records are evaluated for extraction, detectability, and routing, separating overt wording effects from extraction-time normalization and downstream metadata.The outcomes cover whether skills contain policy-relevant actions, whether detectors flag them, and whether routers select them for later tasks.
4. Evaluation
Evaluation across SkillX and Anything2Skill shows that experience-to-skill extraction substantially lowers detector flags while preserving downstream triggering and enabling attacks to persist after source-record deletion. The results also show system-dependent extraction yields, collateral activation on benign tasks, and sensitivity to skill metadata.
- Evaluation setup: The evaluation uses 150 trajectories, two proxy detectors, and routing-level trigger or ASR measures rather than certified security controls or live deployment success.The dataset contains 65 framed poisoned, 65 directly malicious baseline, and 20 clean trajectories; detectors combine regex pattern matching with an LLM judge.
- Detection and whitewashing: 98.5% raw-trajectory LLM-judge detection falls to 11.4% for extracted skills, while pattern detection drops from 44.6% to 11.0%.The same whitewashing gap appears across SkillX and both A2S modes, although extraction yield and output representation differ.
- Cross-system extraction: A2S DOC mode lowers LLM-judge detection from 100.0% on raw trajectories to 44.4% on extracted framed outputs while compiling skills from 41.5% of framed trajectories.SDK mode extracts 10.8% of framed trajectories and has 28.6% derived LLM-judge detection; directly malicious baselines are rejected in DOC mode.
- Persistence: 80.0% of skill-pathway attacks still trigger after poisoned source trajectories are deleted, showing that source-memory purging does not remove the derived attack.Post-deletion triggering is 32.0%, compared with 40.0% before deletion; the direct-context baseline triggers more frequently at 68.0%.
- Downstream triggering: 56.2% of SkillX attack-oriented tasks and 89.2% of A2S attack-oriented queries trigger the policy-violation proxy, while clean-task activation reaches 20.0% and 95.0%, respectively.These are routing-level measures; A2S per-category attack rates range from 80.0% to 100.0%, and metadata inclusion changes framed-skill detection from 36.7% to 10.0%.
5. Discussion
The discussion shows that experience-to-skill extraction can weaken security checks and motivates provenance-aware, behavior-aware defenses. However, the findings remain exploratory and scoped to controlled, proxy-based evaluation with one model and two systems.
- Security implications: Extraction is not security-neutral because it changes vocabulary, metadata, and representation, weakening detectors calibrated on raw trajectories.Detection should therefore be evaluated at every artifact boundary rather than assumed to transfer from source memory to derived skills.
- Scope of generalization: Evidence from two independently implemented systems suggests the extraction and downstream-triggering risks are not tied to one implementation.Whether analogous risks affect reflection memories or tool synthesis remains a hypothesis.
- Defense directions: Provenance should propagate from trajectories to derived skills so deleting or quarantining a source can trigger descendant review or removal.Behavior-aware checks should complement static metadata review because upstream filtering can alter downstream routing behavior.
- Limitations: All experiments used a controlled dataset derived from public AppWorld API patterns, excluding real user data, accounts, live services, and external-service actions.Triggering was measured using local proxy detectors and a small code-generation check.
- Limitations: The claims use one model and a policy-violation proxy, making reported success routing-level rather than deployment-level while leaving the defense study exploratory.The authors identify multiple models, independently collected experience, and live execution as natural next steps.
6. Related Work
Prior work addresses memory poisoning, experience reuse, provenance laundering, and lifecycle defenses, whereas SkillJack targets the experience-to-skill transition as a distinct security surface. Its key distinction is that poisoned records can become persistent behavioral artifacts that outlast removal of their source records.
- Memory poisoning and indirect prompt injection: Memory-poisoning research studies persistent attacks on agent memories or retrieval stores, alongside benchmark settings for prompt-injection risks in tool-using agents.These lines include AgentDojo, InjecAgent, AgentPoison, and MINJA.
- Experience reuse and skill extraction: Experience-to-skill systems increasingly automate reusable knowledge from task histories, including insights, workflow memories, organized memories, hierarchical skills, and structured skill contracts.Examples include ExpeL, AWM, A-MEM, SkillX, Anything2Skill, and AutoSkill.
- Provenance, laundering, and lifecycle defenses: Prior provenance work shows that transformations can launder untrusted memory content, motivating origin-bound authority and provenance links from derived skills to source records.TMA-NM formalizes laundering through summarization and related transformations; the paper applies an analogous concern at the experience-to-skill boundary.
- Positioning: Unlike memory-poisoning attacks triggered by retrieval or skill-learning work focused on capability improvement, SkillJack targets compilation of poisoned experiences into persistent behavioral artifacts.The lifecycle distinction is that clearing or quarantining the original record may no longer remove the derived artifact.
7. Conclusion
SkillJack shows that a self-evolving agent’s experience-to-skill pipeline can transform poisoned experiences into persistent, reusable skills. The pipeline obscures detection while preserving harmful behavior and enabling persistence after source deletion.
- SkillJack is the first attack showing that a self-evolving agent’s experience-to-skill pipeline turns poisoned experience into a persistent, reusable skill.
- 98.5% of raw trajectories were flagged by an LLM judge, compared with only 11.4% of the skills distilled from them.This demonstrates attack laundering during skill extraction.
- 56.2% of attack tasks on SkillX and 89.2% on Anything2Skill triggered the intended violation through the distilled skills.
- 80% of skill-mediated triggers survived deletion of the records that produced them.The attack therefore persists after removal of its original poisoned records.
A. Experiment Configuration Details · A.1. Payload Templates
The experiment uses legitimate-looking task descriptions to frame policy-relevant actions across risk categories. It reports framing vocabulary and derived-skill examples while omitting executable payload details.
- A.1. Payload Templates: Legitimate-looking task descriptions frame policy-relevant actions in trajectories for each policy-risk category.
- A.1. Payload Templates: Executable payload details are omitted from the report.
- A.1. Payload Templates: The omission documents the threat without providing an operational attack recipe.
- A.1. Payload Templates: Table 4 summarizes the framing vocabulary used for each policy-risk category.
- A.1. Payload Templates: Table 4 provides an example derived skill for each policy-risk category.
- A.1. Payload Templates: The framing vocabulary and derived skills describe experimental stimuli rather than recommended operational procedures.
A.2. Dataset Statistics · A.3. LLM and Infrastructure
The experimental dataset contains 150 trajectories that produce 505 extracted skills, while the routing library contains 356 skills split between poisoned and clean entries. Experiments use an API-accessed DeepSeek-v4-flash setup requiring about 3,047 LLM calls and 5 hours, without GPUs or embedding models.
- A.2. Dataset Statistics: 150 trajectories comprise the full experimental dataset.
- A.2. Dataset Statistics: 505 extracted skills are yielded by the full experimental dataset.
- A.2. Dataset Statistics: 505 extracted skills form the union of the two SkillX poisoned extraction sets.
- A.2. Dataset Statistics: 356-skill routing library contains 305 poisoned and 51 clean skills.
- A.3. LLM and Infrastructure: DeepSeek-v4-flash is used via API for all experiments.
- A.3. LLM and Infrastructure: Approximately 3,047 LLM calls and 5 hours of runtime cover all experiments.
- A.3. LLM and Infrastructure: No GPU or embedding models are required because skill routing uses LLM-native progressive disclosure.
A.4. Runtime Monitor Signatures
The runtime monitor detects generated actions using regular-expression signatures, but A2S’s normalized malicious phrasing required expanding the SkillX-based list from 16 to 54 signatures. This enhanced set reflects post-hoc tuning to address vocabulary laundering during skill extraction.
- Signature Design: The original runtime monitor uses a fixed list of 16 regular-expression signatures over the SkillX action vocabulary.It flags a generated action when its text matches any listed pattern.
- Signature Design: 38 additional signatures expand coverage for A2S DOC-mode vocabulary, producing 54 total signatures.The additions target normalized phrasing used by A2S-extracted skills.
- Post-hoc Tuning: The enhanced signature set was expanded after observing A2S failures, so its effect reflects post-hoc tuning.The added patterns are largely normalized variants of the original signatures.
- Post-hoc Tuning: A2S extraction rephrases overtly malicious actions into neutral operational language, requiring monitors calibrated to blunt vocabulary to be retuned.Examples include patterns for monitoring services, credential retrieval, and automatic tips.
B. Detailed Per-Category Results
The cross-layer experiment includes a per-category breakdown, presented in Figure 3 of the main text.
- Figure 3 presents the per-category breakdown for the cross-layer experiment.
C. Illustrative Case Studies
The case studies show poisoned trajectories being converted into benign-looking skills that evade detection while retaining harmful actions across four policy-risk categories. Trigger rates vary substantially between SkillX and A2S, with persistent backdoors reaching 100.0% on A2S.
- Data exfiltration: 55.6% of data-exfiltration tasks trigger on SkillX extracted skills, compared with 85.0% on A2S, while the raw trajectory is flagged but the skills are not.The skill copies the entire Spotify library to an attacker-controlled address without consent.
- Privilege escalation: 75.0% of privilege-escalation tasks trigger on SkillX extracted skills, compared with 80.0% on A2S, the highest SkillX category, despite skills evading detection.The extracted skill gathers stored credentials across applications under one unified management action.
- Unauthorized transfer: 28.6% of unauthorized-transfer tasks trigger on SkillX extracted skills, compared with 93.3% on A2S, the lowest SkillX category, as framing travels differently across routers.The skill authorizes recurring transfers to an attacker-controlled account.
- Persistent backdoor: 62.5% of persistent-backdoor tasks trigger on SkillX, compared with 100.0% on A2S, as error-recovery framing conceals a covert trigger that leaks user content.The durable leak activates whenever the specific error condition recurs; A2S triggers on every task in this category.