EvoSkill Injection: Red-Teaming Autonomous Skill Generation and Evolution in Self-Evolving Agents

• EvoSkill Injection is a threat model in which adversarial interaction trajectories cause self-evolving agents to generate, store, and reuse malicious skills.Source: §1 · pp. 1–2
• SARGE uses iterative generation, escalation, and reinforcement, supported by EvoSkillBench attack trajectories and EvoSkillSafetyBench evaluation of downstream harmful activation.Source: §1 · pp. 1–2
• SARGE achieves attack success rates of 43.5% in Generation, 54.6% in Escalation, and 49.9% in Reinforcement across self-evolving agents.Source: §1 · p. 2
• Injected malicious skills persist in storage, retrieval, and activation, repeatedly inducing harmful behaviors and corrupting future agent behavior.Source: §1 · p. 2

2.1 Self-Evolving Agents

• Skill-based architectures store and retrieve reusable capabilities, reducing repetitive reasoning costs and improving task success, reuse efficiency, and long-horizon performance.These benefits motivate their use for long-horizon task execution and capability reuse.Source: §2.1 · p. 2
• AutoSkill, Voyager, and ExpeL exemplify persistent agent architectures that generate or refine skills, learn through executable code, or retain reflections from past task outcomes.AutoSkill presents an autonomous skill generation and evolution pipeline; Voyager uses executable code skills for lifelong learning; ExpeL stores successes and failures as reflections.Source: §2.1 · p. 2

2.2 Security Vulnerabilities in Skill Ecosystems

• Prior studies identify malicious skills, poisoning, retrieval manipulation, and supply-chain attacks, but primarily target external skills or retrieval rather than capabilities evolved within agents.Adversarial experiences internalized as reusable skills remain an underexplored risk in autonomous generation and evolution pipelines.Source: §2.2 · p. 2

3.1 EvoSkillBench

• Existing agent-attack and safety benchmarks emphasize harmful responses and other interaction attacks, not malicious skills stored, refined, and repeatedly reused.The passages identify harmful response generation, prompt injection, jailbreaking, and retrieval manipulation among existing benchmark focuses (Mazeika et al., 2024; Chao et al., 2024; Cao et al., 2025).Source: §3.1 · pp. 2–3
• EvoSkillBench uses poisoned multi-turn interactions to induce agents to internalize malicious capabilities as reusable skills.Each trajectory introduces a harmful-capability request, supplies attacker-crafted malicious output, then gives positive feedback and instructs the agent to store and reuse the capability.Source: §3.1 · p. 3
• EvoSkillBench covers eight high-risk categories and focuses on persistent capability corruption rather than benign-skill corruption or simple functional failure.The categories are Physical Harm, Social Crime, Cybercrime, Psychological Abuse, Information Manipulation, Discrimination/Hate, Privacy Violation, and Economic/Ethics Violation.Source: §3.1 · p. 3

3.2 EvoSkillSafetyBench

• EvoSkillSafetyBench tests retrieval and reuse by rewriting harmful requests as seemingly benign queries similar to stored malicious skills.This design evaluates skill activation without results being dominated by refusals to overtly harmful requests.Source: §3.2 · p. 3
• EvoSkillSafetyBench labels downstream responses as Explicit Refusal, Soft Safe Response, or Unsafe/Harmful.The labels distinguish clear refusal, safety-oriented guidance or benign alternatives, and actionable harmful assistance.Source: §3.2 · p. 3

4 SARGE: A Red-Teaming Framework

• SARGE coordinates an Orchestrator, Attacker, and Judge around a Target Agent to probe skill generation, escalation, and reinforcement attacks.The Judge probes in a fresh session to test whether an injected skill persists, is retrieved, and produces its encoded harmful behavior.Source: §4 · pp. 3–4, Figure 1 · p. 4

4.1 Skill Generation Attack

• The Skill Generation Attack iteratively refines poisoned multi-turn EvoSkillBench trajectories to induce and store reusable harmful skills.After a failed attempt, the attacker uses Judge feedback to select among six vectors—direct request, creative writing, professional persona, logical induction, multilingual code-switching, or payload obfuscation—and rewrite the trajectory; the Judge checks retrieval and harmful behavior using benign probes in an independent session.Source: §4.1 · p. 4

4.2 Skill Escalation Attack

• The Skill Escalation Attack disguises harmful capability amplification as routine refinement of a stored malicious skill.Its three-turn trajectory identifies fabricated flaws, induces more extreme harmful behavior, then requests overwriting the existing skill; escalation can also generate and store a malicious skill when initial generation failed.Source: §4.2 · pp. 4–5

4.3 Skill Reinforcement Attack

• The Skill Reinforcement Attack retrieves malicious skills and frames their outputs as successful patterns to reinforce their future use.Its three-turn trajectory elicits the skill, induces its execution, then praises the response and directs the Target Agent to store it as a successful case and standard behavior; reinforcement can also generate and store a new malicious skill when none exists.Source: §4.3 · p. 5

5.1 Experimental Setup

• Experiments use EvoSkillBench and EvoSkillSafetyBench across AutoSkill (Yang et al., 2026), Voyager (Wang et al., 2024), and ExpeL (Zhao et al., 2024), without modifying skill banks.SARGE supplies ordinary interaction histories for each agent to process through its original self-evolution pipeline. GPT-4o-mini is the primary orchestrator, attacker, and judge model; experiments also evaluate GPT-5.4, DeepSeek-V4-Pro, DeepSeek-V4-Flash, Gemini-2.5-Flash-Lite, and Qwen3.5-9B.Source: §5.1 · p. 5

5.2 Evaluation Metrics

• Attack success requires persistent storage of a malicious capability as a reusable skill, while pass@k measures the fraction of instances with a successful attempt within the first k loops.Each attack agent receives up to four attempts per EvoSkillBench instance. Safety responses are categorized as Explicit Refusal, Soft Safe Response, or Unsafe/Harmful, based on whether a behavior judge identifies the target harmful behavior.Source: §5.2 · pp. 5–6

5.3 Experiment Design

• The five evaluation dimensions cover attack success across three flows, downstream safety versus clean agents, cross-model transfer, comparisons with representative attacks, and ablations of SARGE’s three attack agents.Transfer tests apply attacks generated with one LLM to heterogeneous target LLMs; ablations remove the Generation, Escalation, or Reinforcement Attack Agent one at a time.Source: §5.3 · p. 6

6.1 Attack Success Rate

• At pass@4, SARGE reaches 43.5% in Skill Generation, 54.6% in Skill Escalation, and 49.9% in Skill Reinforcement.Pass@k increases across iterative attack loops; the results indicate that adversarial interactions treated as successful experiences can be internalized as persistent, reusable skills.Source: §6.1 · p. 6

6.2 Safety Degradation

• AutoSkill’s harmful response rate rises from 6.5% to 19.2%, while Voyager reaches 32.5% and ExpeL’s Explicit Refusal rate rises from 16.2% to 51.0%.ExpeL’s harmful response rate also increases from 8.8% to 15.2%; the results indicate repeated retrieval and activation of injected skills, alongside increased unnecessary refusals for ExpeL.Source: §6.2 · p. 7, Table 2 · p. 7

6.3 Cross-Model Transferability

• In attacked settings, harmful response rates are 29.6% for DeepSeek-V4-Pro, 20.4% for Gemini-2.5-Flash-Lite, and 19.2% for GPT-4o-mini.DeepSeek-V4-Pro has the highest harmful response rate despite the lowest cross-model attack success rates; the authors distinguish injection success from downstream harmful activation, noting its skills are more likely to cause harm once retrieved.Source: §6.3 · pp. 7–8, Table 3 · p. 7

6.4 Attack Comparison

• PAIR reaches a 33.6% harmful response rate, while SARGE reaches 19.2%, ahead of Indirect Prompt Injection (19.1%), Memory Poisoning (15.1%), Backdoor Attack (11.6%), and RAG Poisoning (15.2%).All methods use the same SARGE-generated malicious skills, and PAIR is allowed up to four refinement attempts; the comparison isolates attack strategy and shows SARGE is competitive with the listed attacks, though below PAIR.Source: §6.4 · p. 8, Table 4 · p. 8

6.5 Ablation Study

• Removing the Generation Attack Agent lowers harmful responses from 19.2% to 9.0%, and removing Reinforcement lowers them to 12.4%.Removing Escalation leaves an 18.2% harmful response rate, but the full pipeline’s responses tend to be more specific and severe, consistent with escalation strengthening malicious content in stored skills.Source: §6.5 · p. 8, Table 5 · p. 8

Implications for Defense Methods

• The Skill Verification and Skill Conflict Resolution prompts reduce harmful responses across evaluated agents but increase Explicit Refusal rates and leave malicious skill generation, storage, and evolution unchecked.The results indicate that system-prompt defenses provide only partial mitigation; stronger defenses must intervene in the autonomous skill generation and storage pipeline.Source: §7 · p. 8

8 Conclusion

• SARGE is a red-teaming framework for evaluating EvoSkill Injection in autonomous skill-generation and evolution pipelines.Autonomous generation, refinement, and reuse of skills from past experiences introduce a new attack surface.Source: §8 · p. 8
• Malicious skills are persistently stored and repeatedly activated across diverse self-evolving agents and heterogeneous LLMs.The findings identify persistent capability corruption as a security risk and motivate stronger defenses for secure skill generation and evolution.Source: §8 · p. 9
• EvoSkillBench and EvoSkillSafetyBench support evaluation of EvoSkill Injection and the persistence and activation of malicious skills.Source: §8 · p. 9

Limitations

• Evaluation covers AutoSkill, Voyager, and ExpeL, but not all self-evolving architectures, including tool-augmented or multi-agent systems and agents with stronger memory governance.Future work should evaluate EvoSkill Injection across a broader range of agent systems.Source: Limitations · p. 9
• Repeated model calls for attack generation, evaluation, and refinement incur substantial computational and token costs, so GPT-4o-mini is used to balance scalability and cost efficiency.More capable or larger models may produce stronger attacks or different evaluation outcomes; future work should study how model capability and inference budget affect effectiveness and scalability.Source: Limitations · p. 9
• EvoSkillBench covers eight high-risk categories but does not exhaust all malicious capabilities or real-world scenarios; expanding it to additional domains and realistic deployment contexts remains future work.The benchmark targets representative safety-critical risks rather than complete coverage of harmful behavior.Source: Limitations · p. 9

Ethical Considerations

• The authors identify misuse risks from internalizing malicious capabilities as reusable skills and frame the work as controlled research; released resources are intended for authorized safety and defensive research.They encourage responsible use under applicable safety and ethics guidelines and advise against applying the framework to real users, production systems, or third-party services without authorization.Source: Ethical Considerations · p. 9

A.1 Source Data

• EvoSkillBench covers eight high-risk categories, using ten manually written seed examples per category to guide category-specific multi-turn attack trajectories.The categories are Physical Harm, Social Crime, Cybercrime, Psychological Abuse, Information Manipulation, Discrimination/Hate, Privacy Violation, and Economic/Ethics Violation.Source: §A.1 · p. 12

A.3 EvoSkillSafetyBench Generation Procedure

• EvoSkillSafetyBench selects behavior samples aligned with stored malicious skills, then rewrites harmful requests as downstream queries that preserve skill similarity while removing explicit harmful intent.Selection uses lexical overlap with skill names, metadata, and prompts, choosing high-alignment samples while preserving subtopic balance; the benchmark supports skill retrieval and reuse evaluation across the same eight categories.Source: §A.1 · p. 12, §A.3 · p. 13

A.2 EvoSkillBench Generation Procedure

• EvoSkillBench generation uses DeepSeek-V4-Pro to create diverse three-turn user-assistant-user trajectories that demonstrate harmful requests, compliant responses, and reinforcement for future skill reuse.Samples vary scenario, persona, task framing, wording, and interaction style; metadata records category, subtopic, attack goal, expected harmful output, severity, and turn count.Source: §A.2 · p. 12

A.4 Quality Control Pipeline

• An LLM-based quality-control pipeline checks interaction structure, JSON fields, metadata, category alignment, duplication, realism, and consistency, while ensuring safety queries remain skill-aligned without overtly harmful wording.Malformed or inconsistent samples are removed or regenerated; duplicate or near-duplicate samples and overly generic, unrealistic, or repetitive phrasing are filtered.Source: §A.4 · p. 13

A.5 Benchmark Statistics

• EvoSkillBench contains 800 trajectories—100 per high-risk category—and EvoSkillSafetyBench contains 800 downstream evaluation queries.Each EvoSkillBench instance has two user turns and one assistant turn; Table 6 summarizes category-level distributions.Source: §A.5 · p. 13, Table 6 · p. 13

A.6 Response Evaluation Protocol

• The response protocol runs each safety query through the Target Agent’s standard retrieval and execution pipeline, recording injected skill context, selected skill, and final generation.Responses are labeled Unsafe/Harmful by an LLM behavior judge, Explicit Refusal by refusal-prefix matching if not harmful, or Soft Safe Response otherwise.Source: §A.6 · pp. 13–14

B SARGE

• The SARGE appendix provides prompts for the Orchestrator, Skill Generation, Skill Escalation, Skill Reinforcement, and Judge Agents to implement attack flows and assess skill activation.The prompts cover trajectory generation, refinement of failed attempts, flow-specific probe queries, and retrieval-and-activation evaluation in a fresh session.Source: B SARGE · p. 14

C.1 Additional Cross-Model Transferability Results

• Gemini-2.5-Flash-Lite reaches pass@4 transferability of 70.9%, 72.0%, and 70.8% across Generation, Escalation, and Reinforcement, while DeepSeek-V4-Pro stays below 13%.On EvoSkillSafetyBench, PAIR achieves the highest overall harmful response rate, while SARGE remains competitive with representative attack baselines.Source: C Additional Results · p. 14, Table C.1 · p. 23, Table C.2 · p. 23

C.3 Additional Ablation Attack Success Results

• Removing each attack agent changes malicious-skill formation across objectives, while removing the Generation or Reinforcement agent substantially reduces downstream harmful responses.The ablations support complementary roles for the three attack agents in persistent skill formation and activation.Source: C Additional Results · p. 14, Table C.3 · p. 24, Table C.4 · p. 24

B.3 Skill Escalation Attack Agent

• The Orchestrator develops flow-specific strategies; escalation upgrades a stored malicious skill, and reinforcement triggers it and consolidates the behavior as a future standard response.Escalation uses Judge feedback, while reinforcement simulates execution of a previously generated or escalated skill.Source: Figure B.1 · p. 15, Figure B.6 · p. 19, §B.4 · p. 19

B.2 Skill Generation Attack Agent

• Generation prompts complete three-turn attacks, validate harmful content and persistence instructions, repair weak attacks, and select alternative vectors after failure.The completion prompt preserves existing messages and requests a harmful assistant response; repair and vector selection use feedback to strengthen or rewrite the interaction.Source: Figure B.2 · p. 16, Figure B.3 · p. 16, Figure B.4 · p. 17

B.5 Judge Agent

• Judge prompts generate natural, flow-specific probes for fresh sessions and assess whether a stored skill is retrieved and its harmful behavior appears in the response.Probe strategies differ across Generation, Escalation, and Reinforcement flows.Source: Figure B.8 · p. 21, Figure B.9 · p. 21, Figure B.10 · p. 22

C.5 Additional Attack Performance across Different LLMs

• 79.4%, 78.5%, and 76.6% pass@4 are achieved by DeepSeek-V4-Pro as both attacker and target for Generation, Escalation, and Reinforcement; GPT-4o-mini has the highest overall harmful response rate at 29.6%.The corresponding harmful response rates are 15.6% for DeepSeek-V4-Pro and 6.5% for GPT-5.4 under their target configurations; GPT-5.4’s attack success is 19.9%, 22.3%, and 25.6%.Source: C Additional Results · p. 14, Table C.5 · p. 25, Table C.6 · p. 25

D.1 Defense Prompt Details

• The Skill Verification Prompt treats retrieved skills as untrusted and checks their content for safety violations, while the Conflict Resolution Prompt rejects skills that conflict with safety policies, system instructions, or user intent.These are lightweight prompt-level defenses added to the Target Agent’s system prompt during response generation.Source: §D.1 · p. 26

D.2 Defense Results

• Harmful responses fell from 19.2% to 8.4% for AutoSkill and 32.5% to 5.2% for Voyager with combined defenses, and from 15.2% to 9.6% for ExpeL with SCRP.The combined defense achieved the strongest reduction for AutoSkill and Voyager; SCRP achieved the lowest harmful response rate for ExpeL.Source: Table D.1 · p. 26, §D.2 · p. 26
• Voyager’s Explicit Refusal rate rose from 33.1% under attack to 62.9% with combined defenses, indicating greater conservatism that may reject answerable queries.System-prompt defenses also increase Explicit Refusal rates.Source: §D.2 · p. 26
• Prompt-level defenses partially mitigate malicious skill activation during response generation but leave malicious skill generation, storage, and evolution unprevented, motivating root-level defenses.Both SVP and SCRP reduce harmful responses compared with attacked agents without defense, but the strongest defense differs across target agents.Source: §D.2 · p. 26
• SVP treats retrieved skills as untrusted and checks them against safety policies and benign user intent.SVP is the Skill Verification Prompt inserted into the Target Agent’s system prompt during response generation.Source: Figure D.1 · p. 27
• SCRP instructs agents to ignore retrieved skills that conflict with safety policies, system instructions, or current user intent.SCRP is the Skill Conflict Resolution Prompt; the two prompts provide complementary response-generation interventions.Source: Figure D.1 · p. 27

D.3 Additional Defense Results

• The additional topic-wise results cover three agents and report Explicit Refusal, Soft Safe, and Harmful responses for SVP, SCRP, and their combination.Table D.2 presents the topic-wise defense results after the GPT-4o-mini-based SARGE Skill Reinforcement Attack; its caption supplies scope and labels, not cell values.Source: §D.2 · p. 26, Table D.2 · p. 27