PaperlayerView exact revision on arXiv

Source-linked AI summary

EVOMAL: Self-Poisoning in Self-Evolving Coding Agents

Xiaodong Wu, Yu Shi, Qi Li, Zhimin Zhao, Xiangman Li, Bram Adams, Ahmed E. Hassan, Jianbing Ni

arXiv:2608.25776v1cs.CRcs.AI

TL;DR

Self-evolving coding agents can poison their own persistent skill libraries when retrieved malicious skills become templates for authored copies. The paper introduces EVOMAL to amplify and measure this CREATE-path attack, finding substantial cross-model infection and persistence, while counter-prompt sharply reduces ASPR.

  • Problem

    Self-poisoning is a CREATE-path security gap in which retrieved malicious skills are re-authored into agent-owned persistent skills.

  • Method

    EVOMAL plants ordinary-looking banner-and-payload skills and evaluates how retrieval, imitation-based authoring, and cascading persistence propagate malicious copies.

  • Results

    Across six models, ASPR ranges from 20.3% to 41.8%, reaches 86.7% with task-family targeting, and remains 68% for Qwen3 after plant removal.

  • Takeaways & Limitations

    Security controls must address agent-authored skills and their re-entry into persistent libraries, not only attacker-submitted names, code, and signatures.

  • Takeaways & Limitations

    Worm risk depends jointly on banner-copying willingness and copied-skill reach on later tasks, and DS-V4's cascade can collapse despite its isolated proxy prediction.

Abstract

from arXiv · show

Self-evolving LLM coding agents write their own tools by imitating retrieved skills from shared skill libraries. We identify a vulnerability in this loop: during authoring, a retrieved malicious skill can become the template for a new skill that preserves the payload. We call this self-poisoning: the agent authors, stores, and runs the resulting malicious skill. We exploit it through EvoMal, an attack that amplifies self-poisoning by wrapping an interchangeable payload in a banner, a set of benign-looking structural elements that induces an imitating agent to reproduce the enclosed code. The attacker plants malicious skills in the library without invoking them. The agent then authors and executes new skills carrying the harmful code. Each authored copy can re-enter the library and be imitated again, forming a self-propagating worm that persists after the planted skills are removed. We define the agent self-poisoning rate (ASPR) as the fraction of tasks that add a newly authored malicious skill to the library. Across six models on 153 tool-relevant SWE-bench Verified tasks, ASPR ranges from 20.3% to 41.8%, and the poisoned libraries hold 4.9 to 9.0 times as many malicious skills as were planted. The vulnerability also appears without a banner: DeepSeek-V4-Pro reaches 11.1% ASPR with the payload alone. Tailoring the planted skill descriptions to one task family raises ASPR to 86.7%. After the planted skills are removed, Qwen3 retains a round-5 ASPR of 68% because agent-authored copies remain. These copies evade existing defenses, which focus on attacker-submitted names, code, and signatures. We propose counter-prompt, a defense that discourages banner-style copying and reduces EvoMal's ASPR to at most 6.7% with no significant task-completion loss.

1 Introduction

Self-evolving coding agents create a CREATE-path vulnerability in which retrieved malicious skills are re-authored, stored, and propagated. EVOMAL measures this self-poisoning across models and shows persistence, scaling, and mitigation results.

  • The vulnerability: Self-poisoning occurs when an agent re-authors a retrieved planted skill as a new malicious skill, creating an unmediated admission path into its trusted library.The authored copy can later be retrieved and executed without the attacker invoking the planted skill.
  • Attack design: 41.8% ASPR on DS-V4 followed banner amplification, compared with an 11.1% no-banner baseline.ASPR is the fraction of tasks that add a newly authored malicious skill to the library.
  • Feasibility and scale: 20.3% to 41.8% ASPR was observed across six models, up to 32 percentage points above a payload-free control.The poisoned libraries contained 4.9 to 9.0 times as many malicious skills as initially planted.
  • Scaling and persistence: 86.7% ASPR resulted from tailoring planted descriptions to one task family, while Qwen3 retained 68% ASPR after planted skills were removed.Three of six models remained infected after removal, demonstrating persistence across generations.
  • Defense: Counter-prompt reduced EvoMal's ASPR to ≤6.7% without a statistically significant task-completion loss, though it is soft and model-dependent.The paper therefore also proposes a structural signed-quarantine gate.

2 Background

Self-evolving agents maintain persistent executable-skill libraries and alternate between reusing retrieved skills and authoring new ones. The CREATE-path lets retrieved code become a stored template, distinguishing this vulnerability from ordinary invocation-only tools.

  • Background: Self-evolving agents retrieve similar skills, reuse or author new ones, and store newly authored skills for future tasks.This persistent library turns one-off solutions into reusable tools.
  • Positioning: EVOMAL combines attacker-planted executable payloads, agent authorship into a persistent store, and self-propagation; Table 1 identifies it as the only work with all three properties.The comparison positions EVOMAL against prior attack classes.
  • Scope: The paper excludes markdown Claude Skills loaded through progressive disclosure and MCP tools that the agent only invokes.Its definition targets executable skills authored and persisted by the self-evolving agent.
  • CREATE-path and REUSE-path: The CREATE-path authors and stores a fresh skill that reproduces a retrieved pattern, enabling later self-propagation.The REUSE-path instead invokes a retrieved skill by name and does not require self-evolution.

3 Related Work

Prior work studies skill-library poisoning, self-evolving-agent risks, and defenses centered on attacker-submitted artifacts. EVOMAL differs by targeting the CREATE-path, where the agent re-authors and persists the malicious pattern.

  • Skill-library poisoning: Prior skill-poisoning attacks remain on the REUSE-path: the attacker-submitted artifact is invoked unchanged and is not persisted as a new skill.Examples include tool-selection manipulation, fragmented payloads, embedded malicious code, and documentation-based triggers.
  • Self-evolving agents: Self-evolving-agent research also examines safety erosion, insecure tool creation, adversarial experiences, and self-propagating messages, configuration, or memory.These lines of work do not combine EVOMAL's attacker-planted executable payload, agent authorship, and self-propagation.
  • Defenses: Existing defenses inspect attacker names, retrieved text, signatures, metadata, or generated code, placing them primarily on the REUSE-path.The paper contrasts these approaches with CREATE-path poisoning, where the agent names and stores the harmful copy.
  • Distinction: EVOMAL targets the unguarded authoring-and-storing step, where an attacker’s pattern becomes an executable skill authored by the agent itself.This changes the harmful artifact from the attacker submission to the agent-authored copy.

4 Problem Formulation

The formulation models a publish-only attacker and an external defender around a self-evolving agent whose CREATE-path authors and persists skills. EVOMAL measures infection, propagation, and defense under this setting.

  • 4.1 Threat Model: The attacker has publish access but no model, runtime, prompt, task, retrieval-index, or skill-revocation access.Planted skills are inserted among benign library entries and need only be retrieved once.
  • 4.1 Threat Model: The attack achieves payload execution and persistent library infection through agent-authored CREATE-path skills rather than invoking planted tools by name.Stored copies can be retrieved and re-authored across generations after the original plant is removed.
  • 4.1 Threat Model: The attacker can choose broad or task-family-targeted reach by changing planted descriptions without additional victim knowledge.Targeted descriptions rank into the top-k for a single task family.
  • 4.1 Threat Model: The defender controls prompts, library settings, retrieval policy, similarity thresholds, and known attack patterns, but cannot retrain the model or disable tool access.These constraints reflect deployment settings where API-only operation and tool access are necessary.
  • 4.1 Threat Model: Defense prioritizes preventing authoring, secondarily blocking execution, and minimizing infection because missed infections can spread across subsequent tasks and generations.The stated objective tolerates a small benign-utility loss to reduce asymmetric infection costs.
  • 4.2 Formal Objective: The agent is modeled as A = (M,R,P,L0), with retrieval over a deployed library followed by task execution and optional skill authoring.Newly authored skills enter the library for later retrieval.
  • 4.2 Formal Objective: ASPR counts CREATE-path tasks where an authored stored skill reproduces a planted pattern; the REUSE-path is outside scope and Ux = 0 on all runs.The evaluation separates retrieval and reproduction, while callback rate separately measures copied-payload execution.

5 EVOMAL

EVOMAL plants ordinary-looking skills whose banner structure induces self-evolving agents to copy an interchangeable payload into newly authored skills. Those copies enter the library and can propagate through later retrieval and authoring.

  • Attack overview: EVOMAL operates along the CREATE-path by planting skills, shaping descriptions for retrieval, and spreading authored copies across cascade rounds.The construction requires no agent or network access and compares generic and targeted retrieval scopes while keeping code bodies fixed.
  • Planted skills: A planted skill combines a benign-looking banner with an interchangeable payload that an agent reproduces when using retrieved code as a template.The banner uses ordinary Python structure, including a copy-verbatim comment, a task-shaped decorator, and import-time registration.
  • Planted skills: Import-time registration can execute the payload when the skill loads, even if the skill’s functions are never invoked.The payload is separate from the banner and can represent exfiltration, a reverse shell, a cryptominer, or ransomware.
  • Attacker scopes: Generic descriptions provide broad retrieval coverage, whereas targeted descriptions and names make the same code fit a chosen task family and leave it dormant elsewhere.Because retrieval ranks descriptions without inspecting code bodies, descriptions are the attacker’s main control over exposure.
  • Cascade: The cascade protocol updates the library with agent-authored skills each round, allowing infection to persist independently of the original planted seed.If ASPR holds or rises after seed removal, the infection sustains itself as a worm of the agent’s own skills.

6 Evaluation

The evaluation measures self-poisoning across models, benchmarks, payloads, task families, and cascade conditions using ASPR and related CREATE-path rates. Results show broad feasibility, strong task-family scaling, model-dependent persistence, and generalization across settings.

  • Experimental setup: The study uses mini-SWE-agent with Voyager-style skill memory, top k=5 retrieval, 232 benign skills, and six models across SWE-bench Verified and Pro.The evaluation retains Python tasks whose issue statements invoke software-engineering tooling.
  • Metrics and setup: ASPR is the fraction of tasks that add a newly authored malicious skill, decomposed into planted-skill retrieval and conditional reproduction.The in-context rate measures retrieval into top-k context, while the conditional-copy rate measures reproduction among retrieved cases.
  • Feasibility (RQ1): 20.3% to 41.8% of tool-relevant SWE-bench Verified tasks produce agent-authored malicious skills across six models.These rates are 18.3 to 32.0 percentage points above the payload-free control and correspond to 31 to 64 newly authored malicious skills over 153 tasks.
  • Feasibility (RQ1): 11.1% ASPR remains for DS-V4 with the payload alone, while the full banner reaches 41.8%; nearly every copied skill is stored, executed, and reaches C2.The three banner layers reduce DS-V4 ASPR by 13.0, 19.6, and 30.7 percentage points when removed from top to bottom.
  • Scaling (RQ2): Targeted descriptions and names raise ASPR by +11.8 to +26.9 percentage points, peaking at 86.7% on Qwen3 pytest tasks.The code bodies remain identical, so the comparison isolates retrieval pull and task-family matching.
  • Persistence (RQ3): After seed removal, Qwen3 reaches 68% ASPR at round 5, while other models range from weaker increases to immediate baseline recovery or decay.Persistence depends on whether agent-authored skills remain retrievable after planted skills are withdrawn.
  • Persistence (RQ3): At replacement rate r=0.25 DS-V4 decays, at r=0.50 it reaches roughly 53%, and at r=0.75 it reaches 60.1% while still rising at round 5.The results identify replacement rate as a critical propagation parameter.

7 Ablation Studies

EVOMAL remains effective across agent scaffolds and tool languages, while deployment settings change its success mainly by altering poisoned-entry competition and retrieval exposure.

  • Task targeting brings OpenHands, Claude Code, and mini-SWE-agent to 60% ASPR on the pytest family, while counter-prompt reduces every scaffold to ≤0.7%.
  • 19.6% ASPR from one seed rises to 41.8% at eight seeds but reaches only 43.1% at sixteen, indicating saturation near the default.More seeds increase the chance of top-k retrieval, but copy behavior remains fixed after retrieval.
  • 41.8% ASPR at retrieval depth k=5 changes to 43.1% at k=10 and 30.7% at k=3, so retrieval-window size offers limited defense.Additional depth mainly adds benign neighbors once planted skills reliably enter the retrieval set.
  • 60.1% ASPR occurs at 12.5% poisoning density, compared with 41.8% at 3.4%, because fewer benign entries compete for top-k slots.Shrinking the benign pool from 232 to 64 raises poisoning density and vulnerability.

8 Why Self-Evolution Propagates

Self-evolution adds an agent-controlled admission path: authored skills can re-enter the library and propagate according to both copying and future retrievability.

  • Agent-authored skills enter the library outside deployer mediation and are subsequently retrieved and copied alongside curated entries.This creates a second admission path beyond the static library’s deployer-mediated trust boundary.
  • The cascade models each infected entry as producing infected authored skills, with expected reproduction ρ determined multiplicatively by copy, reach, and payload-preservation factors.The analysis uses a homogeneous Galton–Watson approximation with round-independent offspring factors.
  • ρ > 1 permits survival, whereas ρ < 1 drives extinction; both copy rate and retrieval reach must be sufficiently large for sustained propagation.A high conditional-copy rate alone cannot determine the cascade regime.
  • DS-V4 copies more than Gemma4, at 66.7% versus 60.0%, yet collapses while Gemma4 self-sustains because copied skills differ in later-task reach.DS-V4’s narrow matching and crowding reduce effective retrieval below isolated measurements, limiting the branching approximation.

9 Defenses (RQ4)

Existing screening misses the CREATE-path because the agent-authored skill is outside attacker-submitted artifacts; counter-prompting sharply reduces copying, while structural quarantine offers stronger containment.

  • 9.1 Existing Defenses Miss the CREATE-path: Submission-side defenses cannot inspect the agent-authored infection, so name, code, signature, and REUSE-path screening miss the CREATE-path.The defense must intervene when the agent reads retrieved content or writes a newly authored skill.
  • 9.1 Existing Defenses Miss the CREATE-path: 0 of 275 authored infections are flagged by an oracle planted-name blocklist, while Bandit’s 85% catch rate falls to 7% after a one-line rewrite.The injection classifier reaches up to 47% false positives on the library, and the safety classifier catches only a quarter.
  • 9.1 Existing Defenses Miss the CREATE-path: After admission, defenses in the Theorem 1 class provide no additional CREATE-path reduction beyond any reduction caused by filtering seed retrieval.Blocklists reduce copying only through their effect on whether planted skills are admitted or retrieved.
  • 9.2 Counter-prompt: ≤1.8% ASPR follows from the counter-prompt across models and SWE-bench Pro, with every malware class, targeted family, and round-5 endpoint at ≤2.6%.One Devstral 15-task pytest family is the sole exception, at 6.7%, with zero callbacks reported.
  • 9.2 Counter-prompt: Counter-prompt effectiveness depends on retaining refusal of banner-style boilerplate, while rewording, token renaming, and adaptive attacks remain covered at low utility cost.Omitting the refusal instruction raises ASPR toward the undefended rate.
  • 9.3 Structural Gate: Structural quarantine drives CREATE-path ASPR and self-propagation to negligible levels by making agent-authored skills unretrievable.An unforgeable curator signature and controlled admission log enforce the separation, while persistent seeds can still fire once at the external-seed rate.

10 Conclusion

EVOMAL identifies self-poisoning as a persistent, self-propagating threat in self-evolving coding agents and shifts the security boundary toward agent-authored skills.

  • Across six models, EVOMAL shows that planted code can be reproduced, stored, and executed by the agent itself, remaining potent after planted skills are removed.
  • Conventional name-, code-, and signature-based screening is insufficient because defenses must secure authoring and control re-entry of generated skills.Counter-prompting is lightweight and effective, whereas structural quarantine offers stronger containment at the cost of limiting self-evolution.

Ethical Considerations

The study isolates attack execution in a network-isolated sandbox and limits released artifacts to safe test stubs. It reports no human subjects and no terms-of-service violations.

  • Attack execution and containment: All payload executions occur in a sandbox with no real secrets or external network egress.The wrapper strips the environment, supplies fake placeholders and credentials, uses a fresh HOME, and binds exfiltration locally.
  • Affected parties and disclosure: The attack targets no specific production deployment and exploits a design-class property shared by self-evolving agents.The study minimizes harm through sandboxed experiments, safe stubs, and publication of the counter-prompt defense.
  • Dual-use considerations: The release publishes banner structures and conditional-copy mechanisms but withholds production-class payload implementations.Shipped stubs contain minimal malware-class imitations that cannot reproduce production malware behavior.
  • IRB: No human subjects are involved, so IRB review is not applicable.
  • Use of LLM compute: Inference uses locally hosted open-weight models or standard vendor APIs under published terms, with no jailbreaking or rate-limit circumvention.

Open Science

The paper provides artifacts, reproducibility details, defense alternatives, and extensive counter-prompt evaluations. The defense suppresses self-poisoning while preserving task completion and remains effective against adaptive variants.

  • Open Science: The release includes planted-skill generators, targeted attacker sets, payload variants, and artifact components needed to evaluate the paper’s contributions.Executable releases exclude production-class implementations, while restricted bodies are available to reviewers.
  • Reproducibility: The headline configuration runs in approximately 8 hours on two 80 GB GPUs or approximately 30 minutes of API time, while the full six-model matrix takes approximately 48 hours.
  • Defense alternatives: Registry-side defenses have important limitations: the regex sanitizer prevents persistence but allows 49/153 payloads to reach the endpoint, while similarity thresholding and signing do not reliably separate planted skills.The sanitizer also flags 17% of benign telemetry-shaped helpers and is brittle to renaming.
  • Robustness: Renaming every token leaves the counter-prompt effective, reducing ASPR from 34.6% without defense to 1.3% with it.This tests semantic rather than exact-token matching.
  • Counter-prompt effects: Every model reaches ≤1.3% ASPR with 0/153 callbacks under the counter-prompt.The defense also reduces benign copying, especially for models with higher initial benign-copy rates.
  • Task completion: No model shows a statistically significant task-completion drop under the counter-prompt in either the no-attack or under-attack condition.The only significant change is a +14.4 percentage-point improvement for MiniMax in the no-attack cell.

A.4 Adaptive and Multi-Round Robustness

The adaptive and multi-round analysis tests EvoMal’s banner construction, wrapper invariance, retrieval dependence, and task-family concentration. Results show that semantic adaptation does not defeat the counter-prompt, while relevance strongly shapes propagation.

  • Adaptive banners: Four adaptive banners recover 29% to 35% ASPR undefended, yet none exceeds 1.3% under the counter-prompt.
  • Adaptive banners: Renaming all identifiers to semantic equivalents yields 34.6% undefended ASPR and 1.3% defended ASPR, indicating meaning-based rather than literal matching.
  • Counter-prompt ablations: Shortened and paraphrased counter-prompts keep ASPR at 0 to 0.7%, whereas generic cautionary ablations raise ASPR to 30.1% and 31.4%.The refusal instruction against banner-style boilerplate is the critical component tested.
  • Attack construction: The planted wrapper uses three trigger points: a decorator on calls, a registration hook at import, and a banner instructing the agent to reproduce both.The wrapper remains invariant while benign helper bodies vary across planted skills.
  • Retrieval similarity: At similarity [0.40,0.45), conditional-copy rate is 36%, rising to 71% in [0.55,0.60), so loosely related retrievals can still self-poison.Planted and benign skills share the [0.40,0.65) similarity band, leaving no useful threshold cut point.
  • Task and repository concentration: CREATE-path success reaches 84% on pytest tasks, 54 to 64% on scientific-Python repositories, 16% on Django, and 0% on SymPy.The gradient tracks similarity between repository tooling and planted software-engineering helpers.
  • Description amplification: Stronger imperative claims lower regex-family ASPR by 9.4 percentage points, from 50.0 to 40.6%.Additional REQUIRED and MUST assertions can trigger suspicion rather than increase imitation.
  • Full-distribution robustness: Full-distribution ASPR is 25.8% on Verified, compared with 41.8% on the tool-relevant subset and 18.7% on excluded tasks.On Pro, off-subset ASPR is 17.1% versus 24.6% on the subset; relevance remains active across the distribution.

C.2 Targeting Across SWE-bench Datasets

Targeted vulnerability varies with benchmark vocabulary and task-family relevance rather than reflecting a fixed model property. The CREATE-path also generalizes across frameworks and programming languages, while theoretical conclusions remain assumption-bounded.

  • Targeting across datasets: Config-parsing targeting replicates on SWE-bench Pro at 46.4%, a +21.8 percentage-point increase over the Pro generic-attacker baseline.
  • Targeting across datasets: DS-V4’s ranking changes across datasets, falling from first on Verified to third on Pro with a −17.2 percentage-point drop.Measured vulnerability depends on how benchmark vocabulary matches planted helpers, so Verified results do not identify a universally most vulnerable model.
  • Generality across target languages: Across Bash, JavaScript, and Python, ASPR is 39.2%, 39.2%, and 39.9%, respectively, under otherwise fixed conditions.Retrieval is language-independent by construction, and the plain-text banner transfers across syntax.
  • Step-budget effects: At five, ten, and fifteen steps, ASPR is 21.6%, 23.5%, and 41.8%, showing that insufficient budgets can truncate authoring.The attack saturates at the locked 15-step value; ten steps cuts off about a third of CREATE-paths mid-authoring.
  • Model limitations: The branching-process approximation is limited when empirical trajectories systematically deviate from E[Z_t] = Z_0 ˆρ^t.Such discrepancies indicate failures of assumptions about copy rates, retrieval reach, trigger probability, or single-parent attribution.
  • Theorem scope: Theorem 3 establishes structural extinction of agent-authored propagation but does not prevent initial compromise or bound harm from executing the initial seed.

D.5 Per-Model Fit of ˆρproxy

The proxy reproduction model does not fit the observed cascade uniformly: descendant retrieval, rather than copying propensity alone, separates collapse from persistence. For models that retrieve descendants, descendant copy rates are preserved or higher than seed rates, so copying does not explain decline.

  • Model fit: The proxy reproduction number overestimates the observed absolute scale by roughly 10× and misorders DS-V4 and Devstral.The comparison is between ˆρproxy = cqφ and observed per-round growth ˆρobs.
  • Descendant retrieval: DS-V4 and MiniMax re-retrieve authored infected descendants 0/765 times per model across rounds 1–5, whereas the other four models retrieve them 287–741 times.This descendant-retrieval asymmetry appears in the attacker-removed cascade.
  • Descendant retrieval: A descendant that is never retrieved contributes zero expected offspring regardless of copy rate, making in-cascade descendant retrieval the operative variable for DS-V4 and MiniMax.The expected offspring count is E[offspring] = q·c·φ.
  • Descendant copying: For the four models with nonzero descendant retrieval, descendant copy rates are preserved or elevated relative to seed copy rates.Devstral is symmetric at cdesc/cseed = 1.01, while Qwen3, Gemma4, and GPT-OSS copy descendants somewhat more often.
  • Descendant copying: Three of four descendant-to-seed copy-rate ratios exceed the pre-registered [0.7,1.3] symmetry band, and every deviation is toward higher descendant copying.These deviations strengthen branching and therefore cannot explain cascade decline.
  • Descendant copying: The four models’ descendant-copy-rate ranking matches their seed-copy-rate ranking, with Spearman correlation 0.80.This supports consistency between seed and descendant copying where descendants remain retrievable.
Loading 2608.25776v1…