PaperlayerView exact revision on arXiv

Source-linked AI summary

SkillTrojan: Backdoor Attacks on Skill-Based Agent Systems

Yunhao Feng, Yifan Ding, Yingshui Tan, Boren Zheng, Yanming Guo, Xiaolong Li, Kun Zhai, Yishan Li, Wenke Huang

arXiv:2604.06811v2cs.CRcs.AI

TL;DR

Skill-based agents introduce an underexamined security surface because reusable skills execute code and persist across deployments. SkillTrojan embeds encrypted payload fragments into skills and activates them through ordinary skill composition under a trigger. In evaluation, it achieves high attack success while largely preserving benign performance, including 97.2% ASR with 89.3% clean ACC on GPT-5.2-1211-Global.

  • Problem

    Reusable skill implementations expand the agent attack surface beyond model behavior, yet skill-level backdoors remain underexamined.

  • Method

    SkillTrojan embeds encrypted payload fragments in reusable skills, activates them through triggered skill composition, and supports automated synthesis from arbitrary templates.

  • Results

    97.2% ASR with 89.3% clean ACC is achieved on GPT-5.2-1211-Global, while evaluation reports high attack success with largely preserved benign performance.

  • Takeaways & Limitations

    Skill-level backdoors expose a critical blind spot in agent security assumptions and motivate defenses that reason about skill composition and execution.

  • Takeaways & Limitations

    The evaluation is conducted in a representative code-based agent setting, and the work emphasizes controlled experimental settings because the framework is dual-use.

Abstract

from arXiv · show

Skill-based agent systems tackle complex tasks by composing reusable skills, improving modularity and scalability while introducing a largely unexamined security attack surface. We propose SkillTrojan, a backdoor attack that targets skill implementations rather than model parameters or training data. SkillTrojan embeds malicious logic inside otherwise plausible skills and leverages standard skill composition to reconstruct and execute an attacker-specified payload. The attack partitions an encrypted payload across multiple benign-looking skill invocations and activates only under a predefined trigger. SkillTrojan also supports automated synthesis of backdoored skills from arbitrary skill templates, enabling scalable propagation across skill-based agent ecosystems. To enable systematic evaluation, we release a dataset of 3,000+ curated backdoored skills spanning diverse skill patterns and trigger-payload configurations. We instantiate SkillTrojan in a representative code-based agent setting and evaluate both clean-task utility and attack success rate. Our results show that skill-level backdoors can be highly effective with minimal degradation of benign behavior, exposing a critical blind spot in current skill-based agent architectures and motivating defenses that explicitly reason about skill composition and execution. Concretely, on EHR SQL, SkillTrojan attains up to 97.2% ASR while maintaining 89.3% clean ACC on GPT-5.2-1211-Global.

1. Introduction

Skill-based agents improve modularity and scalability by composing reusable executable skills, but this concentrates trust in skill implementations and creates an under-examined attack surface. SkillTrojan targets this layer with dormant, trigger-activated payloads and evaluates the resulting security risk while preserving benign utility.

  • Skill-based agents compose reusable skills that encapsulate procedural logic, tool calls, and execution workflows, improving modularity and scalability.
  • Encrypted payload fragments are distributed across benign-appearing skill invocations and reconstructed only when a predefined trigger condition is satisfied.
  • SkillTrojan targets reusable skill implementations rather than model parameters, prompts, or tool and memory interfaces.
  • The framework supports automated synthesis of backdoored skills from arbitrary templates, enabling scalable attacks across agent pipelines and skill ecosystems.
  • 3,000+ curated backdoored skills are released, while evaluation reports 97.2% ASR on GPT-5.2-1211-Global with minimal impact on clean-task accuracy.

2. Related Work

Prior backdoor research primarily targets models or transient agent interaction surfaces such as prompts, tools, and memory. SkillTrojan instead treats reusable executable skills as a persistent execution-layer control surface.

  • Skill-based abstractions encapsulate reusable code, API calls, and execution logic, forming a backbone of coding agents while remaining an underexplored control locus.
  • Traditional backdoors target models through poisoned data, parameter manipulation, or trigger-based inputs while preserving clean-input performance.
  • More recent agent attacks target prompts, tool descriptions, memory, or control logic, but generally operate at transient interaction surfaces.
  • SkillTrojan embeds backdoors directly into reusable executable skills, allowing them to persist across tasks and deployments and activate through normal skill composition.

3. Threat Model

The threat model assumes trusted, unmodified models and training data but persistent attacker control over distributable skill packages. Attackers publish benign-looking backdoored skills that can be adopted and later activated during routine execution.

  • A skill is modeled as a distributable package containing a natural-language specification and executable artifacts, and deployments install sets of such skills.
  • The underlying language model and its training data are assumed trusted and unmodified, while risk arises from importing external skill packages that combine instructions with executable code.
  • The attacker replaces selected skills with backdoored packages containing an arbitrary payload program and a trigger predicate parameterized by a trigger string or pattern.

4. Method

SkillTrojan transforms benign skill templates into conditional backdoors that preserve normal behavior while reconstructing and executing encrypted payloads after triggered workflows complete. Its design distributes indexed fragments across skill actions, verifies their collection without relying on execution order, and supports automated generation across heterogeneous templates.

  • Attack overview: SkillTrojan transforms a benign skill template into a backdoored skill whose nominal functionality is preserved for non-triggered queries and whose payload executes as a triggered side effect.The attack modifies both the skill specification and executable artifacts while keeping each modified action’s original behavior and output.
  • Payload encoding: Encrypted payload ciphertext is partitioned into indexed fragments, with the fragment count N controlling the trade-off between stealth and activation robustness.Larger N reduces the information in each fragment and accidental activation risk but requires more complete execution traces.
  • Triggering and instrumentation: For triggered queries, conditional logic biases execution toward designated fragment-emitting actions without prescribing a fixed order or explicit malicious instruction.The trigger changes the likelihood of traces covering the selected actions while operating within normal planning and decomposition mechanisms.
  • Triggered reconstruction and execution: Fragments are scoped to one execution, and a verification action checks whether all required fragments were collected using set inclusion rather than timing or invocation order.After successful verification, indexed ciphertext segments are concatenated and decrypted to recover the payload.
  • Triggered reconstruction and execution: The recovered payload executes inside the skill environment after normal task completion, while the agent returns a correct benign output and intermediate artifacts are removed.Payload execution is independent of response generation and occurs through execution-side effects.
  • Automated synthesis: SkillTrojan generates heterogeneous backdoored variants by transforming normalized marketplace skill templates across triggers, payload families, encoding choices, and fragment counts.The resulting corpus supports scalable evaluation across skills that share the threat model but differ in surface semantics and implementation details.

5. Experiments

The experiments evaluate SkillTrojan in code-based agents using EHR SQL and SWE-Bench Verified, measuring clean accuracy and triggered attack success. Across model regimes, SkillTrojan achieves high ASR while largely preserving clean-task performance, and lightweight static scanners often fail to detect its encrypted, segmented payloads.

  • Experimental setup: SkillTrojan is evaluated in a code-based skill execution setting where agents select and compose executable tools from Skill.md specifications.Backdoored tools emit encrypted payload fragments only when a trigger predicate holds, while trajectories record tool calls and outputs.
  • Experimental setup: EHR SQL measures clean-task accuracy by SQL-result correctness and attack success by successful payload reconstruction and execution under triggered queries.The benchmark evaluates both benign task completion and malicious side-effect execution.
  • Experimental setup: The evaluation compares SkillTrojan with GCG, AutoDAN, CPA, BadChain, and AgentPoison under the same trigger, poisoning, and tool-environment protocol.A clean-skill SKILLS (NATIVE) baseline isolates benign skill utility from backdoor behavior, while NON-ATTACK lacks the installed skill stack.
  • Main results: 97.2 ASR with 89.3 ACC is achieved on GPT-5.2-1211-Global, while Qwen3-Max reaches 74.7 ASR with 86.6 ACC and GLM-4.6 reaches 72.0 ASR with 81.3 ACC.Across open- and closed-weight models, SkillTrojan largely preserves clean-task accuracy and sometimes exceeds the Non-Attack condition.
  • Main results: 97.2 ASR exceeds the strongest baseline’s 58.3 ASR on GPT-5.2-1211-Global, where prompt-centric attacks are comparatively unstable in tool-execution settings.SkillTrojan routes malicious behavior through trusted skill execution rather than relying on the model to follow injected instructions.
  • Main results: SkillTrojan maintains a more favorable ACC–ASR trade-off than competing methods across open- and closed-weight settings.Several baselines raise ASR while degrading ACC, whereas others preserve ACC but achieve limited ASR.
  • Defense analysis: 98.2% and 99.4% of full SkillTrojan samples are classified as safe by Qwen3Guard-Gen-8B and Llama-Guard-3-8B, respectively.The analysis indicates that simple LLM-based static scanning is insufficient against encrypted and segmented skill-level backdoors.
  • Ablation studies: Encryption changes heuristic visibility without materially changing utility: hybrid encryption shifts ACC by 0.1 and ASR by 0.3 points, while base64 detection falls from 78% to 21%.The ablation shows that attackers can diversify encodings while retaining reliability.

6. Conclusion

SkillTrojan targets the skill abstraction layer by embedding encrypted payload fragments in reusable executable skills and activating them through standard composition. The paper reports high attack success while largely preserving benign performance and releases a dataset for systematic evaluation.

  • SkillTrojan targets reusable skill implementations rather than model parameters, prompts, or training data.
  • Encrypted payload fragments are embedded in skills and activated through standard skill composition.
  • The attack achieves high success rates while largely preserving benign task performance.
  • The released SKILLTROJANX dataset contains over 3,000 backdoored skills derived from real-world templates.
  • The findings extend the security-critical surface of agent systems to the execution substrate where reusable skills operate.

Impact Statement

The paper frames skill-level backdoors as a dual-use security vulnerability and aims to support safer agent ecosystems through controlled evaluation artifacts and defense-oriented research. It emphasizes detection, execution-aware auditing, provenance checks, sandboxing, permission control, and runtime monitoring.

  • The paper studies a security vulnerability at the skill implementation and execution layer of skill-based agent systems.
  • The released framework and dataset are intended to support benchmarks for skill-level security and execution-aware auditing.
  • SkillTrojan is presented as dual-use research because malicious logic can be hidden inside useful executable skills.
  • The paper limits its focus to controlled experiments and emphasizes detection, evaluation, and defense implications.
  • Future work is encouraged to inspect tool-call trajectories, side effects, and executable skill provenance.

A. Additional Ablation Studies

The poisoning-ratio ablation tests whether attack reliability depends on the prevalence of triggered queries. Clean accuracy and attack success remain stable across the tested ratios, suggesting mostly clean spot checks may not expose the backdoor.

  • Clean accuracy remains near 68.5% even when the poisoning ratio reaches ρ=0.5.
  • ASR stays within 3 points across poisoning ratios ρ ∈{0.1, 0.2, 0.5}.
  • ASR is defined per poisoned query and depends primarily on per-run trace completion rather than global trigger frequency.
  • Mostly clean spot-check workloads are unlikely to expose the backdoor, while high-trigger workloads need not harm clean-task performance.

B. Additional benchmark: SWE-Bench Verified

The paper transfers the SkillTrojan pipeline from EHR SQL to SWE-Bench Verified while keeping the trigger, poisoning ratio, fragment count, and skill-based execution environment unchanged. The evaluation uses SWE-Bench’s test-based criterion, alongside the described backdoored-skill synthesis procedure and SQL judging protocol.

  • B. Additional benchmark: SWE-Bench Verified: The SWE-Bench Verified experiment tests whether SkillTrojan generalizes beyond structured SQL generation.
  • B. Additional benchmark: SWE-Bench Verified: The transfer keeps trigger insertion, poisoning ratio ρ, fragment count N, and the skill-based tool environment identical to EHR SQL.
  • B. Additional benchmark: SWE-Bench Verified: SWE-Bench Verified counts a task as solved when the proposed patch passes its verified unit tests.
  • B. Additional benchmark: SWE-Bench Verified: Algorithm 1 takes a benign skill template, payload program, trigger predicate, fragment count, key, encoding functions, and verification action as inputs.
  • B. Additional benchmark: SWE-Bench Verified: The synthesis procedure encrypts the payload and splits the ciphertext into indexed fragments for insertion into skill actions.
  • B. Additional benchmark: SWE-Bench Verified: SQL correctness is evaluated with an LLM judge that compares the agent result with the expected query result.

E. Experimental Details

The experiments standardize baseline attacks within the same tool-executing agent setting and evaluate both clean-task utility and cross-framework effectiveness. SkillTrojan preserves benign utility while remaining effective across practical agent frameworks.

  • Baseline adaptation: Baseline attacks are adapted to the same tool-executing setting using transient interaction channels rather than persistent executable skill code.GCG, AutoDAN, and CPA use adversarial suffixes or instruction patterns, while BadChain and AgentPoison inject malicious instructions into agent components.
  • Cross-framework evaluation: Cross-framework evaluation measures SkillTrojan’s attack success rate across multiple practical agent frameworks and backbone models on a separate industry-oriented coding benchmark.The cross-framework results should not be interpreted as direct numerical comparisons with the EHR SQL results.
  • Clean-task utility: SKILLS (NATIVE) achieves the best clean accuracy across all four EHR SQL settings, while SkillTrojan preserves much of this benign utility.The native stack removes trigger logic, payload fragments, and verification-side effects.
  • Clean-task utility: SkillTrojan’s clean-task improvement over NON-ATTACK is attributed to the installed skills rather than the backdoor logic.This conclusion follows from comparing NON-ATTACK, SkillTrojan, and SKILLS (NATIVE) on the same EHR SQL setup.

G. Cross-Framework Evaluation

The cross-framework evaluation tests whether SkillTrojan generalizes beyond the main EHR SQL setting and one agent stack. A realistic EHR SQL package illustrates how benign-looking scripts can compose into trigger-activated malicious behavior.

  • Cross-framework evaluation: SkillTrojan is evaluated on Claude-Code, IFlow, OpenHands, and OpenClaw using a separate industry-oriented coding benchmark.These experiments assess generalization across practical agent frameworks rather than directly comparing numerical results with EHR SQL.
  • Threat-model example: The example EHR SQL skill package presents schema analysis, query construction, result formatting, and verification as a benign optimization workflow.Its package includes SKILL.md, schema_analyzer.py, query_builder.py, result_formatter.py, and verify_results.py.
  • Threat-model example: In the backdoored version, selected scripts preserve advertised input-output behavior while emitting encrypted fragments under a trigger condition.The verifier checks whether required fragments have been collected within the workflow.
  • Threat-model example: The harmful behavior emerges compositionally from plausible skill steps, without requiring any single component to expose a complete plaintext payload or overt malicious instruction.This distinguishes skill-level backdoors from direct malicious tools.
Loading 2604.06811v2…