Source-linked AI summary
Supply-Chain Poisoning Attacks Against LLM Coding Agent Skill Ecosystems
Yubin Qu, Yi Liu, Tongcheng Geng, Gelei Deng, Yuekang Li, Leo Yu Zhang, Ying Zhang, Lei Ma
TL;DR
The paper addresses whether poisoned third-party skills can make coding agents execute malicious host actions despite alignment and architectural defenses. It introduces DDIPE and an automated adversarial-skill generation pipeline, finding 11.6%–33.5% bypass rates across four frameworks and five models, with residual cases and production vulnerabilities.
Problem
Prior work had not examined whether poisoned skill supply chains could covertly hijack coding agents’ action spaces, including file writes, shell commands, and network requests.
Method
The paper introduces DDIPE and an LLM-driven seed–mutation–validation pipeline that embeds malicious logic in documentation examples and templates and generates adversarial skills.
Results
11.6%–33.5% bypass rates occurred across four frameworks and five models; 2.5% of payloads evaded both static and alignment defenses, and four production vulnerabilities were confirmed.
Takeaways & Limitations
The findings support auditing skill semantics and treating skill permissions and architectural safeguards as model-dependent security boundaries.
Takeaways & Limitations
The evaluation spans five models and four frameworks but does not cover the Llama model family, and DDIPE’s three camouflage techniques were not individually ablated.
Abstract
from arXiv · showhide
LLM-based coding agents extend their capabilities via third-party agent skills distributed through open marketplaces without mandatory security review. Unlike traditional packages, these skills are executed as operational directives with system-level privileges, so a single malicious skill can compromise the host. Prior work has not examined whether supply-chain attacks can directly hijack an agent's action space, such as file writes, shell commands, and network requests, despite existing safeguards. We introduce Document-Driven Implicit Payload Execution (DDIPE), which embeds malicious logic in code examples and configuration templates within skill documentation. Because agents reuse these examples during normal tasks, the payload executes without explicit prompts. Using an LLM-driven pipeline, we generate 1,070 adversarial skills from 81 seeds across 15 MITRE ATTACK categories. Across four frameworks and five models, DDIPE achieves 11.6% to 33.5% bypass rates, while explicit instruction attacks achieve 0% under strong defenses. Static analysis detects most cases, but 2.5% evade both detection and alignment. Responsible disclosure led to four confirmed vulnerabilities and two fixes.
1 Introduction
The paper identifies poisoned agent skills as a supply-chain route to action-space compromise and introduces PoisonedSkills and DDIPE to evaluate it at ecosystem scale. Across models and frameworks, implicit document payloads bypass defenses, with residual risk confirmed in production.
- Motivation: Agent skills expose a supply-chain attack surface because their operational documentation is trusted and agents hold file, shell, and network privileges.A contaminated skill can therefore produce host-level side effects rather than merely alter generated text.
- Approach: DDIPE embeds malicious logic in benign-looking code examples and configuration templates that agents reproduce during normal task execution.The approach avoids requiring explicitly malicious instructions and targets both model alignment and framework defenses.
- Results: 11.6%–33.5% bypass rates were achieved across four frameworks and five models, whereas explicit instruction injection achieved 0% under the best-defended setup.The evaluation addresses scalable generation and two-layer bypass across heterogeneous agent configurations.
- Approach: An LLM-driven seed–mutation–validation pipeline scales 81 seeds into 1,070 adversarial skills covering 15 MITRE ATT&CK categories.The pipeline is designed to generate diverse payloads situated in plausible contexts, reducing reliance on manual crafting.
- Results: Responsible disclosure produced four confirmed security issues and two deployed fixes across production frameworks.The paper evaluates production validation in addition to controlled experiments.
2 Preliminaries and Related Work
The preliminaries define agent skills as reusable packages whose metadata and execution bodies feed a retrieve–load–execute pipeline. Related work has studied context, tool-selection, and text-level poisoning, but not poisoned skills that induce code execution on the host.
- Agent Skills: Agent skills bundle tool-invocation logic, API resources, contextual prompts, and executable backend logic into reusable workflow units.Their metadata contains descriptions, code examples, and configuration templates, while the execution body contains instructions and scripts.
- Execution Pipeline: A coding agent retrieves top-k skills from a marketplace, injects their metadata into context, and produces an action through an LLM executor.The pipeline is retrieve–load–execute, and skill metadata enters context without content-level integrity verification.
- Related Work: Prior supply-chain and agent-specific studies address model data, context manipulation, tool-selection rankings, skill-file tampering, or text-generation pollution.These lines of work do not examine execution of malicious code on the victim machine through the open skill marketplace.
- Research Gap: The paper targets the post-loading phase: whether embedded payloads can induce action-space compromise after poisoned skill content enters the agent context.This distinguishes file, shell, and network side effects from text-only corruption.
3 Threat Model
The threat model studies whether a poisoned skill can induce harmful post-loading execution after retrieval succeeds, despite model and framework defenses. The attacker influences the agent only through skill content, which can cause reproduced payloads to become host-level operations.
- The study assumes retrieval succeeds and focuses on the post-loading phase, where a loaded skill may trigger harmful execution despite safety-alignment and architectural defenses.
- The attacker publishes an adversarial skill but cannot alter user queries, access system prompts, or bypass runtime isolation; skill content is the only influence path.
- Skill metadata containing descriptions, code examples, and configuration templates can induce the agent to reproduce malicious patterns as reference implementations.
- Because the agent executes its own generated output, reproduced payloads translate into file, shell, network, or other action-space operations on the victim machine.
- The attacker targets matched query classes so the agent covertly executes predetermined actions involving system control, asset exfiltration, or infrastructure poisoning.
4 Methodology
The methodology formulates successful poisoning as jointly embedding a concealed payload and inducing its execution, then operationalizes DDIPE and an LLM-driven generation pipeline across a structured attack taxonomy. The approach combines code and configuration poisoning with camouflage, incident-derived taxonomy construction, and iterative seed–mutation–validation.
- 4.1 Problem Formulation: A successful attack requires both adversarial documentation containing a concealed payload and agent reproduction and execution of that payload when a query activates the skill.
- 4.1 Problem Formulation: Condition 1 embeds malicious logic within legitimate code examples or configuration templates while assuming the intermediate retrieval phase succeeds.
- 4.1 Problem Formulation: Condition 2 requires the agent to reproduce the payload in normal output and execute it through its action-space interface.
- 4.2 DDIPE: DDIPE embeds malicious logic in documentation structures rather than standalone commands, exploiting coding agents’ tendency to reproduce authoritative reference patterns.
- 4.2 DDIPE: Code Example Poisoning places payloads in Markdown code blocks, whereas Configuration Template Poisoning plants them in YAML, JSON, or Makefile deployment templates.
- 4.2 DDIPE: Camouflage uses functional-compliance disguises, silent exception suppression, and false-authority suggestions, which can be chained across attack vectors.
- 4.3 Generation Framework: The LLM-driven generation framework iteratively transforms malicious logic into diverse adversarial skill components through seed, mutation, and validation stages.
- 4.3.1 Attack Taxonomy Construction: The attack taxonomy is grounded in more than 150 real supply-chain poisoning incidents from 2021 and 2026, with techniques mapped to MITRE ATT&CK categories spanning seven tactical stages.
5 Evaluation
PoisonedSkills was evaluated across diverse agent systems and models, showing that DDIPE can trigger direct execution despite layered defenses. Attack success varied substantially by model, architecture, payload brevity, and category, while responsible disclosure confirmed vulnerabilities in deployed systems.
- Experimental Setup: 1,070 adversarial skills were evaluated across four agent frameworks and five models along three evaluation dimensions.The evaluation spans model alignment, architectural defenses, and sandbox isolation strategies.
- Aggregate Results: All three models executed payloads under Claude Code, with Sonnet 4.6 reaching DER 2.3% and MiniMax-M2.5 reaching DER 13.3%.MiniMax executed 142 instances, 5.7× Sonnet’s rate under the same framework.
- Aggregate Results: GLM-4.7’s DER increased from 2.4% on Claude Code to 27.1% on OpenHands, an 11.3× amplification when architectural protection was removed.Sonnet 4.6’s bypass rate increased to 22.0% on OpenHands, showing model-dependent amplification.
- Payload Characteristics: Executed payloads averaged 1,850 B versus 2,214 B for refused samples under Claude Code with Sonnet 4.6.A 479-byte pip configuration script was executed by all three models, which treated it as routine environment configuration.
- Cross-Model Disagreement: Model alignment produced distinct blind spots: Sonnet executed 11 exclusive samples, GLM 14, and MiniMax 78.Sonnet’s exclusive executions primarily used DevOps disguises, while GLM often halted when task intent and payload behavior diverged.
- Cross-Model Disagreement: Only 1.6% of payloads bypassed all three models simultaneously under Claude Code, compressing single-model bypass rates of 13–20%.The all-model bypass count was 17, compared with 144 for Sonnet, 180 for GLM, and 219 for MiniMax.
- Case Studies: Four vulnerabilities were confirmed through responsible disclosure, and two resulted in deployed fixes.Case studies included Codex sandbox semantic bypass, Gemini CLI CI/CD amplification, and Claude Code skill-loading disclosure.
6 Discussion
The discussion identifies scalable adversarial skill generation and systemic architectural gaps as central risks. It also reports that heterogeneous model deployment can reduce joint bypasses, while static scanning leaves residual cases that evade both detection and alignment.
- Defense Interaction: Strong alignment produced only 5 sleeper activations across frameworks, whereas weak alignment produced 219.The relationship indicates that architectural defenses matter more when model alignment is weak.
- Defense Diversity: Cross-model ensemble voting reduced the joint attack surface from 13–20% for a single model to 1.6%.The paper presents heterogeneous model deployment as a practical defense multiplier within the evaluated setting.
- Residual Risk: Static scanning caught 90.7% of adversarial samples, but 99 samples evaded all four detection layers and 27 also penetrated model alignment.These residual cases disguised attacks as compliance reports, configuration writes, or DevOps hooks.
- Production Vulnerabilities: The four confirmed vulnerabilities shared a root cause: frameworks granted tool-execution privileges using syntactic boundaries rather than semantic intent.The discussion characterizes these as architectural blind spots and notes that vendor fixes added semantic context or removed permissions.
7 Threats to Validity
The evaluation has several scope and validity constraints, including measurement choices, possible same-family overfitting, and incomplete ecosystem coverage.
- Measurement: Execution classification uses system-call log matching with T=0.3 and 10% stratified human review.The review achieved Cohen’s κ=0.88.
- Generalization: Claude Opus 4.6 generated payloads, creating a same-family overfitting concern despite evidence of cross-family transfer.Sonnet 4.6’s bypass rate was 13.5%, compared with 16.8% for GLM-4.7 and 20.5% for MiniMax-M2.5.
- Scope: The evaluation covers five models and four frameworks but excludes the Llama ecosystem and Cursor.It also assumes that poisoned skills are retrieved and loaded, while larger skill inventories may dilute attack probability.
8 Conclusion
PoisonedSkills evaluates DDIPE against supply-chain defenses in coding-agent skill ecosystems. Across tested configurations, defenses were not uniformly immune, and responsibly disclosed vulnerabilities supported the findings beyond the laboratory.
- 8 Conclusion: PoisonedSkills evaluates supply-chain defenses using Document-Driven Implicit Payload Execution across 1,070 adversarial skills, four frameworks, and five models.No tested configuration was immune, with even the strongest defense allowing 2.3% direct execution.
- 8 Conclusion: 2.4% to 27.1% was the execution-rate increase after removing guardrails for weakly aligned models, while strongly aligned models were barely affected.Only 1.6% of payloads bypassed all tested models simultaneously.
- 8 Conclusion: Four responsibly disclosed production vulnerabilities confirmed that the findings extended outside laboratory settings.The authors conclude that effective defense requires semantic auditing, skill-level permission scoping, and alignment-calibrated architectural safeguards.
Data Availbility
The paper makes its data and source code available through its website.
- Data Availability: The paper’s data are available at its website.
- Data Availability: The paper’s source code is available at its website.
- Data Availability: The availability statement directs readers to the authors’ website.