Source-linked AI summary
CompoSkill: Compositional Skill Chain Attacks from Individually Scanner-Passing LLM Agent Skills
Mingxiao Liu, Zhoumian Jiang, Jianan Ma, Jian Zhang, Jialuo Chen, Xinhao Deng, Zhen Wang
TL;DR
Per-skill certification may miss risks that emerge when individually scanner-passing skills are composed in autonomous-agent workflows. CompoSkill models and synthesizes such chains in white-box and black-box settings, achieving formation rates up to 83.3% and 80.6%, respectively.
Problem
Existing marketplace scanners certify skills individually, leaving limited evidence about whether scanner-passing skills remain safe when composed in agent workflows.
Method
CompoSkill models capability-level skill composition with a Skill Composition Graph and dual white-box and black-box attackers that synthesize risky chains.
Results
White-box chain formation reaches 83.3% and black-box chain formation reaches 80.6% on CompoSkill-Bench, while scanners block only a limited fraction of risky compositions.
Takeaways & Limitations
Scanner-passing status does not compose: individually benign skills can still form harmful source–bridge–terminal execution paths.
Abstract
from arXiv · showhide
Autonomous AI agents tackling Long Horizon Tasks depend on marketplace skills that are certified one at a time: a scanner returns a safety verdict for each skill and declares the ecosystem safe if every package passes. We show that this assumption fails under skill composition. A skill may pass the per-skill scanner individually yet participate in a risky composition when an agent connects its outputs, capabilities, or side effects with those of other scanner-passing skills. This makes skill composition risk a path level property rather than a node level property, explaining why existing skill scanners that inspect individual packages achieve limited interception. To study this threat, we present CompoSkill, a framework that constructs skill composition attacks through a dual attacker system. The white-box attacker knows the victim's installed skill pool and directly injects explicit skill-id sequences; the black-box attacker knows only a role profile, downloads the top marketplace skills for that scenario, builds a Skill Composition Graph, and searches for high risk chains whose implicit lures never name skill identifiers. We further construct CompoSkill-Bench, a benchmark of 1,140 records built from long-horizon professional workflows across five threats and six scenarios on OpenClaw and Nanobot. CompoSkill achieves risk Chain Formation Rates (CFR) up to 83.3% in the white box setting and 80.6% in the black box setting, while existing skill scanners block only a limited fraction of the risky compositions. Finally, we observe a bridge-bonus-then-hop-decay pattern: a bridge skill can increase attack success, but Attack Success Rate (ASR) decreases once additional hops make the risk chain longer than three skills. These results expose a systematic gap in single skill certification for autonomous AI agents.
Introduction
Autonomous agents compose marketplace skills across long-horizon workflows, creating attack paths that isolated per-skill screening can miss. CompoSkill formalizes this composition risk and introduces attackers that synthesize multi-hop chains from installed or marketplace skills.
- Introduction: Autonomous agents execute complex, multi-turn professional workflows using skills that provide specialized procedural knowledge, scripts, and resources.These agents can access file systems, terminals, web browsing, and persistent memory, while no single skill covers an entire workflow.
- Introduction: Skills create a broad attack surface through privileged capabilities, policy-shaping documentation, and implicit dataflow edges between skill outputs and inputs.Per-skill marketplace audits inspect packages independently, leaving composition-dependent risks insufficiently addressed.
- Introduction: Individually scanner-passing benign skills can compose into harmful source–bridge–terminal attack chains during real agent workflows.The paper frames skill safety as a path-level property rather than only a node-level property.
- Introduction: Existing composition studies remain limited: SkillProbe audits 2,500 skills across 8 models, SkillReact evaluates 211K pairs, and SCR-Bench covers three mechanisms without attacker-driven chain synthesis.The paper distinguishes CompoSkill by automatically synthesizing and executing multi-hop chains from public marketplace metadata.
- Introduction: CompoSkill uses dual attackers: a white-box attacker accesses the victim’s installed pool, while a black-box attacker builds a Skill Composition Graph from marketplace skills.The black-box attacker searches for high-risk chains without naming skill identifiers.
Related Work
Related work identifies security threats at single injection points and within skill marketplaces, while emphasizing that composition can evade per-component auditing. Existing compositional-risk studies examine protocol-level failures, marketplace auditing, or limited pairwise and single-hop settings.
- LLM Agent Security and Prompt Injection: Prior agent-security studies address jailbreaks, memory poisoning, and multi-turn elicitation as single injection points rather than skill composition.Examples include AgentPoison, MINJA, and Crescendo.
- Skill and Plugin Ecosystem Security: Skill-marketplace attacks use model poisoning, code camouflage, documentation injection, or automated injection generation, while defenses constrain or audit individual skills.The cited systems include BadSkill, PhantomSkill, Dynamic Malicious Skills, SKILL-INJECT, SkillJect, SkillGuard, and Structured Security Auditing.
- Agent Protocol and Skill Compositional Risk: AgentThread formalizes bridged-agent-protocol security, and its specification-level findings include failures that appear only under composition.This frames composition risk as invisible to per-component audits at the protocol layer.
- Agent Protocol and Skill Compositional Risk: SkillProbe audits combinatorial-risk pairs across 2,500 ClawHub skills and 8 LLMs, whereas SkillReact evaluates 211K pairs but remains limited to 2-node, single-hop compositions.SkillProbe performs marketplace auditing rather than runtime exploitation; SkillReact uses human adjudication and an action harness.
Methodology
CompoSkill models marketplace skills by their capabilities, constructs a weighted Skill Composition Graph, and discovers coherent two- or three-skill risk chains under threat-specific constraints. Its benchmark instantiates these chains in realistic professional workflows across six scenarios and two attacker knowledge settings.
- Capability-Level Skill Model: Skills are represented as capability tuples containing input capabilities, output capabilities, and a marketplace-derived risk severity.The capability alphabet is Σ = {file, net, cmd, mem, cfg, db, msg}, with severity values r(s) ∈ {0.2, 0.5, 0.9}.
- Skill Composition Graph: The Skill Composition Graph connects skills when one skill’s outputs overlap another skill’s inputs, weighting edges by the fraction of downstream requirements satisfied.For skills sa and sb, w(sa, sb) = |O(sa) ∩ I(sb)|/|I(sb)|.
- Threat-Constrained Chain Formation: Risk chains are constrained to two or three skills: a source reads sensitive state, an optional bridge transforms it, and a terminal produces an external, persistent, or operationally dangerous effect.Three-skill chains must preserve connectivity through both source-to-bridge and bridge-to-terminal edges.
- Chain Search and Filtering: Black-box chain discovery converts edge weights to log-domain costs, performs constrained k-shortest-path search, and filters graph-valid but professionally implausible paths using metadata-level coherence.The method uses K = 10 final paths and oversamples by α = 10 before filtering.
- Benchmark Construction: The benchmark covers six professional scenarios with 10–14 roles per scenario and installs skills matching daily work routines rather than adversarially selected skills.The scenarios are medical and health, financial and investment, legal and compliance, digital assets and payments, DevOps and system administration, and marketing and information operations.
- Attacker Settings: Each benchmark instance has clean, explicit-injection white-box, and implicit-injection black-box variants, distinguishing utility evaluation from attacks that either name skill IDs or rely on capability-level composition.The black-box attacker starts from a user profile and builds a Skill Composition Graph over likely marketplace skills.
Experiments
Experiments evaluate CompoSkill across four research questions: attack effectiveness, defense bypass, chain length, and an end-to-end case study. Results show strong composition attacks, limited scanner interception, a bridge benefit followed by hop decay, and risk emerging from links among individually safe skills.
- RQ1: Attack Effectiveness: White-box attacks reach up to 59.7% ASR or 83.3% CFR, while black-box attacks reach up to 71.1% ASR or 80.6% CFR without naming skill identifiers.Black-box agents autonomously discover and compose 3-skill chains from professional workflow tasks.
- RQ2: Defense Bypass: 0.50–0.65 Defense Bypass Rates remain after strict blocking, leaving at least half of composition capability despite flagged skills being removed.CFR remains 36.8% for SkillsGuard, 40.8% for Cisco, and 31.6% for NVIDIA SkillSpector, versus 63.2% without scanning.
- RQ3: Chain Length: 64.9% to 80.6% formation for Data Exfiltration and 65.8% to 71.1% for Privilege Escalation show that adding a bridge improves 3-skill chain formation.The bridge intermediate step can naturalize risky handoffs as routine workflow actions.
- RQ3: Chain Length: 55.4% average ASR at L=3 falls to 37.8%, 33.6%, and 21.8% at L=4, 5, and 6, respectively.The study’s five threat curves all improve from L=2 to L=3, while extra hops reduce path coherence and add failure points.
- RQ4: Case Study: The harmful effect arises when an agent links three individually safe source, bridge, and terminal skills, while no single skill both obtains and externalizes the data.Prompt injection delivers the task but is not the root cause; all three skills receive Safe labels in isolation.
Conclusion
Individually scanner-passing skills can become risky when composed into source–bridge–terminal execution paths, shifting safety assessment from isolated skills to runtime trajectories. On CompoSkill-Bench, CompoSkill forms risky chains at rates up to 83.3% white-box and 80.6% black-box, while scanners block only a limited fraction.
- Composition risk: Individually scanner-passing skills can produce harmful behavior when composed into a source–bridge–terminal execution path.The finding shifts the safety question from isolated scanning to the agent’s runtime trajectory.
- Empirical results: 83.3%: CompoSkill achieves white-box skill chain formation rates up to 83.3% on CompoSkill-Bench’s 1,140 records.The benchmark spans five threat categories and six professional scenarios.
- Empirical results: 80.6%: CompoSkill achieves black-box skill chain formation rates up to 80.6%, while existing per-skill scanners block only a limited fraction of risky compositions.The chains do not require a malicious or backdoored skill; participating skills may each pass scanning individually.