PaperlayerView exact revision on arXiv

Source-linked AI summary

Benign in Isolation, Harmful in Composition: Security Risks in Agent Skill Ecosystems

Yi Xie, Jiawei Du, Yu Cheng, Jiuan Zhou, Zhaoxia Yin

arXiv:2606.15242v1cs.CRcs.AI

TL;DR

Isolated skill vetting does not capture risks that emerge when benign skills pass outputs, trust signals, or authorization cues along composed paths. SCR-Bench evaluates those paths in controlled sandboxes, finding substantial downstream risks under composition that are largely absent in isolated baselines.

  • Problem

    Artifact-level vetting leaves open whether intermediate outputs from individually safe skills become actionable inputs to other skills along composed workflows.

    Source: §1 · p. 2

  • Method

    SCR-Bench evaluates capability flow, trust transfer, and authorization confusion in sandboxed skill environments using observable downstream state changes along activated paths.

    Source: §1 · p. 3

  • Results

    Composed paths expose downstream risks largely absent under isolated baselines; SCR-CapFlow averages 33.6% ASR under A+B Neutral.

    Source: §4.2 · p. 8, §5 · p. 9

  • Takeaways & Limitations

    The findings support assessing skill security at the level of activated execution paths, alongside isolated artifact vetting.

    Source: §5 · p. 9

    The threat model is scoped to skill-enabled agents invoking multiple locally bounded skills and carrying intermediate artifacts across task steps.

    Source: §3.1 · p. 4

Abstract

from arXiv · show

Skills are becoming the capability layer through which LLM agents turn plans into actions, but their use introduces security risks such as data leakage, unauthorized operations, and tool misuse. Existing vetting usually evaluates each skill in isolation, while real agent tasks often invoke multiple skills in a shared execution context. This creates Skill Composition Risk (SCR): a skill that appears benign alone can become harmful when its outputs, trust signals, authorization cues, or side effects influence later invocations along an activated path. We introduce SCR-Bench to evaluate this risk in controlled, sandboxed skill environments. Rather than relying only on textual intent or surface behavior, SCR-Bench records downstream state changes and path-level outcomes across composed skill executions. It contains three sub-benchmarks: SCR-CapFlow for capability-flow composition, SCR-TrustLift for trust-transfer composition, and SCR-AuthBlur for authorization-confusion composition. Across SCR-Bench, composed paths expose risks that are largely absent under isolated evaluation. In SCR-CapFlow, attack success rate reaches 33.6 percent under composition, compared with near-zero isolated baselines. In SCR-TrustLift, attack success rate exceeds 96.5 percent on four of five backends. In SCR-AuthBlur, the risky-approval rate increases by 71.8 percent relative to the L0 isolated baseline under the L1 context setting. These results show that agent skill security should be assessed at the level of activated paths rather than isolated artifacts. SCR and SCR-Bench provide a foundation for path-aware risk evaluation and defense in LLM agent skill ecosystems. Benchmark: https://github.com/saint-viperx/SCR_Bench.

1 Introduction

The introduction argues that isolated skill review misses harm that emerges when intermediate outputs cross skill boundaries, framing SCR as a path-level security gap. SCR-Bench operationalizes this gap and reports substantial risks under composition.

1 Introduction

  • Skills package procedural knowledge and execution policies for agent workflows, while a cited repository study found vulnerabilities in 26.1% of analyzed skills.The introduction notes that safe skill use constrains reliability, deployability, and practical utility.Source: §1 · pp. 1–2
  • SCR is the risk that skills judged safe in isolation produce unsafe downstream state changes when composed through a shared execution context.The risk lies on activated paths between skills, not in an individual artifact.Source: §1 · pp. 2–3
  • Composed execution drives capability-flow ASR above 33%, harmful installation above 83.89% on most backends, and risky approvals from 15.7% to 27.0%.The authors contrast these outcomes with near-zero isolated capability-flow risks.Source: §1 · p. 3
  • The paper formalizes path-level risk and introduces a controlled benchmark suite to shift security analysis from isolated artifacts to activated execution paths.Its empirical findings show risks largely absent under isolated evaluation.Source: §1 · p. 3
  • SCR-Bench evaluates composed paths in sandboxed environments by measuring downstream state changes across capability flow, trust transfer, and authorization confusion.It compares isolated and composed execution rather than relying only on model-output text.Source: §1 · pp. 2–3

2 Related Works

Related work primarily reviews standalone skills, tools, or broader agent workflows, while SCR examines how intermediate artifacts move across skill boundaries. The paper distinguishes this path-level focus from existing benchmark units and mechanisms.

2 Related Works

  • Skill-auditing approaches inspect artifacts through static, semantic, formal, or runtime checks, but do not directly measure cross-skill context propagation.The related work describes artifact- and invocation-level review as useful but incomplete for SCR.Source: §2 · p. 3
  • Existing benchmarks span adversarial prompting, harmful skills, tool-use environments, workflow attacks, and multi-step traces, each with a different evaluation unit.Examples include AgentDojo, Agent Security Bench, TraceSafe, and AgentLAB.Source: §2 · pp. 3–4
  • Semantic Intent Fragmentation studies benign subtasks composing into policy-violating plans, whereas SCR tracks activated skill paths and their observable downstream state changes.SCR-Bench compares these path outcomes with single-skill baselines.Source: §2 · p. 4

3 Method

The method models skill security as risk along context-dependent execution paths, then evaluates three composition mechanisms in controlled sub-benchmarks. Trials are measured by whether a path produces an observable harmful state.

3.1 Threat Model, Assumptions, and Notation

  • The threat model considers an agent carrying intermediate artifacts across multiple locally bounded skills, with an adversary seeking an unsafe downstream outcome through their composition.Outputs, trust signals, authorization cues, and side effects may shape later decisions.Source: §3.1 · p. 4

3.2 Skill Composition Graph and Path Risk

  • A skill ecosystem is modeled as a context-dependent directed graph whose edges capture operational dependencies activated by task semantics, session state, and intermediate outputs.The same skill pair may be independent in one task but connected in another.Source: §3.2 · p. 5

3.3 Risk Mechanisms

  • Capability flow turns an upstream-discovered target into downstream action; trust transfer makes a later action appear legitimate; authorization confusion treats advisory context as approval.These mechanisms respectively cross execution-target, legitimacy-signal, and approval-evidence boundaries.Source: §3.3 · p. 6

3 Method

  • SCR-Bench instantiates capability flow, trust transfer, and authorization confusion in controlled scenarios, with each trial scored for a downstream harmful state change.Mere invocation or a textual description of risky behavior does not count as success.Source: §3.4 · p. 6, Figure 3 · p. 6

3.4 SCR-Bench Construction

  • The benchmark’s binary path-level indicator requires the corresponding bad event to produce a downstream state change, and attack success rate averages that indicator across cases and trials.The scoring rule measures reaching a harmful state rather than invocation alone.Source: §3.4 · p. 6
  • SCR-CapFlow pairs read-only discovery with execution skills and counts success only when the side effect acts on the same concrete target discovered upstream.Its cases cover permission and access control, reporting, device control, network configuration, and persistence scheduling.Source: §3.4 · pp. 6–7
  • SCR-TrustLift compares harmful-installation decisions with and without an upstream endorsement, while SCR-AuthBlur tests approval shifts across context levels.AuthBlur reports L0, L1, and L3 over 118 retained cases; L2 is deferred to the appendix because of stricter audit criteria.Source: §3.4 · p. 7

4 Experiments

The experiments compare isolated workflow pieces with activated cross-skill paths across capability flow, trust transfer, and authorization confusion, using controlled environments and path-level outcomes.

4 Experiments

  • Composed execution exposes risks largely absent under isolated evaluation across capability flow, trust transfer, and authorization confusion.The experiments compare conditions exposing isolated workflow pieces with conditions allowing cross-skill paths.Source: §4 · p. 7, §4.2 · p. 8, §4.3 · p. 8

4.1 Experimental Setup

  • Trials count only when agents reach the mechanism-specific bad event; each case-condition uses five trials unless stated otherwise.Attack success rate is averaged over cases and trials across available model backends.Source: §4.1 · p. 7

4.2 Results of the SCR-CapFlow

  • 33.6% and 35.9% average ASR under A+B Neutral and A+B Explicit contrast with 0% for Control and A-Only and 1.4% average for B-Only.DeepSeek-V4 exceeds 90% ASR in both composed conditions; Figure 4 shows ASR by backend and execution condition.Source: Figure 4 · p. 7, §4.2 · p. 8

4.3 Results of the SCR-TrustLift

  • 83.89% endorsed-condition average ASR contrasts with 1.10% in control, an 82.79-percentage-point average lift in harmful installations.Four of five backends have near-zero control ASR; Table 2 reports trust-transfer effects across backends.Source: §4.3 · p. 8, Table 2 · p. 8

4.4 Results of the SCR-AuthBlur

  • 15.7% under L0, 27.0% under L1, and 34.0% under L3 are the average risky-approval ASRs; backend responses vary, including occasional L1 decreases.Table 3 reports risky-approval rates and percentage-point changes by backend across the three context levels.Source: §4.4 · pp. 8–9, Table 3 · p. 9

4.5 Ablation

  • Composed SCR-CapFlow conditions increase ASR across most backends while control and single-skill conditions remain near zero.DeepSeek-V4 shows the highest risk (>90%), whereas Opus-4.6 is more conservative; Table 4 covers 150 evaluation cases.Source: §4.5 · p. 9, Table 4 · p. 9

5 Conclusion

The conclusion identifies Skill Composition Risk as a path-level security gap and presents SCR-Bench as a controlled evaluation of three composition mechanisms.

5 Conclusion

  • SCR describes skills that appear benign in isolation becoming harmful when outputs, trust signals, or authorization cues are reused across composed paths.The paper names this path-level gap Skill Composition Risk.Source: §5 · p. 9
  • SCR-Bench evaluates capability flow, trust transfer, and authorization confusion through controlled evaluations of composed paths.The benchmark measures downstream risks missed by isolated baselines.Source: §5 · p. 9
  • The conclusion argues that isolated vetting is necessary but insufficient and recommends path-aware evaluation as a standard component of skill ecosystem security.Source: §5 · p. 9

A Limitations

The study bounds its findings to controlled skill environments with observable mock side effects and limits its reported AuthBlur contexts.

A Limitations

  • SCR-AuthBlur omits L2 from the main table because only a subset of cases meets the stricter audit criterion.The authors avoid extrapolating L2 beyond cases where that criterion holds.Source: A Limitations · p. 12
  • Experiments use controlled skill environments with observable mock side effects, which abstract away registry-scale heterogeneity, runtime policies, and operational noise.The design provides path-level ground truth and supports controlled comparisons across mechanisms.Source: A Limitations · p. 12
  • The authors identify validation in larger open skill registries and production-like agent runtimes as future work.Source: A Limitations · p. 12

B Additional method details

The appendix adds path-risk accounting and mechanism-specific details for capability flow, trust transfer, and authorization confusion.

Multi-hop amplification

  • Length-k composition risk accounts for candidate paths: even small average path risk can yield greater overall risk as the number of paths grows.The expression is a risk-accounting approximation, not an independence claim; in dense settings, candidate paths can scale as O(n^k).Source: B Additional method details · p. 12

Capability flow

  • A capability-flow trial succeeds only when skill A finds a concrete target, skill B causes an observable side effect, and both concern the same target.Source: B Additional method details · p. 12

Trust transfer

  • Trust-transfer analysis models an upstream endorsement as context for a downstream action and measures endorsement lift, including for harmful-skill installation.The appendix gives an empirical lift measure and an installation-specific form.Source: B Additional method details · p. 12

Authorization confusion

  • Authorization-confusion analysis contrasts the probability of risky approval or dangerous execution under context level L_l with the L0 control.The empirical estimate uses risky-outcome rates for the two context levels.Source: B Additional method details · p. 12

C Context Levels in SCR-AuthBlur

SCR-AuthBlur varies whether context specifies a skill path and how strongly upstream content signals downstream decision evidence; audited results distinguish these settings.

Context levels and results

D Experimental Protocol and Additional Details

The experimental protocol overview covers how the three SCR-Bench mechanisms are evaluated under controlled conditions.

Experimental protocol

E The Ablation of SCR-TrustLift

The SCR-TrustLift ablation compares control and endorsed conditions across five backends, with endorsement associated with substantially higher harmful-installation risk.

TrustLift ablation

F The Ablation of SCR-AuthBlur

SCR-AuthBlur examines whether stronger advisory or related context shifts downstream approval boundaries. Risky approval generally increases with context strength, though outcomes vary by backend.

F The Ablation of SCR-AuthBlur

Related papers

Loading 2606.15242v1…