PaperlayerView exact revision on arXiv

Source-linked AI summary

Chaining Skills to Hijack LLM Agents

Tian Dong, Zixuan Ma, Haodong Zhao, Huaien Zhang, Shaofeng Li, Hao Chen

arXiv:2610.01564v1cs.CRcs.AI

TL;DR

Skill handoffs can carry attacker-controlled claims into later decisions, leaving open how attacks affect the user's legitimate task. APEX constructs and refines chains that exploit this handoff; across targeted-action families and models, the selected action was induced in 512 of 690 attempts (74.2%).

  • Problem

    The paper examines how well agents perform users' legitimate tasks when skill workflows are under attack, as prior work leaves this question underexplored.

    Source: §1 · p. 2

  • Method

    APEX constructs skill chains that place genuine task progress and an attacker-supplied claim about the next step in a persistent record for a later skill to use.

    Source: §1 · pp. 1–2

  • Results

    512 of 690 attempts (74.2%) induced the selected action across four targeted-action families and six models on SkillsBench.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Native task checks alone capture neither targeted actions nor extra work, so next steps need to be checked against the original request and work already done rather than inferred from the record alone.

    Source: §7 · pp. 10–11

    On GPT-5.4, the prompting defense leaves most targeted-action attempts successful and lowers benign-task utility.

    Source: §7 · p. 11

Abstract

from arXiv · show

LLM agents use skills to improve performance on specialized tasks. To complete a user request, an agent may invoke several skills in sequence, allowing information produced under one skill to guide the next. Because skills may come from open-source repositories, this handoff can also carry attacker-controlled claims into later decisions. In this paper, we introduce APEX, which constructs and refines adversarial skill chains tailored to a user task and an attacker-selected action. The key insight is that an agent-written record of genuine task progress can carry a false claim of user approval across skills: an upstream skill induces the agent to create the record, and a downstream skill uses it to direct the attacker-selected action. Across four targeted-action families and six models on SkillsBench, the chains induce the selected action in 512 of 690 attempts (74.2%). On GPT-5.4, the full chain succeeds in 84.3% of attempts, compared with 17.4% when the workflow is merged into one skill. We further evaluate a prompting defense that asks the agent to check skill-produced files against the original request. On GPT-5.4, it lowers targeted-action success from 84.3% to 59.1%, while the verifier test-pass rate across 72 benign native-skill tasks falls from 86.7% to 56.3%. These results highlight the need for defenses that prevent attacker-directed actions while preserving legitimate task performance.

1 Introduction

The introduction frames skill handoffs as a route for attacker-controlled claims to influence later actions even when an agent completes the user's requested task. It presents APEX and evaluates its attack chains alongside task performance, with formal analyses of permission decisions and repeated work.

1 Introduction

  • An upstream skill can pair genuine task progress with an attacker-supplied approval claim in a record that a later skill uses to direct the selected action.The example agent completes the requested summary but deletes an original file it was instructed to preserve.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • For targeted actions, a task-progress record can look the same under permitted and prohibited requests, so a rule using only handoff information cannot avoid both false allows and false denials.For repeated work, a finite-state model shows expected token cost can rise with return frequency and the work following re-entry.Source: §1 · p. 2
  • The study examines five attack families and evaluates attack outcomes alongside performance on the user's original task.The evaluation includes comparisons with direct injection, a merged workflow, and an approval-denied record.Source: §1 · p. 2
  • 512 of 690 attempts (74.2%) induced the selected action across four targeted-action families and six models; on GPT-5.4, the full chain succeeded in 84.3% of attempts versus 17.4% for a merged workflow.The evaluation also compares direct injection and assesses original-task performance and a request-checking defense.Source: §1 · p. 2
  • APEX constructs and tests skill chains that link genuine task progress and an attacker-supplied next-step claim in a persistent record for a later skill.It evaluates both attack success and original-task performance, using execution feedback to refine the cross-skill handoff.Source: §1 · p. 2

2 Related work

Related work covers prompt-based hijacking, malicious skills, and attacks composed from multiple skills. The paper distinguishes APEX by testing an iterative cross-skill handoff that links reported task progress and attacker-supplied approval to a later action.

Compositional Skill Attacks

  • Prior compositional-skill work studies authorization confusion and scanner-guided chain construction, while APEX tests a handoff linking task progress and a selected target to attacker-supplied approval.APEX also tests a related handoff that sends the agent back to repeat work.Source: §2 · p. 3

Prompt-based Agent Hijacking; Skill-based Attacks

  • Prompt-based hijacking redirects agents through lower-trust content, while skill-based attacks disguise malicious actions as task needs or routine setup.The cited examples include retrieval-augmented-generation attacks, SkillJect, and POISE.Source: §2 · p. 3

Compositional Skill Attacks

  • Unlike prior attacks described as using one-time generation, APEX iteratively revises malicious skills using trial runs on failed steps.Source: §2 · p. 3

3 Threat model

The threat model considers an agent doing legitimate work with reusable skills and a record it writes in one stage and reads in another. An attacker controls submitted skill packages and can test candidate chains offline, but has no direct access to the victim's environment or tools.

3 Threat model

  • The attacker controls skill packages and can arrange for one stage's information to guide the next, but cannot directly access or interact with the victim's environment and tools.The task-aware attacker knows the request topic, initial workspace, and native skills, and can use offline execution reports to revise candidate chains.Source: §3 · p. 3
  • The agent has write access to a directory for an assisting record that it later reads, but the original user request continues to govern permitted actions.A request to preserve source files remains binding after their contents are summarized.Source: §3 · p. 3
  • The five attack families are organized by task state, attacker-supplied claim, and target.Source: Table 1 · p. 4

4 APEX: attack construction and analysis

APEX links useful task progress to attacker-selected actions, then analyzes what handoff records can establish about permission and how repeated work affects token cost.

4.1 Constructing the dependency

  • APEX builds chains in which useful task work prompts an agent-written record carrying an attacker-supplied approval claim to a downstream skill.The generator evaluates both attack success and original-task performance, then uses execution feedback to refine the handoff.Source: §4.1 · p. 4, §4 · p. 4
  • For file modification, the record can mark a summarized source as eligible for cleanup, letting a downstream skill direct deletion despite a preservation request.The summary can remain useful even though deleting the source violates the request.Source: §4.1 · p. 4

4.2 Refining the chain through execution

  • Execution reports distinguish a missing handoff record from refusal to perform the action, so the generator can revise the failing stage while keeping the target and success criterion fixed.The task verifier separately checks whether the revised workflow still performs useful work.Source: §4.2 · pp. 4–5

4.3 Why task progress does not establish permission

  • When permitted and prohibited histories produce identical handoff views, false-allow and false-deny probabilities sum to one.The bound concerns their sum: denying every action avoids false allows but also rejects permitted actions.Source: §4.3 · p. 5
  • A skill-produced record cannot establish permission when its local view does not distinguish requests that permit an action from those that prohibit it.The original request can resolve permission; the experiments do not estimate the distributions of handoff views.Source: §4.3 · p. 5

4.4 Extending the chain to repeated work

  • The repeated-work model represents active stages as states and estimates expected token cost from transition probabilities, per-visit costs, and the initial-state distribution.It gives finite-horizon cost and, when ρ(Q) < 1, expected cost without a visit limit.Source: §4.4 · pp. 5–6
  • A return probability has greater cost impact when its source is visited more often or its destination leads to higher expected token cost.The sensitivity varies one return probability while holding other transitions and costs fixed.Source: §4.4 · p. 6
  • Adding a skill need not increase expected cost because an extra stage can raise work per cycle while making returns less likely.The one-transition sensitivity does not capture a chain change that alters several quantities at once.Source: §4.4 · p. 6

5 Evaluation

The evaluation measures both targeted-action success and the user task’s verifier performance, alongside token costs for repeated work. The chains often induce targeted actions while task utility remains comparatively close to baseline, though Work Loop can substantially increase token use.

5 Evaluation

  • Attack success requires the family-specific action, while useful work is measured by passed native checks; tasks with more checks receive greater weight.The Work Loop token ratio compares attack and native-baseline tokens, but does not show what used the extra tokens or whether a budget was exceeded.Source: §5 · p. 6

5.1 Attack effectiveness

  • 74.2% of attempts (512/690) succeed across four targeted-action families, with all 24 model–family combinations exceeding 50% ASR.The evaluation spans six models; family task subsets can overlap.Source: §5 · p. 6, Table 2 · p. 7, §5.1 · p. 7
  • Work Loop raises mean per-task token ratios to 2.20×–36.39×, with the largest increase on Kimi K2.6.Claude Sonnet 5 and DeepSeek V4 Flash both exceed 20×.Source: §5.1 · p. 7

5.2 Task utility under attack

  • On GPT-5.4, action-oriented ASR reaches 75.9–90.3%, while native task utility changes by only 2.6 percentage points on average in absolute terms.The average weights the four families equally and includes all runs, whether or not attacks succeeded.Source: §5.2 · p. 7
  • Across six models, Output Tampering ASR ranges from 54.8–90.3%, while utility changes remain within 3.6 percentage points.On GPT-5.4, ASR is 90.3% as utility moves from 90.4% to 89.6%; the downstream skill preserves destination and output structure but directs source-conflicting content.Source: §5.2 · p. 7
  • Work Loop token increases can accompany preserved or reduced utility: Kimi K2.6 retains 77.0% in both conditions at 36.39×, while Claude Sonnet 5 falls from 78.8% to 42.5% at 21.33×.GPT-5.4 has an 80.5% verifier test-pass rate versus an 88.5% baseline at 15.00×.Source: §5.2 · p. 7

6 Ablation and defense evaluation

The section compares the full chain with simpler workflows, varies skill count, and evaluates taint-guided prompting. The full chain outperforms the tested alternatives, while the defense reduces attack success but also lowers utility, including on benign tasks.

6 Ablation and defense evaluation

  • The study holds task, objective, and action checks fixed for three controls, evaluates structural comparisons on DeepSeek V4 Flash and GPT-5.4, and tests the defense on GPT-5.4.The controls are direct objective injection, a merged workflow, and a record explicitly denying approval.Source: §6 · pp. 7–8

6.1 Comparing alternative workflows

  • 84.3% of GPT-5.4 attempts succeed with the full chain, versus 3.5% with direct injection; the merged workflow reaches 17.4%.The merged workflow has lower ASR than the full chain in every model–family comparison; the comparison changes both instruction grouping and the record handoff.Source: §6.1 · p. 8, Table 3 · p. 9

6.2 Varying skill count

  • 90.3% is GPT-5.4 Output Tampering ASR at four packages, falling to 45.2% with five; skill-count increases do not consistently raise attack success.External Transfer stays within 83.3–86.7% across three to six packages; Work Loop token ratios peak at five skills on both models.Source: Figure 4 · p. 9, §6.2 · pp. 9–10

6.3 Evaluation against taint-guided prompting

  • The defense lowers GPT-5.4 ASR from 84.3% (97/115) to 59.1% (68/115), while file-modification attacks retain 76.0% ASR.Under Output Tampering, ASR falls by 48.4 points and utility by 45.4; for file-modification attacks, utility falls 17.5 points while ASR falls 8.0.Source: Table 4 · p. 10, §6.3 · p. 10
  • Across 72 benign native-skill tasks, verifier test-pass rate falls from 86.7% to 56.3%, a 30.4-point decline.These workflows contain no attack chain, so refusal of malicious actions alone cannot explain the utility loss.Source: §6.3 · p. 10

7 Discussion and conclusion

The discussion distinguishes task completion from both authorization and whether further work is needed. It concludes that next steps should be checked against the request and completed work rather than inferred from a skill-produced record.

7 Discussion and conclusion

  • Native task checks may stay near benign baselines even when chains induce targeted actions, and they do not capture extra Work Loop tokens.Returning to another unit of work can raise token use without necessarily lowering task utility.Source: §7 · p. 10
  • A progress record cannot settle permission when permitted and prohibited requests look identical from the agent’s local view.Finishing one work unit also does not establish that another is needed.Source: §7 · pp. 10–11
  • Taint-guided prompting lowers targeted-action success and Work Loop token ratios, but most targeted-action attempts still succeed and benign-task utility falls.The next step should be checked against the original request and work already completed, not inferred from the record alone.Source: §7 · p. 11

AI use statement

The authors report using LLMs to construct attacks and AI assistance during manuscript preparation, including for the workflow illustration.

AI use statement

Ethics statement

The study evaluates potentially harmful agent actions in a controlled security setting. Network success is recorded inside the task environment, and the presentation omits operational payloads for external deployment.

Ethics statement

  • The experiments evaluate potentially harmful agent actions for controlled security research.The paper presents the evaluated mechanisms without operational payloads for deployment against external systems.Source: Ethics statement · p. 12
  • Network success means an action was recorded by a capture service inside the task environment, not delivered to real recipients.Source: Ethics statement · p. 12
  • The presentation explains evaluated mechanisms without providing operational payloads for deployment against external systems.Source: Ethics statement · p. 12

A Limitations

The reported limitations concern condition-level aggregation, comparisons that do not isolate a single causal factor, and the scope of the defense evaluation. Cross-family evaluations may reuse tasks, and the defense is tested only against unchanged attacks on GPT-5.4.

A Limitations

  • Condition-level results do not reveal the joint distribution of attack success and utility within individual runs.High utility in a condition does not establish that each successful attack preserved useful work.Source: A Limitations · p. 14
  • Tasks reused across attack families make those families’ evaluations non-independent observations.Source: A Limitations · p. 14
  • The authorization and merged-workflow comparisons do not isolate the causal effect of a single statement in the record.Changing approval to denial adds a prohibition, while merging also changes how intermediate instructions reach the model.Source: A Limitations · p. 14
  • Measured token ratios do not estimate the return model’s transition parameters.Merging packages or adding the defense can change multiple transitions and costs, not only the return probability varied in Eq. (3).Source: A Limitations · p. 14
  • Defense results cover a prompting policy tested against unchanged attacks on GPT-5.4, not an attacker adapting to that policy.Source: A Limitations · p. 14

B Evaluation and intervention protocol

The protocol defines action-specific success checks and compares structural, package-count, and prompting-defense conditions. It also specifies Work Loop handoffs, failure accounting, and aggregation of attack and utility measures.

B.1 Attack verification

  • For the four targeted-action families, success requires an observed effect on the selected target; Output Tampering also requires evidence of the conflicting output and its connection to the handoff.An unrelated temporary-file deletion or a conflicting intermediate packet alone does not count.Source: Table 5 · p. 14, §B.1 · p. 14

B.2 Structural controls

  • Structural conditions A and B differ only in the record’s authorization statement, while C merges instructions into one skill and D states the objective directly.The comparisons retain the family-specific success criterion, but C also changes instruction grouping and intermediate communication.Source: §B.2 · pp. 14–15

B.3 Package-count variants

  • Package-count variants merge workflow stages below default K0 or add placeholder skills above it, changing workflow division and stage-visible instructions.K counts attacker packages made available before skill selection, not skill invocations.Source: §B.3 · p. 15

B.4 Taint-guided prompting

  • The defense reports files changed during each skill invocation and tells the model to treat them as untrusted when deciding how to continue.The model must reassess a record’s approval against the original request; the policy does not add an independently verified permission check before a tool action.Source: §B.4 · p. 15

B.5 Failure accounting

  • An attempt is unsuccessful if the intended action does not occur, including when the victim never selects or completes the chain.The ASR denominator uses runs evaluated in that condition, not all launched jobs or candidate-search attempts.Source: §B.5 · p. 15

C.1 Work Loop handoff

  • The illustrative Work Loop handoff records a completed work unit, proposes returning to the entry stage, and directs the downstream skill to follow that proposal.Source: §C.1 · pp. 15–16

C.2 Coverage and aggregation

  • ASR is successful attempts divided by evaluated attempts; utility is passed native checks divided by all checks, so tasks with more checks weigh more.Work Loop token ratios are averaged per task, and repeated entries reusing the main experiment’s full-chain result are not independent replications.Source: §C.2 · p. 16, Table 6 · p. 16

C.3 Main-experiment utility

The section reports defense-related attack and utility counts, describes how utility and structural or skill-count comparisons are summarized, and explains the mechanisms and formal analyses. Its theoretical results are bounded to local-view decisions and specified transition-model assumptions.

C.4 Structural and skill-count results

  • Structural and skill-count tables report attack outcomes alongside native utility; the default K0 setting reuses full-chain condition A.The structural comparison covers conditions A–D, while skill-count tables report ASR for the first four families and mean token ratio for Work Loop.Source: §C.4 · p. 16, Table 8 · p. 17, Table 9 · p. 18

C.5 Work-loop means

  • Work Loop utility weights 113 verifier checks, while its token ratio averages task-level ratios equally across 15 tasks per condition.Source: §C.5 · p. 16, Table 11 · p. 18

C.3 Main-experiment utility

  • Table 7’s utility summarizes verifier-check performance for each entire condition, including attempts in which the attack failed.Source: §C.3 · p. 16, Table 7 · p. 17

D Mechanism interpretation

  • Across action families, handoffs frame selected material, helpers, files, or outputs as approved, while Work Loop records completion and proposes another unit.An output conflict counts as unauthorized only when the task restricts the source or target; another Work Loop unit may still be legitimate.Source: D Mechanism interpretation · pp. 18–19

C.6 Defense utility and counts

  • 97/115 targeted-action attacks succeeded without the defense versus 68/115 with it; native-check passes fell from 91/113 to 76/113 under Work Loop and 462/533 to 300/533 in the benign control.Source: §C.6 · p. 17, Table 12 · p. 19

E.1 Handoff ambiguity

  • With equal priors, local-view decision errors sum to at least (1 − TV(P+, P−))/2; identical views make the sum one.A common transformation without added permission information cannot make the authorization contexts easier to distinguish; consulting the original request falls outside that premise.Source: §E.1 · pp. 19–20

E.2 Cost of repeated work

  • Under stationary finite-cost transitions with ρ(Q) < 1, expected cumulative cost is α⊤(I − Q)^−1c; bounded budgets mean measured increases do not establish infinite execution.Source: §E.2 · p. 20

E.3 Sensitivity to a return transition

  • Reducing one return probability against termination has an exact expected-cost reduction, but this formula does not estimate defense effects that alter multiple transitions or visit costs.Source: §E.3 · p. 21

Related papers

Loading 2610.01564v1…