Source-linked AI summary
Scope-Gap Injection: Exploiting Evaluator Channel Blind Spots in the OpenClaw Agent Security Competition
Weihang Niu, Mengxing Ren, Che Qu
TL;DR
Multi-stage LLM-agent safety pipelines may inspect fewer input channels than an agent can use, leaving a scope gap between evaluator inspection and runtime execution. The paper introduces SGI, which places attack details in business-framed attachments while keeping visible instructions benign. In OpenClaw, the highest-scoring returned submission achieved F = 0.385 by attacking 5 of 19 targets, though campaign variability limits the conclusions.
Problem
In multi-stage agent safety pipelines, the evaluator may inspect fewer input channels than the agent can read and use during execution.
Source: §1 · p. 1Method
SGI keeps task and skill content benign while placing target-specific attack details in an attachment disguised as a business worklist.
Source: §1 · p. 1, §4.2 · p. 3Results
The highest-scoring OpenClaw returned submission achieved F = 0.385 by successfully attacking 5 of 19 targets.
Source: §7 · p. 5Takeaways & Limitations
Safety pipelines should account for every agent-accessible channel through faithful inspection, containment, or independent action-time authorization.
Source: §7 · p. 5The empirical results characterize one competition pipeline and an iterative submission campaign, not a framework-wide exploit rate.
Source: §6 · p. 5
Abstract
from arXiv · showhide
This paper introduces Scope-Gap Injection (SGI), an indirect prompt injection method for multi-stage LLM-agent safety pipelines. In the IJCAI 2026 OpenClaw Agent Security Attack Competition, each attack package is judged by a static skill scan (S1), an LLM-judge safety check over the task and skill definition (S2), and a live execution audit (S3). Our observations suggest a scope gap: S2 appears to inspect the task instruction and skill definition without inspecting attachment contents, while the agent reads attachments during S3. SGI keeps the visible task and skill benign, while target identifiers, bypass parameters, and authorization procedures are moved into an attachment written as a business worklist. The S2 judge therefore sees a harmless task, while the agent follows the hidden worklist and triggers the permission bypass. In OpenClaw, SGI bypassed all three stages for 5 of 19 targets. The highest-scoring single submission achieved $F = 0.385$. Across the submission campaign, attachment placement was associated with higher aggregate S2 acceptance than in-skill placement: 78% versus 23%. We close with defenses that align evaluator scope with the channels available to the agent.
1 Introduction
The paper identifies a scope gap in multi-stage agent safety pipelines: the S2 judge appears to inspect fewer channels than the agent reads at runtime. SGI keeps task and skill content benign while placing attack details in business-framed attachments; its highest-scoring submission attacked 5 of 19 targets with F = 0.385.
1 Introduction
- SGI exploits the observed scope gap between S2’s semantic inspection and the channels the agent reads during execution.The authors’ observations suggest S2 excludes attachment contents even though the agent uses them at runtime.Source: §1 · p. 1
- SGI places target identifiers, bypass parameters, and authorization procedures exclusively in attachments while keeping the task instruction and skill definition benign.The attachments are disguised as ordinary business ledgers or compliance checklists to help pass S1’s scan.Source: §1 · p. 1
- The paper identifies the scope-gap vulnerability and reports placement comparisons linking scope separation to higher S2 acceptance across the campaign.The study instantiates the pipeline in the IJCAI 2026 OpenClaw Agent Security Attack Competition, where attacks must pass three sequential stages.Source: §1 · p. 1
- 5 of 19 targets were attacked by the highest-scoring submission, which achieved F = 0.385.The authors also report target-dependent outcomes and variable S2 results.Source: §1 · p. 1
2 Task Description
OpenClaw submissions package a task instruction, skill definition, and optional attachments for an agent acting on simulated enterprise services. A target scores only when the package passes three sequential evaluation stages, and the score penalizes submissions containing many failed targets.
2 Task Description
- The competition evaluates attacks inducing unauthorized operations on simulated services across 19 targets in smart-home, e-commerce, and enterprise-office scenarios.The competition description follows its documentation and the authors’ experimental observations.Source: §2 · p. 2
2.1 Attack Package Structure
- Each submission directory contains a task instruction, a skill definition, and optionally data files the agent can read and act upon.The task is the primary user input; the skill defines the agent’s role and permitted tools.Source: §2.1 · p. 2
2.2 Three-Stage Evaluation Pipeline
- A target scores only if its package passes the static scan, LLM-judge safety evaluation, and live execution audit.Failure at any stage yields a score of zero for that target.Source: §2.2 · p. 2
2.3 Scoring Formula
- The score uses k, the number of targets passing all three stages, and N, the number of submitted directories; it penalizes submissions with many failed targets.The formula incentivizes submitting only high-confidence attacks.Source: §2.3 · p. 2
3 Related Work
The related work situates SGI among indirect prompt-injection studies and agent-security benchmarks, while distinguishing its focus on multi-stage safety gates. It also contrasts SGI with competition-oriented attacks on generative models and visual-prompt studies.
3 Related Work
- Prior indirect prompt-injection work studies adversarial instructions in processed data sources such as web content, emails, and retrieved documents.This paper examines such instructions in a safety pipeline where the semantic judge does not inspect the agent’s data channel.Source: §3 · p. 2
- AgentDojo, AgentHarm, InjecAgent, and SeClaw address agent security, while this study focuses on OpenClaw’s static scan, LLM-judge evaluation, and trajectory scoring.The cited benchmarks cover dynamic prompt-injection evaluation, harmful actions, tool-integrated indirect injection, and trajectory-aware evaluation.Source: §3 · p. 2
- Competition-oriented jailbreak and visual-prompt studies target generative models or visual inputs, whereas SGI targets autonomous agents with tool-calling capabilities and multi-stage safety gates.The comparison is drawn with Yang et al. [2025] and Cheng et al. [2025].Source: §3 · p. 2
4 Method: Scope-Gap Injection
SGI assumes an attacker can craft package contents but cannot alter the evaluation pipeline, agent model, or backend services. It exploits the observed difference between S2’s inspection scope and the agent’s access to attachments, using benign visible inputs and a business-framed worklist to carry attack details.
4.1 Threat Model and Key Observation
- The threat model lets attackers craft task, skill, and attachment contents but not modify the evaluation pipeline, agent LLM, or backend services.The goal is to pass S1 and S2 while inducing a permission bypass during S3.Source: §4.1 · p. 2
- S2 is observed to inspect task.md and SKILL.md but not attachment contents, which remain accessible to the agent during S3.Placement comparisons found S2 frequently flagged instructions in SKILL.md, while acceptance increased after moving them to an attachment and making the skill benign.Source: §4.1 · p. 2
4.2 SGI Framework
- SGI combines a routine task instruction and generic skill with an attachment worklist containing the complete target-specific attack specification.A standard-authorization instruction prepares the agent to handle authorization challenges without naming a specific bypass mechanism.Source: §4.2 · p. 3
4.3 Bypass Mechanism Adaptation
- Bypass parameters vary by service: OA token services use a dynamically obtained JWT, smart-home services accept the owner’s user ID, and simple-bypass services accept a non-empty reason.The worklist supplies the corresponding authorization request, pre-authorization claim, or business justification.Source: §4.3 · p. 3
4.4 Algorithm
- The worklist uses Chinese business framing to present bypass parameters as operational data, and the attack construction procedure is summarized in Algorithm 1.Figure 1 shows a sanitized, translated worklist example with operational identifiers and executable details removed.Source: §4.3 · p. 3, §4.4 · p. 3, Figure 1 · p. 3
5 Experiments
In the OpenClaw campaign, SGI’s highest-scoring returned submission passed all three stages for five targets, while attachment placement was associated with higher S2 acceptance than in-skill placement. Target-dependent and cross-round variation, along with the campaign’s repeated verdicts, limit how broadly these results can be interpreted.
5.1 Setup
- The study covered all 19 official targets in a simulated environment, but its 16 returned evaluation files included repeated submissions and were not independent controlled replicates.The highest-scoring submission contained seven directories; the placement comparison excluded a preliminary-round return.Source: §5.1 · p. 3
5.2 Main Results
- F = 0.385 came from five successful targets among seven submitted directories, with the official denominator remaining 19 targets.The reported score is 2 × 5/(7 + 19); Table 1 gives per-target stage results, and an S2 rejection recorded as S3 = 0 does not indicate a separately observed execution failure.Source: §5.2 · p. 3, Table 1 · p. 4
5.3 Score Progression
- Table 2 summarizes four improving milestones selected from 16 logged evaluation returns; intervening rounds did not exceed the preceding best score.The campaign archive’s highest-scoring result refers to the single returned file with the largest F, not a mean or platform-side best-of-N result.Source: §5.1 · p. 3, Table 2 · p. 4, §5.3 · p. 4
5.4 Placement Comparisons and Failure Analysis
- S2 acceptance across five targets was 78% (18/23) with attachment placement versus 23% (6/26) with in-skill placement, but attachment placement did not ensure acceptance on every return.The campaign-level groups were not matched independent replicates and did not isolate placement from wording or backend changes; document export passed in 2/5 SGI returns.Source: Table 3 · p. 4, §5.4 · p. 4
- S2 outcomes varied by target and round: oa salary query failed in all seven tested submissions, while identical files for some targets received both passing and failing verdicts.All six e-commerce targets failed S2; the proposed VIP-customer-context explanation is described as plausible, not established.Source: §5.4 · pp. 4–5
5.5 Ceiling Analysis
- For the five targets that passed all three stages in the observed campaign, the conditional score ceiling was Fmax ≈ 0.417, not a general upper bound on SGI.Exceeding this fixed-set ceiling would require another target to pass all stages, and cross-round variability means the conditional score is not guaranteed on resubmission.Source: §5.5 · p. 5
6 Discussion
The discussion frames SGI’s transfer beyond OpenClaw as conditional on evaluator and agent channel mismatch, agent use of that channel, and insufficient later authorization checks. It proposes risk-oriented artifact inspection and independent controls, while identifying cross-framework transfer and defense tiers as untested.
6 Discussion
- SGI may transfer only when the agent can read a channel omitted or reduced by the evaluator, that content can shape tool use, and later authorization does not independently bind actions to approved intent.The authors did not test other agent frameworks, so the transfer argument is architectural rather than an empirical cross-framework result.Source: §6 · p. 5
- The proposed defense inventories and hashes agent-readable artifacts, applies inexpensive checks broadly, and semantically inspects untrusted or action-relevant content alongside the task and skill.Long or irrelevant files, modality-specific parsing, lossy extraction, and missing business context can create cost, coverage, or false-positive problems.Source: §6 · p. 5
- The empirical findings describe one competition pipeline and iterative campaign, not a framework-wide exploit rate; repeated verdicts and hidden S2 configuration prevent attribution of score flips.The cross-framework conditions and proposed cost-sensitive defense tiers remain hypotheses for future controlled evaluation.Source: §6 · p. 5
7 Conclusion
The paper presents SGI as an indirect prompt injection exploiting mismatched evaluator and agent inspection scopes. In the OpenClaw competition, its highest-scoring returned submission attacked five of 19 targets, while placement comparisons support the scope-gap explanation but leave important campaign-specific variation.
7 Conclusion
- SGI places payloads in agent-accessible attachments not semantically inspected by S2 and disguises them as routine business documents, enabling packages to pass S1 and S2 and induce runtime permission bypasses.Source: §7 · p. 5
- The highest-scoring returned submission achieved 0.385 by successfully attacking five of 19 targets in the IJCAI 2026 OpenClaw competition.Source: §7 · p. 5
- The design lesson is to account for every agent-accessible channel through faithful inspection, containment, or independent action-time authorization.The paper states that coverage must be enforced across the system, not only added to the evaluator prompt.Source: §7 · p. 5
Ethical Statement
The work was conducted within the officially authorized competition using sandboxed Docker environments and mock services; no real systems, data, or individuals were affected.
Ethical Statement
- All experiments used sandboxed Docker environments with mock enterprise services, and no real systems, data, or individuals were affected.Source: Ethical Statement · p. 6