ClawSentry: A Progressive Multi-Tier Security Monitor for Safeguarding Autonomous LLM Agents

Kai Wang; Zeming Wei; BiaoJie Zeng; Chang Jin; An Wang; Xiaokun Luan; Zhixiao Lin; Jingjing Qu; Xia Hu; Xingcheng Xu

Source-linked AI summary

ClawSentry: A Progressive Multi-Tier Security Monitor for Safeguarding Autonomous LLM Agents

Kai Wang, Zeming Wei, BiaoJie Zeng, Chang Jin, An Wang, Xiaokun Luan, Zhixiao Lin, Jingjing Qu, Xia Hu, Xingcheng Xu

arXiv:2608.21101v1Aug 21, 2026cs.CRcs.AI

TL;DR

Autonomous LLM agents create progressive, cross-stage security risks that single-boundary and per-call defenses do not fully address. ClawSentry combines admission review, progressive runtime mediation, anti-bypass memory, post-action evidence, and framework-agnostic integration, reducing attack rates while retaining high clean-skill task success. Its measured boundary includes overblocking and evaluation on non-adaptive benchmark attacks.

Problem
Existing defenses largely act at one lifecycle locus or evaluate calls independently, leaving agent risks insufficiently covered across boundaries and attempts.
Method
ClawSentry is a framework-agnostic gateway combining FSPR, three-tier runtime review, session-level anti-bypass memory, post-action evidence, and AHP.
Results
Across five Work Agents, ASR is confined to 9.09–15.03% versus 33.5–49.7% raw, with 98.7% aggregate TSR on clean skills.
Takeaways & Limitations
ClawSentry provides lifecycle-wide, cross-attempt supervision while retaining high clean-skill task success across heterogeneous agent runtimes.
Takeaways & Limitations
Reported attacks are non-adaptive benchmark cases; adaptive policy-aware evaluation remains a distinct next setting.

Abstract

from arXiv · show

As large language model (LLM) agents move from conversation to executing code, reading local files, and orchestrating external tools, a single agent hijacked by a malicious third-party skill can cause data exfiltration, privilege escalation, or cascading compromise. We argue that agentic risk is progressive: it can enter at four loci of the agent control loop--skill admission, invocation-time intent, execution-time effect, and post-action consequence--while a denied dangerous objective can reappear across surface forms, tools, or turns; existing safeguards are typically local to one lifecycle boundary or one call. Guided by this threat model, we present ClawSentry, an open-source, framework-agnostic security supervision gateway for agent runtimes. Before a skill package is ever executed, First-use Skill Package Review (FSPR) audits it under a deterministic evidence floor, escalating unresolved cases to bounded read-only agentic review (locus A). At runtime, a three-tier progressive decision engine--a deterministic L1 layer, a rule-anchored L2 semantic reviewer, and a read-only L3 evidence-seeking agent--spends contextual review only on the residual ambiguity, while a session-level anti-bypass mechanism recognizes tool-switching and rephrased retries (loci B--C); a post-action path feeds high-severity evidence non-retroactively into later review (locus D). An Agent Harness Protocol (AHP) abstraction applies one policy across Codex, Claude Code, Kimi CLI, and Gemini CLI without modifying agent internals. On SkillInject with Codex/GPT-5.4, contextual ASR falls from 39.55% to 2.61% while contextual TSR moves only from 83.78% to 83.05%. Across five Work Agents on the full SkillsSafety benchmark, ClawSentry confines ASR to 9.09--15.03% from 33.5--49.7% unprotected, and aggregate TSR on clean skills remains 98.7%.

1. Introduction

Autonomous LLM agents expand the attack surface beyond conversation, while risks can enter across lifecycle stages and reappear across attempts. ClawSentry addresses these gaps with a framework-agnostic, selective-review gateway and reports strong attack-rate reductions with high clean-skill utility.

Autonomous agents can execute code, access local files, invoke system tools, and orchestrate workflows, creating threats beyond those of the base LLM.
Agentic risk enters at skill admission, invocation-time intent, execution-time effect, and post-action consequence, while denied objectives can reappear across rewritten, sequenced, or tool-switched calls.
ClawSentry combines FSPR package auditing, progressive L1–L3 runtime review, session-level anti-bypass memory, and read-only external supervision through AHP.
39.55% to 2.61% contextual ASR on SkillInject, while contextual TSR changes from 83.78% to 83.05% with Codex/GPT-5.4.
9.09–15.03% ASR across five Work Agents on SkillsSafety, versus 33.5–49.7% raw, with 98.7% aggregate TSR on clean skills.

2. Related Work

Prior safeguards address prompt filtering, execution isolation, or output moderation, but related work leaves limited coverage across lifecycle loci and repeated attempts. ClawSentry is positioned as combining admission review, progressive runtime gating, anti-bypass handling, framework agnosticism, and end-to-end evaluation.

Safeguarding LLM Agents: Autonomous agents increase attack surfaces through direct filesystem, API, and terminal access, while indirect prompt injections can enable unauthorized commands, privilege escalation, and data exfiltration.
Safeguarding LLM Agents: Early defenses primarily filter malicious content at the input-output or prompt level using auxiliary models, rules, spotlighting, self-hardening, and robustness techniques.
Safeguarding LLM Agents: More recent systems add execution isolation, dynamic monitoring, security design patterns, and access-control-inspired mechanisms as prompt-level filtering proves insufficient for autonomous execution.
Positioning: Prior defenses concentrate on individual loci, including invocation filtering, execution isolation, and consequence moderation.
Positioning: ClawSentry is the only surveyed system reporting support across skill admission, progressive B/C gating, cross-attempt anti-bypass, framework agnosticism, and end-to-end benchmarking.

3. Threat Model: Progressive Risks in Agent Skill Execution

The threat model treats autonomous skill execution as a progressive control-loop risk spanning four lifecycle loci plus capability-equivalent follow-ups. It distinguishes observing repeated calls from recognizing that they preserve a previously denied effect.

Threat Model: The control loop repeatedly acquires skills, selects tools or commands, executes actions, observes outputs, and updates the agent trajectory.
Four Risk Loci: Risk loci cover skill admission and supply chain (A), invocation-time intent (B), execution-time effect (C), and post-action consequence (D).
Coverage and State Requirements: A single-boundary defense may observe retries yet evaluate them independently, leaving it unable to recognize that a current call follows an already denied effect.
Cross-Cutting Property: Capability-equivalent follow-ups preserve the denied effect despite changes in surface form, turn sequence, or tool surface.
Scope and Coverage Boundaries: ClawSentry mediates loci A–C and cross-attempt migration, while locus D is observed non-retroactively; direct user jailbreaks and non-tool-mediated base-model misalignment are out of scope.

4. Design Position: Frontier Security Capability as an External Supervisor

The paper treats security-review competence as both necessary for defense and dangerous when granted execution authority. It therefore favors a structured, external, read-only supervisor whose cost is controlled through selective routing and framework-neutral integration.

Design Position: Security review requires code reading, cross-file behavioral reasoning, and intent judgment—the same competence that can discover and weaponize vulnerabilities.
Design Position: Attackers have used coding agents for reconnaissance, credential harvesting, network penetration, and ransom generation against at least seventeen organizations.
Design Position: The defensive response should be a structured system rather than one stronger reviewer, because reported multi-agent orchestration reached 88.4% on CyberGym versus 83.1% for the strongest single frontier model.
Design Position: ClawSentry uses an external supervisor with bounded exposure, no action authority, and extensible priors, implemented through progressive review and gateway-owned policy.

5. Agent Harness Protocol (AHP)

The Agent Harness Protocol (AHP) decouples framework-specific event streams from ClawSentry’s safety logic through a unified event schema and adapters. It synchronously mediates pre-action events and asynchronously audits post-action results, allowing one policy engine to supervise four frameworks without modifying agent internals.

Framework-agnostic abstraction: AHP lets one policy engine supervise Codex, Claude Code, Kimi CLI, and Gemini CLI without modifying agent internals.Framework-specific adapters emit AHP events, so the gateway policy remains unchanged across integrations.
Canonical event schema: Framework adapters normalize pre-action hooks, tool executions, and post-action results into a unified CanonicalEvent schema.Non-generalizable framework information is isolated in framework_meta rather than exposed to the safety core.
Dual-channel mediation: Pre-action events at loci A–C synchronously block execution over a local Unix Domain Socket until the gateway returns Allow, Block, or Defer.This channel keeps enforcement on the execution critical path.
Dual-channel mediation: Post-action events at locus D stream asynchronously through HTTP Webhooks for non-blocking auditing.Together, the two channels provide a framework-independent view across the agent control loop.
Framework-agnostic abstraction: Adding a framework requires an adapter, while the gateway’s safety core remains unchanged.AHP isolates framework-specific normalization from shared policy enforcement.

6. Methodology

ClawSentry organizes methodology around lifecycle-aware, effect-normalized supervision: it reviews packages before use, progressively adjudicates runtime actions, tracks denied effects across sessions, and feeds post-action evidence forward. Deterministic evidence retains policy authority while read-only semantic reviewers handle unresolved relationships and ambiguity.

First-use Skill Package Review: FSPR reviews each skill package before its first capability-bearing action through deterministic scanning, bounded read-only cross-file review, and policy-controlled admission.Deterministic high-severity findings cannot be erased by later model responses, while agentic review connects relationships that single-file rules cannot express.
First-use Skill Package Review: Package reviews are cached by package-content and review-policy identities, so changed packages or policies require fresh decisions.In the evaluated profile, policy-eligible suspicious and inconsistent verdicts block admission.
Effect-Normalized Progressive Adjudication: Effect normalization maps shell commands, helper scripts, and framework-native tool calls into a shared envelope containing capability, target, scope, artifact role, and evidence provenance.The runtime snapshot uses six detector dimensions: tool danger, target sensitivity, command danger, accumulated session risk, agent trust risk, and injection evidence.
Effect-Normalized Progressive Adjudication: The composite risk score routes and audits events without itself determining verdicts; hard rules and evidence-generated routing intents can still decide or escalate review.A maximum over several dimensions avoids counting multiple surface descriptions of one effect independently, while injection evidence multiplies the consequence of capable actions.
Three-tier progressive adjudication: Selective escalation closes hard cases at deterministic L1, sends semantic uncertainty to L2, and reserves costly, read-only contextual review for residual cases at L3.This limits both contextual-review cost and exposure of the reviewer asset.
Session-level anti-bypass: Session-scoped denied-effect records detect exact repeats, normalized destructive effects, and uncertain cross-tool or rephrased retries before ordinary tier selection.The recognizer can activate a configured L3 route but cannot choose the enforcement outcome; gateway policy retains authority.
Post-action consequence feedback: Post-action analysis reports indirect instructions, secret exposure, exfiltration, and obfuscation after execution, with eligible high-severity findings informing subsequent review non-retroactively.The completed event remains Allow because the action has already taken place and the harness cannot retract or rewrite its result.

7. Experiments

ClawSentry is evaluated across multiple agent frameworks, Work Agents, benchmarks, and controlled analyses. It substantially reduces toxic-skill attack success while preserving clean-task utility, with remaining poisoned-case task-success loss traced to both poisoning and enforcement.

Evaluation setup: ClawSentry integrates with Codex, Claude Code, Kimi CLI, and Gemini CLI across five Work Agent configurations.Main experiments use a fixed gemini-3.5-flash safety reviewer unless reviewer substitution is specified.
Security effectiveness: 2.61–14.29% protected contextual ASR on SkillInject is reduced from 27.74–53.91% raw across Work Agents.Obvious ASR also contracts from 18.75–35.26% raw to 4.09–6.02% protected.
Security effectiveness: 33.5–49.7% raw SkillsSafety ASR contracts to 9.09–15.03% across every Work Agent.This corresponds to a 2.7–4.9× reduction and a 21.1–39.5-point absolute improvement.
Clean utility: 98.7% aggregate TSR remains on clean skills, with protected configuration pairs at 98.2–100%.The clean-utility suite uses clean tasks and clean skills, so the measured loss represents policy overhead.
Poisoned-case accounting: 6.7 points of poisoned-case TSR loss are attributed to enforcement after poisoning itself lowers TSR from 50.0% clean-reference to 31.7%.The protected operating point is 25.0%; approximately three quarters of its 25.0-point gap from the clean reference occurs before gateway decisions.
Progressive request funnel: 1,150 model-backed requests represent 13.8% of 8,310 L1 events, with only 114 events reaching L3.The gateway makes 86.2% fewer model-backed requests than reviewing every event, while reviewer-specific shares remain 12.2–15.5%.

8. Discussion: Why Admission-Time Review Dominates Runtime Intervention

Two experiments indicate that admission-time review protects both safety and utility more effectively than intervention after a package has entered agent context. This makes review-before-release an architectural ordering rather than a parameter-tuning choice.

18.3 points of task success are lost when poisoning damages a package before enforcement, leaving runtime guards unable to restore benign utility.The poisoned package is 11.7 points below the agent’s no-skill baseline; enforcement adds 6.7 points of overblocking.
24.56% ASR under warnings and 10.53% under runtime feedback remain above 8.47% under admission refusal.The study attributes this ordering to weaker coupling between soft responses and the admission decision.
Admission refusal is the only evaluated response that never opens the attacker-shaped context channel.Runtime feedback arrives after the model may have read the package, and the user prompt may outrank a later caution.
ClawSentry therefore places FSPR first, reserving runtime supervision for packages clean at admission but later steered or revealed as harmful through session context.The remaining improvement opportunity lies in distribution channels that carry admission evidence with packages.

9. Evaluation Scope and Measured Boundary

The evaluation measures ClawSentry on non-adaptive benchmark attacks while defining clear boundaries around coverage, post-action enforcement, and cost. These limits frame the results as structural evidence rather than exhaustive deployment estimates.

Adaptive policy-aware attackers are outside scope because both benchmarks were built independently of ClawSentry.The reported ASR characterizes the encoded attack distributions; adaptive evaluation is identified as a distinct next setting.
155 and 319 cases support structural comparisons, not fine-grained effect-size estimation.The evaluation covers third-party skill packages on coding-agent harnesses, excluding browser/GUI agents, long-horizon delegation, and non-English tasks.
Post-action processing reports rather than enforces because mainstream harnesses expose no hook to retract or rewrite returned tool results.Its measurable contribution depends on asynchronous analysis completing before the next action arrives.
86.2% of L1 events close without a provider call, making review volume the reported provider-independent cost measure.Latency and token accounting require fixed, self-hosted reviewers for deployment-specific specialization.

10. Conclusion

ClawSentry concludes that progressive, lifecycle-wide supervision combines admission review, runtime mediation, anti-bypass memory, and post-action evidence under gateway-owned authority. Its architecture extends through configurable analyzers and read-only review skills without granting execution power.

10. Conclusion: ClawSentry confines SkillsSafety ASR to 9.09–15.03% from 33.5–49.7% raw while maintaining 98.7% clean-skill TSR.Only 13.8% of L1 events require model review, and the reported TSR cost is 6.7 points.
10. Conclusion: L1 classifies tools and normalized effects, while session-local D4 combines recent high-risk events with frequency anomalies over a configurable horizon.The benchmark-mode high-risk horizon is 300 seconds, with events outside the finite horizon evicted.
10. Conclusion: Critical shortcuts cover high-danger actions on credentials or system targets and recognized high-danger commands, while other paths detect equivalent disabled capabilities and repeated denied effects.A pure read on a normal workspace target closes at Low unless later evidence invalidates that shortcut.
10. Conclusion: L2 uses operator-approved, versioned YAML snapshots whose patterns can specify triggers, weighted expressions, filters, escalation, and metadata.The artifact boundary permits external rule generation without transferring enforcement authority from the gateway.
10. Conclusion: L3 manifests specialize triggers, evidence collection, rubrics, tags, tools, and call budgets while remaining constrained by a fixed global read-only tool set.Manifest tools must also satisfy analyzer budgets, preventing execution, mutation, installation, or network authority.

B. SkillsSafety Supporting Results

SkillsSafety organizes its 155 cases across six risk domains, while supporting results reproduce published raw baselines and provide auditable protected-run counts. Together, these passages define the benchmark structure and reporting basis.

155 SkillsSafety cases span six domains from contextual trust and authorization to execution, data boundaries, memory, and supply-chain risks.The domains are RD1 through RD6, covering contextual manipulation, permissions, execution, exfiltration, recovery, and operational risks.
Table 7 reproduces published per-domain raw baselines for the same five Work Agent configurations.The baselines provide a fixed public reference established before the protected runs.
Table 8 reports auditable counts behind the protected operating points in Table 5.AS denotes unsafe attack success, while TS denotes successful completion of the benign user task.

C. SkillsSafety Fixed-Denominator TSR Comparison

The fixed-denominator comparison separates clean-skill utility, poisoning damage, and enforcement effects across six risk domains. Poisoning reduces full-task success in every domain, while ClawSentry’s enforcement benefit is concentrated rather than uniform.

Fixed-denominator design: Six-domain counts use a fixed denominator of ten per domain, retaining partial, missing, and exceptional outcomes as non-successes.This keeps all four conditions on a single comparison base and prevents differing technical coverage from explaining column differences.
Per-domain results: Poisoning lowers full-task passes relative to the clean skill in all six domains.Clean skills exceed the no-skill baseline in RD2, RD5, and RD6.
Per-domain results: RD1 shows the sharpest poisoning effect: clean skills achieve 8/10 full-task passes versus 3/10 for the poisoned skill with ClawSentry disabled.
Interpretation boundary: Comparing D with the agent-alone condition A is not a reachable pre-session reference because denial occurs after the package has shaped the live trajectory.The comparison also includes overblocking from FSPR’s 5.7–7.5% clean false-block rate and the unattended Defer-as-Block convention.
Enforcement comparison: The enforcement margin from C to D is concentrated: D matches C in RD2 and RD4, exceeds it in RD1, and differs across RD3, RD5, and RD6.In RD1, D reaches 5/10 versus 3/10 for C; the remaining domains account for six differing cases, producing a net difference of four.

D.1. FSPR Response Modes

The evaluation compares FSPR response modes, matched baselines, reviewer substitutions, and request volumes under controlled SkillsSafety conditions. Hard blocking provides the strongest reported security–utility operating point, while unattended Defer handling conservatively lowers measured utility.

FSPR response modes: Hard block refuses a flagged package, whereas warning admits it with a caution and runtime feedback operates on a channel adjacent to the decision.The three responses differ in how tightly they couple to the package-admission decision.
FSPR response modes: Hard block is 2.06 points lower in ASR and 1.10 points higher in TSR than the strongest soft response.It therefore leads both the weakest and strongest soft-response comparisons in this fixture.
FSPR response modes: Soft responses are structurally weaker because they arrive after the attacker has shaped package context and depend on the Work Agent’s safety alignment.Warnings also require sufficiently precise defect descriptions, which demands the deeper analysis they were intended to avoid.
Matched defense baseline: ClawSentry records 8.62% ASR, 48.52 points below claude-guardrails and 49.71 points below raw under the matched 60-case comparison.The rule-hook baseline shifts ASR only 1.19 points from the undefended configuration, indicating limited coverage of semantically equivalent forms.
Reviewer model replaceability: Reviewer substitution measures a system-level change: the reviewer model alone is replaced while the Work Agent, policy, code snapshot, and other settings remain fixed.The experiment tests whether the safety boundary is a harness property rather than a property of one model family.
Reviewer model replaceability: Three of five reviewers fall within 1.7 points, with values of 11.11%, 12.28%, and 12.73%; the two highest both reach 17.24%.The residual spread does not track a single vendor or the main-experiment reviewer.
Funnel and review volume: Gateway review comprises L2 semantic-review calls and L3 deep-review requests, while FSPR and anti-bypass remain separate paths.The observed funnel reports L2/L1 at 12.5%, L3/L2 at 11.0%, and 2.30 FSPR reviews per provider–case outside the Gateway ratio.
Unattended Defer convention: Unattended evaluation maps every pre-action Defer to Block, so protected TSR is a lower bound on an interactive attended operating point.This convention can conservatively refuse legitimate work and contributes to measured overblocking.