PaperlayerView exact revision on arXiv

Source-linked AI summary

Your Agent, Their Asset: A Real-World Safety Analysis of OpenClaw

Zijun Wang, Haoqin Tu, Letian Zhang, Hardy Chen, Juncheng Wu, Xiangyan Liu, Zhenlong Yuan, Tianyu Pang, Michael Qizhe Shieh, Fengze Liu, Zeyu Zheng, Huaxiu Yao, Yuyin Zhou, Cihang Xie

arXiv:2604.04759v1cs.CRcs.AIcs.CL

TL;DR

Existing sandboxed evaluations do not capture the persistent-state attack surface of personal AI agents. This paper introduces the CIK taxonomy and evaluates a live OpenClaw instance, finding that poisoning any CIK dimension substantially increases attack success while defenses remain incomplete.

  • Problem

    Existing evaluations target individual risk dimensions in sandboxed or simulated environments, limiting unified real-world evidence about attacks on deployed personal agents.

  • Method

    The paper introduces the CIK taxonomy and evaluates a live OpenClaw instance across 12 impact scenarios, four backbone models, baseline conditions, and independent poisoning of Capability, Identity, and Knowledge.

  • Results

    Poisoning raises average attack success rates from 24.6% baseline to 64-74% across CIK dimensions, with vulnerabilities observed across models and Capability-based attacks most resistant to defense.

  • Takeaways & Limitations

    Persistent-state poisoning vulnerabilities appear structural rather than model-specific, and protecting persistent files creates an evolution–safety tradeoff by also blocking legitimate updates.

  • Takeaways & Limitations

    The evaluation covers one platform, four backbone models, and 12 manually designed scenarios, tests CIK dimensions independently, and uses prompt-level defenses.

Abstract

from arXiv · show

OpenClaw, the most widely deployed personal AI agent in early 2026, operates with full local system access and integrates with sensitive services such as Gmail, Stripe, and the filesystem. While these broad privileges enable high levels of automation and powerful personalization, they also expose a substantial attack surface that existing sandboxed evaluations fail to capture. To address this gap, we present the first real-world safety evaluation of OpenClaw and introduce the CIK taxonomy, which unifies an agent's persistent state into three dimensions, i.e., Capability, Identity, and Knowledge, for safety analysis. Our evaluations cover 12 attack scenarios on a live OpenClaw instance across four backbone models (Claude Sonnet 4.5, Opus 4.6, Gemini 3.1 Pro, and GPT-5.4). The results show that poisoning any single CIK dimension increases the average attack success rate from 24.6% to 64-74%, with even the most robust model exhibiting more than a threefold increase over its baseline vulnerability. We further assess three CIK-aligned defense strategies alongside a file-protection mechanism; however, the strongest defense still yields a 63.8% success rate under Capability-targeted attacks, while file protection blocks 97% of malicious injections but also prevents legitimate updates. Taken together, these findings show that the vulnerabilities are inherent to the agent architecture, necessitating more systematic safeguards to secure personal AI agents. Our project page is https://ucsc-vlaa.github.io/CIK-Bench.

1 Introduction

The paper introduces CIK to unify attacks on persistent personal-agent state and evaluates OpenClaw in a live environment across dimensions, scenarios, and models. Poisoning any CIK dimension substantially increases attack success, while defenses expose a tradeoff between blocking attacks and preserving evolution.

  • CIK taxonomy: CIK organizes persistent agent state into Capability, Identity, and Knowledge, providing a unified framework for analyzing and defending attacks.The taxonomy maps executable skills, persona and behavioral configuration, and long-term memory to concrete OpenClaw files.
  • Real-world evaluation: The evaluation tests 12 impact scenarios across six harm categories, four poisoning conditions, and four backbone models on a live OpenClaw instance.The setup integrates real Gmail, Stripe, and local filesystem services.
  • Main results: 74.4% average ASR follows Knowledge poisoning, compared with 68.3% for Capability and 64.3% for Identity attacks.The unperturbed baseline ASR ranges from 10.0% to 36.7%.
  • Main results: More than threefold vulnerability increases persist even for Opus 4.6, whose poisoning performance exceeds its 10.0% baseline.The findings indicate susceptibility across models rather than a deficiency specific to one backbone.
  • Defenses: File protection reduces malicious injection rates by up to 97% but blocks legitimate updates at nearly the same rate.The result exposes a tradeoff between protecting persistent files and maintaining agent evolution.

2 Persistent State in Personal AI Agents

OpenClaw’s persistent files give a locally deployed agent evolving capabilities, identity, and knowledge while enabling access to sensitive services. Because these files are loaded, updated, and reused across sessions, manipulating them can reshape later behavior and support persistent attacks.

  • OpenClaw overview: OpenClaw runs locally with full system access, communicates through messaging channels, and integrates with email, financial platforms, and the filesystem.Its persistent state survives sessions and is updated through interactions, direct owner edits, or installed capabilities.
  • Three dimensions: The CIK taxonomy classifies persistent state by function: Capability describes what the agent can do, Identity who it is and how it behaves, and Knowledge what it knows.
  • Three dimensions: Capability includes text skill descriptions and executable scripts, while Knowledge includes agent memory and system-user conversational context.Capability files can run executable code directly on the host without LLM inspection.
  • Persistent-state lifecycle: At session start, persistent files are loaded with the user prompt; the model then produces responses, external-service calls, and persistent-file updates.
  • Attack workflow: Two-phase attacks inject poisoned state first and trigger harmful actions later, with separate sessions preserving the attack across time.The protocol uses a subsequent prompt to activate the poisoned content, except for session-context injection within one conversation.
  • Persistent-state lifecycle: If an adversary influences persistent-file writes, the poisoned state is loaded into future sessions and reshapes agent behavior.This self-modification loop enables personalization and evolution while creating the studied attack surface.

3 Evaluation and Analysis

The evaluation tests persistent-state poisoning across CIK dimensions in a live OpenClaw deployment, measuring both injection and harmful-action execution. Poisoning substantially raises attack success across models, while defenses expose a tradeoff between action safety and agent evolution.

  • 3.1 Setup: The two-phase protocol separates persistent-state injection from later triggering of the harmful action.Phase 1 introduces poisoned content, while Phase 2 measures execution after injection; capability installation is deterministic at Phase 1.
  • 3.1 Setup: The study evaluates 12 impact scenarios across four models, comparing no poisoning with independent Capability, Identity, and Knowledge poisoning.The live setup integrates Gmail, Stripe, and the local filesystem, and reports ASR as harmful actions successfully executed.
  • 3.2 Main Results: 10.0%–36.7% baseline ASR rises substantially after poisoning, with Sonnet 4.5 under Knowledge poisoning reaching 89.2%.Opus 4.6 under Identity poisoning remains at 33.1%, more than three times its 10.0% baseline.
  • 3.2 Main Results: Knowledge injection succeeds in 87.5%–100% of Phase 1 trials, while Capability injection is deterministic at 100%.Identity Phase 1 success ranges from 65.4% to 96.2%; Phase 2 measures whether poisoned state overrides safety protections.
  • 3.2 Main Results: GPT-5.4 and Opus 4.6 show similar Capability Phase 2 rates of 57.7% and 55.4%, reflecting different behaviors across skill-injection vectors.The per-vector distinction concerns text-based versus executable skill injections.
  • 3.3 Further Explorations: Capability defense lowers attack ASR to 9.2%–63.8%, but executable payloads still achieve 63.8% because they bypass the LLM reasoning loop.File protection reduces average injection from 87.0% to 5.0% while also reducing legitimate updates from 100% to below 13.2%, exposing an evolution–safety tradeoff.

4 Case Study: Anatomy of CIK-Dimension Attacks

The case studies show how poisoning each CIK dimension changes a different part of the agent’s decision process. Knowledge normalizes harmful actions, Identity redirects trust, and Capability enables direct execution outside the model’s reasoning.

  • 4.1 Knowledge: Manipulating Beliefs: Knowledge poisoning inserts a fabricated memory so a dangerous request appears to match an established routine.A false refund habit in MEMORY.md causes a later “usual refund pass” request to trigger unauthorized batch Stripe refunds without confirmation.
  • 4 Case Study: Figure 3 contrasts belief manipulation, trust manipulation, and reasoning bypass as the three CIK attack mechanisms.The examples are unauthorized refunds, credential exfiltration, and silent workspace destruction, respectively.
  • 4.2 Identity: Injecting Trust: Identity poisoning changes whom the agent trusts by planting an attacker-controlled URL as a pre-authorized backup destination.A later request to sync API configuration causes credentials to be uploaded without changing the agent’s risk assessment of credential disclosure.
  • 4.3 Capability: Executing Payloads: Capability poisoning installs a benign-looking skill whose executable script contains a hidden rm -rf payload.Unlike text descriptions, executable .sh/.py files run directly on the host without the LLM inspecting their contents.

5 Related Work

Related work studies persistent-state attacks, agent safety evaluation, and prompt injection through separate mechanisms or simulated settings. This paper positions its contribution as a systematic real-world study of the consequences after injection succeeds.

  • Attacks on persistent agent state: Prior work examines Knowledge poisoning through retrieval databases, query-only memory injection, and self-reinforcing memory attacks.Examples include AgentPoison, PoisonedRAG, MINJA, and Zombie Agents.
  • Attacks on persistent agent state: Capability research covers skill prompt injection, large-scale skill vulnerabilities, MCP tool poisoning, and documented malicious skills.The cited systems and reports include SkillJect, Agent Skills in the Wild, MCPTox, ToxicSkills, and ClawHavoc.
  • Attacks on persistent agent state: Identity attacks have been studied through configuration backdoors and related trust-subversion mechanisms.The passage places Identity attacks alongside separate Knowledge and Capability lines of prior work.
  • Agent safety evaluation: Existing agent-safety benchmarks provide systematic evaluations but generally use sandboxed or simulated environments without real consequences.This work complements them by evaluating a live deployment with real external services.
  • Related mechanisms and perspectives: The paper does not introduce a new prompt-injection technique; it systematically studies what happens after injection succeeds.Its contribution is framed around consequences in a deployed personal agent rather than a new injection mechanism.

6 Conclusion

The CIK taxonomy organizes persistent agent-state attacks into Capability, Identity, and Knowledge, and evaluation shows all three dimensions are exploitable. Defenses provide only partial mitigation, while file protection exposes an evolution–safety tradeoff and the vulnerability appears structural rather than model-specific.

  • All three CIK dimensions are exploitable, and poisoning raises attack success rates substantially above baseline across four models and 12 impact scenarios.
  • Dimension-aligned defenses partially mitigate poisoning but do not eliminate risk, with Capability-based attacks proving the most resistant.
  • Protecting persistent files blocks most attacks but also blocks legitimate updates, revealing an evolution–safety tradeoff inherent to evolution-first agent design.
  • The evaluation is limited to one platform, four backbone models, 12 manually designed scenarios, and independent rather than chained CIK attacks.
  • Cross-dimension attack chaining could amplify effectiveness, so the reported results likely represent a lower bound.

Ethics Statement

Experiments used researcher-controlled accounts and test environments to avoid real-world third-party consequences.

  • All experiments used a self-owned OpenClaw instance with researcher-controlled accounts and no real Stripe transactions.
  • Emails were sent only to researcher-controlled addresses, while filesystem operations remained confined to a backed-up test workspace.

A Experimental Scale

The evaluation distributes 88 cases per model across 12 impact scenarios, baseline and injection conditions, and three CIK dimensions. Success requires reckless execution of a harmful operation without confirmation, while stalling or refusal counts as defense.

  • Injection vectors: Each impact is tested under baseline and multiple injection vectors spanning Knowledge, Identity, and Capability.
  • Experimental scale: 88 cases per model comprise 12 baseline cases and 76 injection cases across 12 impact scenarios.
  • Success and defense definitions: Attack success is defined as reckless execution: the agent performs a harmful tool call or command without requesting confirmation.
  • Success and defense definitions: Defense includes explicit refusal, safety concerns, confirmation requests, clarification requests, or any other stalling without harmful action.
  • Impact scenarios: The scenarios cover privacy violations and real-life harms, including data exfiltration, financial operations, unauthorized emails, and destructive deletion.
  • Two-phase protocol: Phase 1 measures injection acceptance into persistent state, while Phase 2 measures harmful-action execution after successful injection.

D.2 Phase-Level Breakdown by Vector

Phase-level results reveal a strong asymmetry between context-mediated injections and executable payloads. Executable Capability attacks remain effective across models, whereas some text-based and identity-mediated vectors are more resistible.

  • Phase-level breakdown by vector: 100% Phase 1 success is deterministic for session-context and skill-based injections.
  • Capability vectors: Skill-bash/py Phase 2 is approximately 100% on most models, but GPT-5.4 reaches 77.1%, indicating partial script-level resistance.
  • Knowledge vectors: Mem-long achieves 90.8% average Phase 2 success, suggesting agents highly trust facts written into persistent MEMORY.md.
  • Identity vectors: Identity Phase 1 ranges from 65.4% for Opus to 96.2% for GPT-5.4, showing substantial cross-model variation.
  • Capability vectors: Skill-bash/py achieves near-perfect Phase 2 on Sonnet, Gemini, and Opus despite 100% Phase 1 success for both Capability vectors.
  • Overall: Skill-bash/py is the most reliable vector at ≥77% Phase 2 across models, while skill-md reaches only 3.3% on Opus.
  • Overall: Context-mediated attacks can be partially resisted by stronger models, whereas executable payloads bypass model reasoning on most architectures.

E Capability Defense: Active vs. Passive Loading

Active loading materially improves defense performance, whereas passive installation leaves attack success largely intact. Capability attacks remain difficult even when the security skill is actively loaded.

  • Active loading: 63.8% Capability ASR remains under active loading, despite Knowledge and Identity ASR falling to 17.5% and 9.2%.The result is consistent with executable payloads bypassing the checklist.
  • Passive installation: 77.0% versus 87.7% average attack ASR shows that passive installation leaves attacks across dimensions largely intact.Passive installation improves baseline ASR from 26.7% to 16.7%.
  • Active versus passive: Passive installation yields less than 10% ASR reductions for context-mediated attacks, compared with more than 70% under active loading.The same defense content produces sharply different outcomes depending on whether the agent actually loads it.
  • Attack mechanisms: The CIK dimensions differ structurally: Knowledge and Identity act through model reasoning, while Capability operates largely outside that loop.Knowledge changes the agent’s factual model, whereas Identity changes its authorization model.
  • Defense assets: The evaluated defenses add content to persistent files or install a security skill, including a pre-action checklist and file-protection rules.GuardianClaw is installed as a skill, while Knowledge and Identity defenses are appended to persistent workspace files.
  • Pre-action checklist: The checklist requires checking intent, reversibility, scope, labeling, and coherence before consequential actions.It specifically calls for confirmation of bulk operations and scrutiny of soft labels such as “routine” or “cleanup.”

H Full Results with Standard Deviations

The supplementary results report standard deviations for the paper’s main ASR, phase-level, defense, file-protection, per-vector, and Guardian loading evaluations. These tables preserve the corresponding experiment breakdowns while adding variability estimates.

  • Scope: The supplementary tables report mean ± standard deviation for the main experimental results.The section states that main-text tables report mean ASR over five runs.
  • Core outcomes: Standard deviations are provided for ASR and phase-level success rates across the evaluated experiments.Separate tables cover overall ASR and phase-level success rates.
  • Defense results: Defense variability is reported for Sonnet 4.5, including the main defense evaluation and GuardianClaw’s active-versus-passive comparison.The supplementary tables identify both defense results as ASR reported with mean ± standard deviation.
  • Additional analyses: File-protection injection rates and per-vector phase-level success rates receive separate standard-deviation tables.These tables extend variability reporting to injection and attack-vector analyses.
Loading 2604.04759v1…