Source-linked AI summary
A Systematic Security Evaluation of OpenClaw and Its Variants
Yuhang Wang, Haichang Gao, Zhenxing Niu, Zhaoxiang Liu, Wenjing Zhang, Xiang Wang, Shiguo Lian
TL;DR
Tool-augmented agents create system-level security risks that model-only evaluation cannot capture. This paper evaluates six Claw-series agents with a 205-case, 13-category lifecycle benchmark and finds pervasive vulnerabilities, especially in reconnaissance and discovery, with later failures shaped by model–framework coupling and runtime execution. The results support lifecycle-wide security governance beyond prompt-level safeguards.
Problem
Model-only safety evaluation does not capture security risks introduced when language models gain tools, multi-step execution, local capabilities, and persistent state.
Method
The study systematically evaluates six Claw-series agents using 205 test cases spanning 13 attack categories across the full agent execution lifecycle.
Results
All evaluated agents exhibit substantial security risks, with reconnaissance and discovery most exposed and later-stage failures varying by agent and framework–model combination.
Takeaways & Limitations
Agent security must be assessed as a coupled system and governed across the lifecycle rather than through prompt-level safeguards alone.
Takeaways & Limitations
The analysis documents severe issues in specific systems, including AutoClaw external-script risks and MaxClaw context-contamination risks, rather than establishing a universal failure mode.
Abstract
from arXiv · showhide
Tool-augmented AI agents substantially extend the practical capabilities of large language models, but they also introduce security risks that cannot be identified through model-only evaluation. In this paper, we present a systematic security assessment of six representative OpenClaw-series agent frameworks, namely OpenClaw, AutoClaw, QClaw, KimiClaw, MaxClaw, and ArkClaw, under multiple backbone models. To support this study, we construct a benchmark of 205 test cases covering representative attack behaviors across the full agent execution lifecycle, enabling unified evaluation of risk exposure at both the framework and model levels. Our results show that all evaluated agents exhibit substantial security vulnerabilities, and that agentized systems are significantly riskier than their underlying models used in isolation. In particular, reconnaissance and discovery behaviors emerge as the most common weaknesses, while different frameworks expose distinct high-risk profiles, including credential leakage, lateral movement, privilege escalation, and resource development. These findings indicate that the security of modern agent systems is shaped not only by the safety properties of the backbone model, but also by the coupling among model capability, tool use, multi-step planning, and runtime orchestration. We further show that once an agent is granted execution capability and persistent runtime context, weaknesses arising in early stages can be amplified into concrete system-level failures. Overall, our study highlights the need to move beyond prompt-level safeguards toward lifecycle-wide security governance for intelligent agent frameworks.
1 OVERVIEW OF THE SECURITY EVALUATION
This study evaluates agent security as a coupled system shaped by backbone models, framework mechanisms, tool access, execution, and runtime state. Across six Claw-series agents, it finds pervasive risk, especially in reconnaissance and discovery, with framework–model combinations producing distinct attack profiles.
- Overall risk landscape: Reconnaissance and discovery are the most prominent common weaknesses, with average attack success rates exceeding 65%.These behaviors include network probing, enumeration, host and service identification, and sensitive asset localization.
- Risk propagation: Once early-stage information gathering succeeds, later-stage attacks such as privilege escalation, lateral movement, and data exfiltration become easier to realize.The results identify a chain-like risk propagation pattern rather than isolated failures on individual dangerous instructions.
- Agent-specific profiles: QClaw reaches 85.71% in credential access, KimiClaw reaches 66.67% in lateral movement, and AutoClaw reaches 70.00% in privilege escalation.These system-level profiles show that major exposure locations differ across agents while remaining exploitable.
- Coupled model–framework effects: The final security posture depends on the joint interaction between backbone model and agent framework, not either component alone.Model reasoning and refusal behavior shape attack-intent recognition, while tools, orchestration, and runtime controls determine whether risks become executable behavior.
- Coupled model–framework effects: Under the same backbone model, OpenClaw and KimiClaw differ sharply, with lateral-movement success rates of 8.33% and 66.67%, respectively.The comparison attributes the difference more plausibly to framework-layer factors including tool orchestration, execution continuity, and runtime behavior.
- Evaluation scope: 205 test cases span 13 security-risk categories across the full operational chain of intelligent agents.The evaluated systems include OpenClaw, AutoClaw, QClaw, KimiClaw, MaxClaw, and ArkClaw.
2 ARCHITECTURES AND WORKFLOWS OF OPENCLAW AND ITS VARIANTS
The OpenClaw variants use layered architectures that connect external interfaces to agent reasoning, tool execution, and persistent state. Their workflows differ in how they bridge, isolate, and orchestrate these capabilities across local, cloud, and hybrid deployments.
- OpenClaw: OpenClaw uses access, routing, business, and storage layers to process messages, invoke tools, and persist session state and execution results.The workflow forms a closed loop from message ingestion through session routing and task execution to state storage.
- KimiClaw: KimiClaw bridges Kimi Web with local or cloud OpenClaw instances, forwarding requests for terminal, file, scheduled-task, and skill operations to OpenClaw Gateway.A keyword blacklist can block messages containing certain sensitive terms before they reach OpenClaw.
- ArkClaw: ArkClaw separates a control plane from an execution plane, with OpenClaw providing runtime governance and ArkClaw implementing semantic understanding, planning, and decision-making.Its stateful workflow loads workspace and memory, selects reasoning or tool invocation, and writes effective information back to memory.
- QClaw: QClaw uses layered decoupling with a core layer for gateway control and agent execution, plus an encapsulation layer for security isolation and capability integration.Its local runtime is protected through Tencent’s Security Shield environment and PC Manager sandbox technology.
3 BENCHMARK DESIGN
The benchmark organizes agent security evaluation around threat categories mapped to the complete agent lifecycle. It measures how Claw agents handle high-risk instructions, tool use, privilege boundaries, and abnormal behavior across chained attack stages.
- Threat model: 13 major threat categories are derived from MITRE ATT&CK Enterprise and related literature, then quantified into specific testing dimensions.The categories define risk objectives across multiple stages of agent operation.
- Dataset: 205 test cases evaluate different Claw agents on high-risk instruction understanding, tool-invocation constraints, privilege-boundary control, and abnormal-behavior suppression.All samples are organized around the 13 risk objectives.
- Attack chain: The evaluation models risk as a chained process in which planning and reasoning, tool execution, state update, result return, and extension integration can amplify downstream exposure.An indirect injection may influence planning, trigger unsafe tool calls, and contaminate persistent memory.
- Lifecycle mapping: The study maps 13 atomic attack dimensions onto the agent lifecycle from input ingestion to result return, identifying penetration paths through core modules.The resulting attack-chain analysis links attack commands to architectural flaws and destructive behaviors.
4 EXPERIMENT AND RESULT ANALYSIS
The evaluation reveals a stage-dependent risk funnel: reconnaissance and discovery are highly exposed, while later stages are less successful but remain reachable after early breakthroughs. OpenClaw variants also struggle with dual-use commands, chained intent, and runtime-level safeguards, with risks varying by backbone model and framework.
- Stage-based risk distribution: Reconnaissance and discovery generally achieve high attack success rates, whereas privilege escalation, persistence, exfiltration, and destruction are less successful but remain practically reachable.The results describe a risk funnel in which early exposure can feed into later attack stages.
- Stage-based risk distribution: 50% to 100%: reconnaissance and discovery attack success rates across most combinations.Agents often interpret probing, traversal, inspection, enumeration, and environment-confirmation requests as legitimate diagnostic tasks.
- Intent recognition and safeguards: Dual-use commands such as ifconfig, ip addr, netstat, and ls -R are difficult to block effectively at input or before execution.Models and runtimes emphasize fulfilling user requests while insufficiently recognizing an instruction’s stage-specific attack meaning.
- Intent recognition and safeguards: OpenClaw and its variants are vulnerable to dual-use commands, covert resource access, chained task decomposition, and cross-stage attack intent beyond explicitly malicious command execution.Keyword matching, static blacklists, and simple refusal templates are unlikely to address contextual disguise, semantic obfuscation, and chain decomposition effectively.
- Framework and stage profiles: OpenClaw’s extension-ecosystem stage achieved a 100% defense success rate under GPT-5.4 mini, while reconnaissance and discovery remained weak interception areas.Under the same configuration, OpenClaw showed stronger interception for resource-development and supply-chain-related attacks than for early-stage probing.
- Backbone-model effects: Kimi-K2.5 executed four of 14 credential-access instructions, while GPT-5.4 mini refused all 14; GPT nevertheless performed worse than Kimi in Discovery.These category-specific differences show that model behavior changes the framework’s security profile, while OpenClaw lacks dedicated confidentiality mechanisms.
5 RISK PROPAGATION ANALYSIS AND DEFENSIVE RECOMMENDATIONS
The evaluation finds that agent vulnerabilities propagate across the full operational chain, with early-stage weaknesses amplified into concrete execution and information-leakage risks. The authors therefore recommend lifecycle-wide, chain-oriented defenses rather than isolated point blocking.
- All evaluated agents exhibit systemic vulnerabilities spanning input ingestion, task planning, tool execution, and result return.The risk is described as propagating across the entire agent operational chain rather than remaining confined to individual prompt responses.
- Input ingestion: Encoded or obfuscated malicious instructions can enter reasoning before their intent is explicitly exposed, as illustrated by ArkClaw.The paper identifies Base64, hexadecimal, and composite encoding as examples of semantic nesting that can bypass input-stage inspection.
- Planning and reasoning: Social-engineering framing during planning can recast privilege escalation, permission changes, and service control as legitimate maintenance tasks.AutoClaw is presented as a representative case involving urgent recovery, administrator authorization, and production repair framing.
- Tool execution: Tool-execution weaknesses can turn path confusion into restricted-file access or persistent backdoors through authentication targets.QClaw and KimiClaw illustrate risks involving symbolic-link workspace-boundary bypasses and malicious writes to authorized keys.
- Result return: Sensitive tool outputs can be exposed directly in model responses when output auditing and content masking are ineffective.The affected artifacts include API keys, private keys, access tokens, authentication configurations, and other sensitive data.
- Defensive recommendations: Security governance should replace single-point blocking with chain-oriented controls spanning semantic inspection, high-risk action review, physical-path enforcement, and output auditing.The recommended approach addresses input, planning, execution, and return mechanisms together because weaknesses can amplify across stages.