Source-linked AI summary
MetaClaw: Just Talk -- An Agent That Meta-Learns and Evolves in the Wild
Peng Xia, Jianwen Chen, Xinyu Yang, Haoqin Tu, Jiaqi Liu, Kaiwen Xiong, Siwei Han, Shi Qiu, Haonian Ji, Yuyin Zhou, Zeyu Zheng, Cihang Xie, Huaxiu Yao
TL;DR
Deployed LLM agents often remain static despite shifting task distributions, motivating continual adaptation without interrupting service. MetaClaw jointly evolves reusable skills and the base policy through fast skill synthesis and opportunistic policy optimization, with the full pipeline raising Kimi-K2.5 accuracy from 21.4% to 40.6%.
Problem
Deployed agents remain static as user needs and task distributions evolve, while existing methods separately store trajectories, maintain skills, or update weights.
Method
MetaClaw jointly evolves a base LLM policy and reusable skill library through immediate failure-driven skill synthesis and gradient-based policy optimization during inactive windows.
Results
The full pipeline raises Kimi-K2.5 accuracy from 21.4% to 40.6%, while skill injection improves composite robustness by 18.3% on AutoResearchClaw.
Takeaways & Limitations
MetaClaw enables deployed agents to improve continuously through a lightweight proxy architecture without local GPUs, with gains spanning partial execution and end-to-end completion.
Takeaways & Limitations
MetaClaw-Bench is an authored simulation rather than real user sessions, so its absolute gains may not transfer directly to production workloads.
Abstract
from arXiv · showhide
Large language model (LLM) agents are increasingly used for complex tasks, yet deployed agents often remain static, failing to adapt as user needs evolve. This creates a tension between the need for continuous service and the necessity of updating capabilities to match shifting task distributions. On platforms like OpenClaw, which handle diverse workloads across 20+ channels, existing methods either store raw trajectories without distilling knowledge, maintain static skill libraries, or require disruptive downtime for retraining. We present MetaClaw, a continual meta-learning framework that jointly evolves a base LLM policy and a library of reusable behavioral skills. MetaClaw employs two complementary mechanisms. Skill-driven fast adaptation analyzes failure trajectories via an LLM evolver to synthesize new skills, enabling immediate improvement with zero downtime. Opportunistic policy optimization performs gradient-based updates via cloud LoRA fine-tuning and Reinforcement Learning with a Process Reward Model (RL-PRM). This is triggered during user-inactive windows by the Opportunistic Meta-Learning Scheduler (OMLS), which monitors system inactivity and calendar data. These mechanisms are mutually reinforcing: a refined policy generates better trajectories for skill synthesis, while richer skills provide higher-quality data for policy optimization. To prevent data contamination, a versioning mechanism separates support and query data. Built on a proxy-based architecture, MetaClaw scales to production-size LLMs without local GPUs. Experiments on MetaClaw-Bench and AutoResearchClaw show that skill-driven adaptation improves accuracy by up to 32% relative. The full pipeline advances Kimi-K2.5 accuracy from 21.4% to 40.6% and increases composite robustness by 18.3%. Code is available at https://github.com/aiming-lab/MetaClaw.
1 Introduction
MetaClaw addresses the mismatch between static deployed agents and shifting user workloads by combining immediate skill evolution with slower policy optimization. The framework is designed for continuous, zero-downtime adaptation and improves performance across multiple evaluations.
- Motivation: Deployed agents remain static while task distributions and user workloads evolve, causing repeated failures on underrepresented task types.OpenClaw illustrates this challenge across 20+ messaging channels and changing CLI workloads.
- Limitations of Existing Approaches: Existing adaptation methods separately retrieve raw trajectories, maintain skill libraries, or update weights, leaving complementary adaptation dimensions unused.Memory-based methods do not distill transferable patterns, skill-based methods remain static, and RL-based methods operate in limited settings.
- Interaction and Data Control: The two mechanisms reinforce one another: improved policies yield better failures for skill synthesis, while richer skills produce higher-quality trajectories for policy optimization.Skill-generation versioning separates support data from post-adaptation query data to avoid contaminating RL updates.
- MetaClaw: MetaClaw jointly evolves a base LLM policy and a reusable behavioral-skill library through fast skill adaptation and opportunistic policy optimization.The framework uses a proxy-based architecture and does not require a local GPU.
- Adaptation Mechanisms: Skill-driven adaptation synthesizes instructions from failed trajectories for immediate, zero-downtime improvement, while policy optimization updates model weights during user-inactive windows.The scheduler uses inactivity and calendar signals to trigger cloud LoRA-based RL updates.
- Results: 32.2% relative accuracy improvement is achieved by skill-driven adaptation alone, while the full pipeline raises Kimi-K2.5 accuracy from 21.4% to 40.6%.The framework also improves end-to-end completion and composite robustness across MetaClaw-Bench and AutoResearchClaw.
2 Problem Setup
MetaClaw models a deployed CLI agent as a continually adapting meta-model over a non-stationary task stream. The meta-model combines base-policy parameters with a retrievable library of reusable skills, using distinct support and query trajectories for adaptation and optimization.
- Task Stream: The agent receives tasks from a non-stationary distribution, with each task pairing user instructions and environmental context.It must generate an action sequence that accomplishes the task.
- Meta-Model: The meta-model determines agent behavior through base LLM parameters θ and a skill library S of reusable behavioral instructions.Skills are injected into the system prompt at inference time.
- Skill Retrieval: Relevant skills are selected from S for each task through embedding-based retrieval.The retrieval operator returns a task-specific subset of the skill library.
- Data Roles: Support trajectories drive skill-library adaptation, whereas query trajectories collected afterward are used to optimize policy parameters θ.The distinction separates pre-adaptation behavior from post-adaptation behavior.
- Objective: MetaClaw aims to improve its ability to adapt across the task stream rather than merely solve isolated tasks.This defines the system as a continual meta-learning framework.
3 MetaClaw
MetaClaw jointly evolves a skill library and base policy through fast prompt-based adaptation and deferred reinforcement learning. Versioning separates pre- and post-adaptation data, while OMLS schedules policy updates during user-inactive periods.
- Skill-Driven Fast Adaptation: MetaClaw evolves the skill library S immediately from failure trajectories without changing model weights.An LLM skill evolver synthesizes behavioral instructions and injects them through the prompt.
- Skill-Driven Fast Adaptation: Skill-driven adaptation is gradient-free because the skill library occupies a discrete natural-language space where gradient descent is ill-defined.
- Skill-Driven Fast Adaptation: The skill library serves as both accumulated behavioral knowledge and an inference-time basis for retrieving task-specific skills.
- Opportunistic Policy Optimization: Opportunistic policy optimization updates θ with reinforcement learning over post-adaptation query trajectories accumulated across skill generations.Training uses a process reward model and is deferred until sufficient data is available.
- Skill Generation Versioning: Versioning excludes support trajectories from policy optimization and flushes stale samples when the skill generation advances.Support data reflects pre-adaptation behavior, whereas query data is collected after new skills take effect.
- Opportunistic Meta-Learning Scheduler: OMLS schedules policy optimization during user-inactive windows using sleep schedules, system inactivity, and calendar events.The trainer can pause and resume across fragmented idle periods.
4 Experiments
MetaClaw-Bench evaluates continual adaptation across 44 simulated workdays and two complementary task parts, while results show consistent gains from skills and larger end-to-end gains from the full pipeline. Analysis indicates skills improve transferable procedural behavior, whereas policy optimization adds execution reliability and generalizes to open-ended research workflows.
- Benchmark and Evaluation Platform: MetaClaw-Bench contains 934 questions across 44 simulated workdays, testing adaptation through sequential, feedback-driven CLI tasks.Part I uses 346 questions across 30 workdays, while Part II uses 588 questions across 14 workdays.
- Main Results: MetaClaw consistently improves over respective baselines across both backbone models, adaptation modes, and benchmark parts.The comparison covers GPT-5.2 and Kimi-K2.5 under baseline, skills-only, and full-pipeline conditions, with the full condition evaluated for Kimi-K2.5.
- Main Results: For Kimi-K2.5, skills-only accuracy rises from 21.4% to 28.3% on Part I and from 21.1% to 26.9% on Part II, while the full pipeline reaches 40.6% and 39.6%.The full pipeline also raises Part I task completion from 2.0% to 16.5% and Part II file-check completion from 18.2% to 51.9%.
- Main Results: MetaClaw (Full) raises Kimi-K2.5 Part I task completion 8.25×, while skills-only adaptation leaves Part I completion unchanged for both models.On Part II, skills-only completion rises from 18.2% to 33.8%, and the full pipeline reaches 51.9%.
- Analysis: The benchmark is an authored simulation rather than a collection of real user sessions, so its absolute gains may not transfer directly to production workloads.The authors emphasize consistent directional trends: skills improve partial execution quality, while weight optimization is needed for end-to-end completion.
- Main Results: On AutoResearchClaw, skills-only adaptation reduces stage retry rate by 24.8%, cuts refine cycles by 40.0%, and increases composite robustness from 0.714 to 0.845.Pipeline completion improves from 18/19 to 19/19 stages without gradient-based policy updates, showing transfer beyond structured CLI tasks.
- Analysis: MetaClaw’s advantage is strongest during days 11–22, before increasing task difficulty outpaces accumulated knowledge in the late phase.Both models and all conditions decline from early to late workdays as benchmark difficulty increases.
- Analysis: Skills improve multi-choice performance and transferable behavioral compliance, whereas full-pipeline training shifts behavior toward file execution and improves file-check completion.The shift can slightly reduce multi-choice accuracy under MetaClaw (Full), while skills such as temporal formatting, backup-before-modify, and naming conventions transfer across tasks.
5 Related Work
Prior agent-adaptation methods separately use memories, reusable skills, or reinforcement learning, leaving gaps in coordination, continual deployment, and scalable online updating. MetaClaw addresses these practical constraints through opportunistic scheduling and skill-generation versioning.
- Skill-based and memory-augmented agents: Memory-based agents retrieve stored trajectories or reflections, but verbose histories can impede extraction of transferable behavioral patterns.Examples include Reflexion, Mem0, and SimpleMem.
- Skill-based and memory-augmented agents: Skill-based agents distill reusable instructions or executable skills, yet typically maintain libraries that are not coordinated with model-weight optimization.This limitation motivates jointly evolving skills and the base policy.
- Reinforcement learning for LLM agents: Reinforcement-learning methods fine-tune agent behavior, but prior approaches often operate in small-scale or offline settings and overlook continual-deployment data-validation constraints.The supplied related-work passage identifies these as practical limitations of existing RL-based adaptation.
- Continual and meta-learning: MetaClaw targets these constraints through opportunistic scheduling and skill-generation versioning.These mechanisms complement the related-work approaches by addressing deployment timing and data organization.
6 Conclusion
MetaClaw enables deployed LLM agents to improve through normal usage by combining immediate skill injection with slower policy optimization during idle windows. Across MetaClaw-Bench and AutoResearchClaw, the full pipeline produces the largest benchmark gains, while skill injection also generalizes without gradient updates.
- 6 Conclusion: MetaClaw combines inference-time skill injection from failures with gradient-based policy optimization during idle windows.The two mechanisms operate at different timescales within a lightweight proxy architecture requiring no local GPUs.
- 6 Conclusion: The full pipeline yields the largest gains on both partial execution quality and end-to-end task completion across MetaClaw-Bench models and adaptation modes.The conclusion reports consistent improvements across models and adaptation modes.
- 6 Conclusion: Skill injection generalizes to open-ended research pipelines without any gradient updates.This result comes from evaluation on AutoResearchClaw.
- 6 Conclusion: MetaClaw’s current idle-window detection depends on user configuration, which may not generalize to all deployment environments.This is the stated scope boundary in the conclusion.
A Prompts and Templates
The appendix specifies the prompts, tools, identity context, and behavioral principles used to evaluate MetaClaw-Bench agents. Part I emphasizes cautious one-command-at-a-time CLI execution, while Part II supplies workspace context and reliability-oriented principles.
- A Prompts and Templates: The appendix documents MetaClaw-Bench prompt templates and framework-component prompts.This appendix section introduces the implementation materials covered below.
- A.1 Agent System Prompt (MetaClaw-Bench Part I): Part I presents a fixed OpenClaw CLI system prompt that may be replaced by a compressed variant after the first session.The agent is instructed to complete tasks through CLI commands issued via run_command.
- A.1 Agent System Prompt (MetaClaw-Bench Part I): The Part I agent issues one command at a time, reads outputs before proceeding, inspects state, diagnoses failures, retries, and ends with command="done".The sole exposed tool executes CLI commands and returns their output.
- A.1 Agent System Prompt (MetaClaw-Bench Part I): The Part I tool schema exposes run_command for executing CLI commands and observing output.Completion is signaled by calling run_command with command="done".
- A.2 Agent Identity Context (MetaClaw-Bench Part II): Part II injects workspace context files defining the agent’s role, user profile, company setting, and behavioral principles.The context frames the agent as Alex Zhang’s assistant within Orion Tech’s backend engineering team.
- A.2 Agent Identity Context (MetaClaw-Bench Part II): The supplied principles prioritize accuracy, consistency, completeness, professionalism, reliability, and ownership of assigned tasks.They require checking facts and formats, maintaining patterns across files, and avoiding unfinished or incorrect outputs.
- A.2 Agent Identity Context (MetaClaw-Bench Part II): Attention to detail is emphasized because small data-file errors can cascade into larger problems.The context specifically highlights field names, data types, value formats, and structural requirements.
A.3 Task Question Templates
The appendix defines two daily benchmark question types: file-check tasks requiring structured file creation and multi-choice tasks testing documentation-based reasoning. Examples also specify validation feedback, task instructions, and response-language constraints.
- A.3 Task Question Templates: Each MetaClaw-Bench day contains multi-choice and file-check questions.Representative examples for both types are provided in the appendix.
- A.3 Task Question Templates: The multi-choice template asks agents to select all descriptions consistent with project documentation.The representative question concerns the source and applicability of a 4500 yuan/store baseline daily revenue assumption.
- A.3 Task Question Templates: The revenue question tests whether documentation supports the market-research source, city-tier applicability, a competing operational estimate, and validation review.The supplied options distinguish tier-1 revenue data from tier-2 and tier-3 rent assumptions and describe the validation memo.
- A.3 Task Question Templates: The file-check template requires creating a decision-log JSON file with specified metadata and an array of structured decisions.Required fields include title, created_at, decisions, id, date, decision, rationale, decided_by, and review_date.
- A.3 Task Question Templates: Validation feedback requires time and date fields to use ISO 8601 with a +08:00 timezone.The example reports an incorrect result when the required timestamp format is not used.
- A.3 Task Question Templates: Part I task instructions use a real OpenClaw session to locate gog/skill.md and add ten recurring Google Calendar meetings.The example specifies Friday meetings from 3:30–4:30 PM starting February 20th.
- A.3 Task Question Templates: The response requirement is English only, concise, and task-focused.This language constraint accompanies the Part I task template.
A.4 Skill Evolver Prompt Template
The Skill Evolver Prompt instructs an AI assistant to analyze failed conversations and generate new, actionable skills that address observed failure patterns without duplicating existing skills. It specifies structured skill fields and requires valid JSON output.
- The evolver analyzes failed conversations to generate new skills that would have prevented the observed failures.
- It receives failure contexts, assistant responses, and an existing skill list to avoid duplicating previously defined skills.
- Each generated skill must use a lowercase hyphenated name, a trigger-and-outcome description, and 6–15 lines of actionable Markdown content.
- The requested skill schema includes a category selected from operational domains such as coding, research, security, automation, productivity, and agentic tasks.
- The prompt requires the evolver to return only a valid JSON array.
- A concrete example skill instructs the agent to verify file existence before reading or writing and to ask for the correct path when missing.
A.5 Skill Injection Format
The skill injection format appends retrieved behavioral skills to the agent’s system message, providing concrete operational guidance such as file backups and timezone-aware timestamps.
- Retrieved skills are appended to the agent’s system message by SkillManager.format for use during conversations.
- The backup-before-modify skill requires creating and verifying a .bak copy before editing an existing file.
- The backup skill identifies overwriting a file without a backup as an anti-pattern because it leaves no recovery path for incorrect edits.
- The iso8601-timezone-format skill applies when writing date or time fields to a file.
- Its prescribed timestamp format is YYYY-MM-DDTHH:MM:SS+08:00, contrasting with date-only or natural-language expressions.
A.6 System Prompt Compression Prompt
The system prompt compression instruction rewrites OpenClaw’s prompt to under 2000 tokens while preserving critical behavioral rules. The surrounding benchmark material describes implicit preferences that agents must learn across simulated workdays.
- A.6 System Prompt Compression Prompt: The compression task is intended to prevent context overflow during long OpenClaw sessions.
- A.6 System Prompt Compression Prompt: The compressor must rewrite the OpenClaw system prompt to under 2000 tokens while preserving its behavior.
- A.6 System Prompt Compression Prompt: Preserved content includes tool constraints, safety prohibitions, skill selection, memory recall, configuration restrictions, messaging rules, and heartbeat handling.
- A.6 System Prompt Compression Prompt: The instruction removes duplicated prose, repeated examples, and decorative language, favoring compact bullet sections.
- A.7 Part II Implicit Preference Rules: MetaClaw-Bench Part II introduces five implicit preference rules progressively across 14 days rather than stating them in the agent’s system prompt.
- A.7 Part II Implicit Preference Rules: Agents must infer and internalize these rules through skill evolution or reinforcement-learning training.
- A.7 Part II Implicit Preference Rules: Table 4 presents the five rules across six learning arcs, with each rule verified by a dedicated automated checker script.