Source-linked AI summary
Memento-Skills: Let Agents Design Agents
Huichi Zhou, Siyuan Guo, Anjie Liu, Zhongwei Yu, Ziqin Gong, Bowen Zhao, Zhixun Chen, Menglong Zhang, Yihang Chen, Jinsong Li, Runyu Yang, Qiangbin Liu, Xinlei Yu, Jianmin Zhou, Na Wang, Chunyang Sun, Jun Wang
TL;DR
Memento-Skills addresses adaptation in frozen LLM agents by using external memory and a Read–Write loop to acquire, refine, and reuse executable skills without parameter updates. Experiments report improved performance, including a 13.7 percentage-point gain over a Read-Write ablation on an unseen test set.
Problem
Frozen LLM agents require adaptation through prompts, context, or external memory, while random exploration makes finding useful skills impractical.
Method
Memento-Skills uses executable skills as external memory and a five-step Observe → Read → Act → Feedback → Write loop to autonomously acquire, refine, and reuse them without updating LLM parameters.
Results
66.0% overall accuracy versus 52.3% for the Read-Write ablation on the unseen test set, a 13.7 percentage-point gain attributed to the skill optimisation pipeline.
Takeaways & Limitations
Executable skills can bridge memory-based and skill-based learning, enabling a deployable agent that improves through deployment experience alone.
Takeaways & Limitations
The evaluation does not establish sandbox safety and calls for real trajectories and end-to-end execution success beyond synthetic queries and accuracy.
Abstract
from arXiv · showhide
We introduce \emph{Memento-Skills}, a generalist, continually-learnable LLM agent system that functions as an \emph{agent-designing agent}: it autonomously constructs, adapts, and improves task-specific agents through experience. The system is built on a memory-based reinforcement learning framework with \emph{stateful prompts}, where reusable skills (stored as structured markdown files) serve as persistent, evolving memory. These skills encode both behaviour and context, enabling the agent to carry forward knowledge across interactions. Starting from simple elementary skills (like Web search and terminal operations), the agent continually improves via the \emph{Read--Write Reflective Learning} mechanism introduced in \emph{Memento~2}~\cite{wang2025memento2}. In the \emph{read} phase, a behaviour-trainable skill router selects the most relevant skill conditioned on the current stateful prompt; in the \emph{write} phase, the agent updates and expands its skill library based on new experience. This closed-loop design enables \emph{continual learning without updating LLM parameters}, as all adaptation is realised through the evolution of externalised skills and prompts. Unlike prior approaches that rely on human-designed agents, Memento-Skills enables a generalist agent to \emph{design agents end-to-end} for new tasks. Through iterative skill generation and refinement, the system progressively improves its own capabilities. Experiments on the \emph{General AI Assistants} benchmark and \emph{Humanity's Last Exam} demonstrate sustained gains, achieving 26.2\% and 116.2\% relative improvements in overall accuracy, respectively. Code is available at https://github.com/Memento-Teams/Memento-Skills.
1 The Self-Evolving Agent Problem
Memento-Skills addresses the cost and statelessness of continual LLM adaptation by keeping model parameters fixed and evolving an external skill memory. Its Read–Write Reflective Learning loop retrieves or creates skills, executes them, and updates the library from feedback.
- Why Frozen LLMs Need External Memory: Parameter updates require large data and compute budgets, motivating deployment-time adaptation through external memory instead.The external skill memory accumulates experience while the underlying LLM parameters remain frozen.
- Why Frozen LLMs Need External Memory: The agent’s statelessness prevents it from learning from deployment experience, whereas episodic memory grows over time.SRDP augments the agent with memory that conditions retrieval and decision-making.
- The Self-Evolving Agent: The Read–Write loop retrieves or generates a skill, executes it, and reflectively updates the library based on the outcome.The system can increase a successful skill’s utility or rewrite skill contents after failure.
- From Theory to Configuration: Memento-Skills treats reusable skill folders containing code, prompts, and declarative specifications as the unit of external memory.Unlike raw trajectory logs, each skill artefact is reusable and executable.
- From Theory to Configuration: A behaviour-aligned contrastive router uses single-step offline RL to optimise skill selection for execution success rather than semantic similarity.This makes routing a KL-regularised policy over candidate skills.
- Empirical Validation: On GAIA and HLE, Memento-Skills improves test accuracy over the static Read-Write baseline by 13.7 and 20.8 percentage points, respectively.The results also report stronger cross-task transfer when the learned skill library aligns with benchmark domain structure.
2 Read–Write Reflective Learning
Memento-Skills treats executable skills as evolving external memory: the agent reads skills to act, then writes reflective updates from execution outcomes. Its behaviour-aligned router selects skills for execution success rather than semantic similarity, while iterative refinement expands and consolidates the skill library.
- 2 Read–Write Reflective Learning: Memento-Skills treats reusable skill folders containing code, prompts, and specifications as external memory, enabling continual learning without parameter updates.The agent alternates between retrieving skills for current goals and writing updates back from post-hoc reflection.
- 2 Read–Write Reflective Learning: After failure, a selector attributes responsibility to one skill and a rewriter adds targeted guardrails or alternative strategies while preserving generality.The diagnosis uses the full execution trace and judge rationale to assign credit at skill level.
- 2 Read–Write Reflective Learning: The Read–Write loop follows Observe → Read → Act → Feedback → Write, using execution feedback to improve future skill behaviour.Writing includes failure attribution and targeted file-level rewriting of prompts or code inside skills.
- 2 Read–Write Reflective Learning: Synthetic positive and hard-negative queries train routing to distinguish execution-relevant behaviour from merely shared terminology or semantic similarity.InfoNCE increases probability on rewarding positives and suppresses hard negatives under the routing softmax.
- 2 Read–Write Reflective Learning: The router casts skill selection as a one-step MDP in which a query is the state, a skill is the action, and reward indicates whether the skill is appropriate.The learned score is interpreted as a soft Q-function and converted into a Boltzmann policy.
3 Self-Evolving Evaluation
Memento-Skills evaluates self-evolving skill libraries through training retries, benchmark test accuracy, and library-growth analyses. Results show substantial GAIA gains, broader HLE skill coverage, and diminishing returns as learned skills increasingly cover the embedding space.
- Evaluation scope: Offline retrieval quality and end-to-end trajectory success cover different failure modes, while sandbox safety remains a separate axis requiring an isolation harness.The evaluation setting therefore treats retrieval and execution as complementary validations rather than sufficient evidence of production safety.
- GAIA results: GAIA training success rises from 65.1% initially to 91.6% after three reflective retries.On the unseen test set, Memento-Skills reaches 66.0% overall accuracy versus 52.3% for the Read-Write ablation, a 13.7-point gain.
- Transfer and limitations: GAIA shows limited cross-task transfer because its diverse questions share little overlap in the reasoning patterns required.Many skills optimized during training were not triggered during testing, suggesting transfer depends on domain alignment.
- Skill library growth: The skill library grows from 5 atomic skills to 41 after GAIA learning and 235 after HLE learning.The HLE library spans a wider embedding space and forms semantically coherent, domain-specific clusters.
- HLE results: HLE training accuracy increases from 30.8% at R0 to 54.5% at R3, with the steepest improvement occurring in the first round.Later rounds produce smaller gains as the reachable space becomes increasingly covered.
- Skill library growth: Reflective learning both refines existing skills and adds new skills, shrinking gaps between covered query regions.These forces explain large early gains and diminishing returns in later rounds.
4 Conclusion
Memento-Skills bridges memory-based and skill-based learning by treating executable skills as evolving external memory. Its reflective loop enables parameter-free continual learning and improves performance across GAIA and HLE.
- Memento-Skills treats executable skill folders as the unit of external memory for LLM agents.The skills contain code, prompts, and declarative specifications.
- Read–Write Reflective Learning lets the agent acquire, refine, and reuse skills from deployment experience without updating LLM parameters.The skill library is updated through reflective read and write operations.
- A behaviour-aligned contrastive router trained with single-step offline RL retrieves skills for execution success rather than surface-level similarity.
- Experiments on GAIA and HLE show that the skill-as-memory formulation substantially outperforms a static-library ablation.
- Cross-task transfer is strongest when skills are aligned with structured domain categories.
A Reading Path
The paper uses interleaving tracks for research and practitioner audiences, connected by shared dialogues and bridges. Readers can follow one path or both for the complete picture.
- Each section begins with a shared Dialogue before branching into Research and Practitioner tracks.
- Research track: The Research track covers formal SRDP setup, convergence proofs, and KL-regularised routing analysis.
- Practitioner track: The Practitioner track covers installation, API walkthroughs, the retrieval pipeline, and benchmark recipes.
- Bridge: Bridge sections connect theoretical results to engineering choices, while the Epilogue closes the narrative.
- Three characters—J, H, and S—annotate the text inline throughout.
B Characters
The paper presents three recurring characters with distinct professional perspectives: theory, experimentation, and production engineering.
- J is a tenured theorist who focuses on proofs and convergence.
- H is a second-year CS PhD student who runs many experiments and favors scaling with more GPUs.
- S is a senior ML engineer whose production experience makes her prioritize unit tests and operational reliability.
C Prompt for Synthetic Router Goals
The synthetic-router-goal prompt generates realistic user-style positive and hard-negative queries for a target skill. It requires plausible, concrete, non-duplicative English queries without directly naming the skill.
- The prompt defines a target skill through its name, description, and keywords.
- The router state is only a text goal, so generated queries must resemble realistic user requests.
- Positive queries should select the target skill, while hard negatives should share its domain but make the skill unsuitable or non-optimal.
- Hard negatives must be plausible, terminology-matched, relevant but useless for the target skill, and free of obvious exclusion cues.
- Queries must be concrete, actionable, non-trivial, varied in phrasing, English-only, and avoid directly mentioning the skill name.
- The prompt supplies accepted positive and negative queries so new generations avoid duplicates.
- The required output is JSON containing query lists with fit, relevance, and uselessness fields.