Source-linked AI summary
WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution
Liyan Tang, Cyrus Rashtchian, Chun-Sung Ferng, Andrew Tomkins, Da-Cheng Juan, Tu Vu
TL;DR
Prior skill-evolution methods do not maintain learned knowledge as a separate, evolving representation. WikiSkill addresses this by consolidating execution experience into a persistent wiki that guides skill updates. Across five benchmarks and five models, it outperforms existing skill-evolution methods and improves over no-skill baselines in most settings.
Problem
Preserving and organizing learning across skill evolution remains a key design question: prior methods do not maintain learned knowledge as a separate, evolving representation.
Source: §1 · p. 2Method
WikiSkill separates execution traces, accumulated knowledge, and procedural skills, continually consolidating experience into a persistent wiki that informs subsequent skill updates.
Source: §1 · p. 2Results
Across five benchmarks and five models, WikiSkill outperforms existing skill-evolution methods and improves over no-skill baselines in most model-dataset pairs.
Source: §7 · p. 14Takeaways & Limitations
Skills transferred from other models can outperform self-evolved skills, suggesting that skill discovery and execution are distinct capabilities.
Source: §1 · p. 3The evaluation directly injects active skills into prompts, leaving skill retrieval and triggering untested despite their importance as skill collections grow.
Source: Limitations · p. 14
Abstract
from arXiv · showhide
Agent skills package specialized knowledge and workflows into reusable resources that extend AI agent capabilities. Recent work automatically discovers such skills from agent experience, which enables agents to progressively adapt through interaction. However, the insights that guide skill development typically remain scattered across optimization histories, limiting their systematic reuse across iterations. We introduce WikiSkill, a framework that co-evolves agent skills with a persistent knowledge base (wiki). At a high level, WikiSkill separates raw execution experience, accumulated knowledge, and executable skills, while continuously consolidating experience into the wiki, which subsequent skill updates can build on. Across diverse benchmarks and models, WikiSkill consistently outperforms state-of-the-art skill-evolution methods and improves over no-skill baselines in most model-benchmark settings. We find that skill evolution complements model scaling: larger models generally benefit more from evolved skills, while smaller models with skills can outperform substantially larger models without them. We also find that evolved skills transfer effectively across models and model families, and skills evolved by other models can outperform self-evolved skills. Finally, our ablation studies confirm that persistent knowledge accumulation in the wiki is critical for effective skill evolution. These results demonstrate the benefits of systematically accumulating and refining agent experience for developing reusable and transferable skills.
1. Introduction
WikiSkill co-evolves agent skills with a persistent wiki that consolidates execution experience into knowledge for future updates. Across five benchmarks and five models, it outperforms existing skill-evolution methods, with ablations confirming the importance of persistent knowledge accumulation.
1. Introduction
- WikiSkill addresses both the difficulty of anticipating needed workflows during manual skill authoring and the fragmentation of knowledge across skill-evolution artifacts.Skills package instructions, scripts, and resources into reusable filesystem modules without updating model parameters. WikiSkill introduces a separate, evolving knowledge layer between raw experience and executable skills.Source: §1 · p. 2
- WikiSkill separates immutable execution traces, structured wiki knowledge, and evolving procedural skills, coordinating inference, wiki maintenance, skill proposals, and validation-based gating.The Skill Proposer uses both wiki knowledge and traces; gating retains updates that improve validation performance. Skills can be rolled back while the wiki persists for future updates.Source: §1 · p. 2
- Across five benchmarks and five models, WikiSkill outperforms existing skill-evolution methods and improves over no-skill baselines in most settings.Evaluation spans mathematical reasoning, web search, spreadsheet manipulation, long-context document question answering, and interactive embodied tasks.Source: §1 · pp. 2–3
- 12.3%, 17.5%, and 23.9% average-performance improvements for Qwen 4B, 9B, and 27B models, respectively, show gains increasing with model scale.Qwen-3.5-9B with WikiSkill also outperforms Qwen-3.6-27B without skills (47.4% vs. 39.4%), showing that evolved skills can compensate for substantial differences in model scale.Source: §1 · p. 3
- 70.2% versus 63.4% on ALFWorld: Qwen-3.5-9B performs better with a Qwen-3.6-27B-evolved skill than its own skill.Evolved skills also transfer effectively across model families; these findings suggest that skill discovery and execution are distinct capabilities.Source: §1 · p. 3
- Ablations confirm that persistent knowledge accumulation is critical to WikiSkill’s gains.Accumulating and refining knowledge across iterations supports effective skill evolution.Source: §1 · p. 3
2. Problem Setup
WikiSkill formalizes iterative skill evolution for tool-equipped LLM agents using disjoint training, validation, and test tasks. It co-evolves procedural skills and persistent knowledge through training rollouts and validation gating to maximize performance on unseen test tasks.
2. Problem Setup
- The task dataset pairs each instance with a ground-truth answer and is partitioned into disjoint training, validation, and test splits.These splits are denoted Dtrain, Dval, and Dtest.Source: §2 · p. 3
- An LLM agent has tools and an initially empty skill set that develops separately for each dataset.Skills are modular filesystem directories packaging procedural knowledge, scripts, and resources (Chen et al., 2026a; Li et al., 2026; Liu et al., 2026). Each contains SKILL.md with a unique name, concise description, procedural instructions, and applicability conditions.Source: §2 · p. 3
- Agents execute tasks through observation–action trajectories, and performance on each task split is the average domain-specific score of their predicted answers.Actions can call tools, and the final action emits the prediction. The scoring function f evaluates each prediction against its ground-truth answer and returns a value in [0, 1].Source: §2 · p. 3
- WikiSkill co-evolves active skills and a persistent Wiki, rolling back skill updates that degrade validation scores while retaining accumulated knowledge.Both components start empty. Training rollouts, pattern consolidation, and validation gating guide evolution toward maximizing final test performance on unseen tasks.Source: §2 · pp. 3–4
3. Methodology
WikiSkill co-evolves agent skills and a persistent wiki through task execution, knowledge consolidation, skill proposals, and validation gating. Its architecture separates raw execution traces, accumulated knowledge, and active procedural skills.
3.1. Three-Layer Knowledge Architecture
- WikiSkill separates immutable execution traces, a continuously accumulating wiki, and active procedural skills linked to their motivating patterns.Raw traces retain reasoning, tool calls, outputs, and final answers; SKILL.md contains skill instructions, while PURPOSE.md maps skills to motivating wiki patterns. Wiki pages document failures, successful strategies, and workarounds, while evolution logs and an impact tracker preserve intervention history and reveal recurring errors.Source: §3.1 · pp. 4–5
3.2. Evolutionary Agents and Wiki Orchestration
- The evolutionary loop coordinates task execution, wiki consolidation, skill proposal, and validation, withholding wiki access from the Inference Agent during training.This restriction follows an ablation finding that wiki access during training negatively affects skill development.Source: §3.2 · p. 5
3.2.1. Skill Provisioning for the Inference Agent
- Following prior work (Ni et al., 2026; Yang et al., 2026), full skill injection into the system prompt eliminates triggering and retrieval failures as experimental confounds.The Inference Agent receives the full content of every active skill, making procedural instructions immediately available during task execution.Source: §3.2.1 · p. 5
3.2.2. Wiki Maintainer: Pattern Consolidation
- The Wiki Maintainer combines the full wiki with sampled successful and failing traces to diagnose failures, extract strategies, and incrementally refine pattern pages.Sampling avoids context-window limits; patch-based pattern changes trigger catalog updates and an appended iteration summary in logs.md. There is no hard limit on patterns created or updated per iteration.Source: §3.2.2 · pp. 5–6
3.2.3. Wiki-Informed Skill Proposer
- The Skill Proposer inspects patterns and traces on demand, producing an atomic proposal that creates one skill or patches one existing skill.To avoid context-window exhaustion, it starts with the wiki index, historical impact tracker, and all training-task outcomes rather than fixed presampled traces, then operates in multi-turn ReAct style (Yao et al., 2023).Source: §3.2.3 · p. 6
3.2.4. Gating and Rollback
- Validation gating retains accepted skill changes or restores the last successful skill set, while preserving the wiki and recording every proposal’s outcome.The validation threshold starts at the empty-skill baseline, updates after acceptance, and terminates evolution upon reaching 1.0. After every evaluation, the harness records proposal metadata, target skill, unified diff, validation score, and acceptance outcome in skill-impact.md to help avoid repeating failed modifications.Source: §3.2.4 · p. 6
4. Experiments and Results
WikiSkill achieves the highest average performance across all five evaluated models, and transferred skills can outperform self-evolved skills. Benefits vary with model scale and task, while transfer depends on skill generality and the receiving model’s ability to execute the procedures.
4.1. Experimental Setup
- Five benchmarks cover mathematics, web search, spreadsheets, long-context question answering, and embodied interaction, comparing closed/open-weight models against skill-evolution and no-skill baselines.Models include Gemini-3.5-Flash (Google DeepMind, 2026), Qwen-3.5-4B/9B-Instruct, Qwen-3.6-27B, and Gemma-4-31B-It (Gemma Team, 2026). Skill-evolution baselines are Trace2Skill (Ni et al., 2026), EvoSkill (Alzubi et al., 2026), and SkillOpt (Yang et al., 2026).Source: §4.1 · p. 7
4.2. Main Results
- Three independent evolution runs per method underpin average test scores, with paired-bootstrap significance testing at p < 0.05.All methods start with empty skill sets and inject evolved skills into inference prompts. Gemini-3.5-Flash remains at 85.9% on ALFWorld because its validation score is already 100%, preventing skill evolution and its use as a transfer source.Source: §4.2 · p. 7, Table 1 · p. 8
4.2.1. Skill Evolution Across Models and Tasks
- WikiSkill achieves the highest average performance for every evaluated model and improves over no skills in most model–dataset pairs.Average gains over each model’s strongest competing skill-evolution method are 3.3 points for Qwen-3.5-4B, 5.1 for Qwen-3.5-9B, 10.0 for Qwen-3.6-27B, 5.8 for Gemma-4-31B, and 12.0 for Gemini-3.5-Flash.Source: §4.2.1 · pp. 7–8, Table 1 · p. 8
- 12.3, 17.5, and 23.9-point average gains accrue to Qwen-3.5-4B, Qwen-3.5-9B, and Qwen-3.6-27B, respectively, showing increasing benefits with scale.SpreadSheet gains likewise increase from +6.5 points for Qwen-3.5-4B to +9.3 for Qwen-3.5-9B and +40.9 for Qwen-3.6-27B. Skills also offset size differences: Qwen-3.5-9B with WikiSkill achieves 47.4% average accuracy, exceeding Qwen-3.6-27B without skills at 39.4%.Source: §4.2.1 · pp. 8–9
- 20.6–39.6-point LiveMath gains contrast with task-dependent benefits, including slight OfficeQA degradation for Qwen-3.5-4B when long-context search workflows fail.ALFWorld gains span 14.0–29.3 points across the four models for which skills evolve. Larger models effectively execute evolved document-search workflows, while Qwen-3.5-4B reverts to default reading behavior.Source: §4.2.1 · p. 9
4.2.2. Cross-Model Skill Transfer with WikiSkill
- 18.1% SpreadSheet performance results when Gemini-3.5-Flash uses Qwen-3.5-4B skills, down from 50.5% without skills, illustrating negative transfer from model-specific workarounds.Qwen-3.6-27B skills instead improve Gemini-3.5-Flash to 63.4% on SpreadSheet. The smaller model’s low-level commands and string-conversion rules constrain end-to-end scripting, while fragmented diagnostics introduce redundant tool calls that can exhaust the interaction budget.Source: §4.2.2 · p. 9
- 50.5% on SpreadSheet is achieved by Qwen-3.5-9B using Qwen-3.6-27B skills, versus 24.3% without skills and 33.6% with self-evolved skills.Transfer also crosses model families: Qwen-3.6-27B skills raise Gemma-4-31B’s LiveMath score to 73.7%, versus 33.9% without skills and 56.7% with self-evolved skills. Smaller sources can help larger targets: Qwen-3.5-4B skills yield 73.1% on LiveMath and 66.9% on ALFWorld for Gemma-4-31B.Source: §4.2.2 · p. 9, Table 2 · p. 10
- 52.9% OfficeQA performance results when Qwen-3.6-27B executes Qwen-3.5-4B skills, versus 42.1% without skills, although the source model degrades from 30.2% to 28.5%.Stronger Qwen models derive greater value from the same procedural knowledge. Trajectory analysis suggests smaller models become distracted by long contexts, distinguishing the ability to discover useful procedures from the ability to execute them.Source: §4.2.2 · p. 10
5. Analysis and Discussion
Persistent wiki access improves skill evolution when reserved for the Skill Proposer rather than the Inference Agent. Accumulated patterns and proposal histories support continued refinement, while skill structure and wiki accumulation vary across models and benchmarks.
5.1. Role of Persistent Knowledge in Skill Evolution
- 63.7% versus 48.7% average benchmark performance results from giving the Skill Proposer persistent-wiki access with Inference Agent access disabled, using Gemini-3.5-Flash.LiveMath improves from 51.3% to 72.6%, and SpreadsheetBench from 49.9% to 76.6%. The no-proposer-access configuration also removes the Wiki Maintainer, eliminating persistent knowledge accumulation across iterations.Source: Table 3 · p. 11, §5.1 · p. 11
- 60.9% versus 63.7% average benchmark performance follows adding Inference Agent wiki access during evolution while retaining Skill Proposer access.LiveMath falls from 72.6% to 64.8%. The authors hypothesize that direct wiki access lets inference bypass skills, making training trajectories less informative for skill development.Source: §5.1 · p. 11, Table 3 · p. 11
5.2. Qualitative Analysis: Skill and Wiki Dynamics
- 118.9-128.6 lines characterize Qwen procedural skills, compared with 45.1 lines for Gemma-4-31B and 81.2 lines for Gemini-3.5-Flash.Across models, wiki accumulation averages 6.3-8.9 pattern creations and 7.0–18.4 edits.Source: §5.2 · pp. 11–12, Table 4 · p. 13
- 142.5 lines and 9.8 wiki patterns characterize SpreadSheet’s longest skills and most patterns; LiveMath has the shortest skills (84.6 lines) and fewest patterns (4.4).These are benchmark-level averages.Source: §5.2 · p. 12, Table 4 · p. 13
- 39%-52% of accepted updates occur in Iterations 0–1 across models, while substantial refinement continues through middle and late stages.Across benchmarks, initial-stage shares are 39%-58%; SealQA has 33% of accepted updates in the middle stage and 28% in the late stage. Preserved errors, rejected proposals, and evolution history provide accumulated context for subsequent updates.Source: §5.2 · p. 12
5.3. Case Study: Anatomy of Wiki-Guided Skill Evolution
- In Qwen-3.6-27B’s ALFWorld case, a rejected Iteration 0 proposal informed an accepted Iteration 1 skill, refined at Iteration 4 using accumulated evidence.The wiki preserved the rejected proposal’s diff and outcome, informing the accepted rule Never Return an Item to Its Origin Location. Later loop evidence and stored trajectories informed Each Operation Type ONCE Per Item.Source: Figure 3 · p. 12, §5.3 · p. 13
6. Related Work
WikiSkill distinguishes itself from prior skill-evolution methods through a persistent Wiki Layer that accumulates structured knowledge for subsequent skill updates. It focuses on skill quality rather than retrieval and evolves procedural skills while keeping the broader agent harness fixed.
Experience-Driven Agent Skill Evolution
- Unlike EvoSkill (Alzubi et al., 2026), Trace2Skill (Ni et al., 2026), and SkillOpt (Yang et al., 2026), WikiSkill maintains a separate, evolving knowledge representation.Its persistent Wiki Layer consolidates experience into structured knowledge across iterations, allowing subsequent skill updates to build systematically on accumulated knowledge.Source: §6 · pp. 13–14
Skill-Augmented Agents and Agent Self-Improvement
- WikiSkill evolves reusable procedural skills while holding the broader agent harness fixed, complementing approaches that optimize the harness itself.Harness-optimization work searches for better prompts, context, tools, memory, and workflows using execution traces and environment feedback (Chen et al., 2026b; Lee et al., 2026; Lin et al., 2026; Lou et al., 2026; Zhang et al., 2026b).Source: §6 · p. 14
- WikiSkill focuses on skill quality separately from retrieving relevant skills for individual tasks.Retrieval research addresses selecting relevant skills as reusable libraries grow (Cho et al., 2026; Shi et al., 2026; Su et al., 2026; Ye et al., 2026; Zheng et al., 2026).Source: §6 · p. 14
7. Conclusion
WikiSkill co-evolves agent skills with a persistent wiki, consolidating experience into knowledge that guides validated skill refinements. It consistently outperforms existing skill-evolution methods across five benchmarks and five inference models, with ablations confirming the importance of persistent knowledge accumulation.
7. Conclusion
- Five benchmarks and five inference models show WikiSkill consistently outperforming existing skill-evolution methods and improving over no-skill baselines in most model-dataset pairs.The improvement over no-skill baselines is not universal.Source: §7 · p. 14
- WikiSkill organizes its workspace into three layers, consolidates experience into a persistent wiki, proposes knowledge-guided skill refinements, and gates changes using validation performance.Skill development builds on increasingly well-supported and integrated knowledge across iterations.Source: §7 · p. 14
- Skill evolution complements model scaling: larger models generally benefit more, while smaller models with evolved skills can outperform substantially larger models without skills.Source: §7 · p. 14
- Evolved skills transfer effectively across models and model families, and transferred skills can outperform self-evolved skills.Source: §7 · p. 14
- Ablations confirm that persistent knowledge accumulation is critical for effective skill evolution.Source: §7 · p. 14
Limitations
WikiSkill leaves skill retrieval and triggering unevaluated, rejects potentially useful performance-neutral proposals, and lacks automated wiki pruning. Its benchmarks also exclude very long-horizon tasks, leaving adaptation within a single long execution rollout for future work.
Limitations
- Direct prompt injection isolates skill quality but leaves skill retrieval and triggering unevaluated, which becomes important as the available skill set grows.Active skills are injected directly into the agent prompt to avoid confounding effects from retrieval.Source: Limitations · p. 14
- Strict validation gating rejects performance-neutral proposals that could enable gains in later iterations.Every accepted proposal must improve the validation score, following prior frameworks for fair comparison (Alzubi et al., 2026; Yang et al., 2026). More flexible acceptance criteria remain future work.Source: Limitations · p. 14
- WikiSkill lacks automated wiki pruning despite continuously accumulating pattern pages, evolution logs, and proposal diffs.Pruning may become necessary during longer evolution runs as knowledge accumulates.Source: Limitations · p. 14
- The benchmark suite excludes very long-horizon tasks spanning hundreds of environment actions or multiple hours.It includes OfficeQA long-context document reasoning and multi-step tool interactions. Online adaptation that refines procedural knowledge within a single long execution rollout remains future work.Source: Limitations · p. 14
A. Method Details · A.1. Algorithm · A.2. Distribution of Accepted Skill Updates
WikiSkill evolves skills through training-task rollouts, wiki consolidation, candidate proposals, and validation gating. Rejected proposals roll back skills without discarding the wiki, which records proposal outcomes and skill differences.
A.1. Algorithm
- The maintainer consolidates a sampled subset of training traces into an intermediate wiki before skill proposals are generated.Wiki maintenance uses the previous wiki state together with the sampled traces.Source: §A.1 · p. 19
- At each iteration, the inference agent rolls out on training tasks using the previous iteration’s active skills.A subset of these training traces is then sampled for wiki maintenance.Source: §A.1 · p. 19
- Rejected proposals roll back only the skills; the wiki is retained and updated with the proposal outcome and skill differences.This produces the final wiki state for the iteration.Source: §A.1 · p. 19
A.2. Distribution of Accepted Skill Updates
- Validation determines whether proposed skills are accepted or rolled back, and Table 5 reports when updated skill proposals are accepted.The supplied passage does not include Table 5’s acceptance distribution.Source: §A.1 · p. 19, §A.2 · p. 19
A.1. Algorithm
- The proposer generates candidate skill modifications after wiki consolidation and before validation gating.The evolution loop takes training tasks, validation tasks, a performance metric R, and an iteration count K.Source: §A.1 · p. 19
- Baseline validation evaluates the initial skills on validation tasks and initializes Rbest from their performance metric.This evaluation precedes the iterative skill-evolution process.Source: §A.1 · p. 19
B. Dataset Details and Splits
The evaluation spans five benchmarks testing mathematical reasoning, web retrieval, spreadsheet programming, long-context financial question-answering, and embodied decision-making. Reported scores average three independent evolutionary runs, with paired bootstrap significance testing used to assess variability.
LiveMathematicianBench
- LiveMathematicianBench (He et al., 2026) uses recent multiple-choice mathematics competition problems to test complex reasoning, quantifiers, and extremal conditions.LiveMath requires a single-step final answer without external tools.Source: B. Dataset Details and Splits · pp. 19–20
SealQA
- SealQA (Pham et al., 2026) tests scholarly factual question-answering through effective search-query formulation and answer extraction from web results.Experiments use the July, 2026 version, with Google Search API access and file reading for multi-step retrieval.Source: B. Dataset Details and Splits · pp. 19–20
SpreadsheetBench
- SpreadsheetBench (Ma et al., 2024) tests correct code under library constraints and complex table transformations.Constraints include formula-evaluation limitations; a bash shell supports Python execution and table manipulation.Source: B. Dataset Details and Splits · pp. 19–20
OfficeQA
- OfficeQA requires synthesizing evidence across long historical Treasury bulletins and multi-page financial tables.Following Yang et al. (2026), prompts include pre-parsed oracle reference pages. Agents retain glob, grep, and read tools to search and cross-reference full bulletin files on disk.Source: B. Dataset Details and Splits · pp. 19–20
ALFWorld
- ALFWorld tests sequential decision-making, spatial reasoning, and error recovery through multi-step household tasks in a text-based simulator.Agents issue text actions and receive simulator feedback, rather than answering static questions.Source: B. Dataset Details and Splits · p. 20
Evaluation protocol
- Three independent runs of the entire evolutionary pipeline underpin averaged test scores, with paired bootstrap significance testing addressing variability from small validation splits.Task splits and available toolsets strictly match prior work (Alzubi et al., 2026; Yang et al., 2026).Source: B. Dataset Details and Splits · pp. 20–21
C. Implementation Details · D. Baseline Details and Optimizer API Call Analysis · D.1. Baseline Methods
WikiSkill uses stratified execution traces for diagnostic feedback and bootstrap tests for performance comparisons. The baselines evolve skills through trace-derived proposals and validation-based selection.
C. Implementation Details
- 8 traces at most inform each Wiki-maintainer iteration: up to five failures for root-cause analysis and three successes for effective strategies and regression prevention.Each execution log is capped at 15,000 characters before prompt injection.Source: C. Implementation Details · p. 21
- 1,000 paired bootstrap iterations per benchmark estimate candidate accuracies and pairwise performance margins.Each iteration samples test instances with replacement to construct an evaluation set matching the original test-set size.Source: C. Implementation Details · p. 21
- Cross-benchmark evaluation uses stratified macro-average bootstrap resampling, independently resampling instances within each benchmark and weighting all benchmarks equally.The resulting statistic is macro-average accuracy.Source: C. Implementation Details · p. 21
- p < 0.05 against every competitor is required for the observed leader to be declared the sole top performer.Otherwise, all methods statistically indistinguishable from the leader at p ≥ 0.05 form a bolded top-tier statistical tie; initial rankings use observed performance or cross-benchmark macro-average performance.Source: C. Implementation Details · p. 21
D.1. Baseline Methods
- Trace2Skill (Ni et al., 2026) and SkillOpt (Yang et al., 2026) hierarchically consolidate success- and failure-derived patches, accepting skill-document updates based on validation performance.Trace2Skill uses parallel success and error analysts, then recursively merges their structured patches. SkillOpt uses the six-stage ReflACT pipeline—Rollout, Reflect, Aggregate, Select, Update, Evaluate—to reflect on full traces and update one monolithic skill document.Source: §D.1 · pp. 21–22
- Unlike Trace2Skill (Ni et al., 2026), EvoSkill (Alzubi et al., 2026) supplies only failure traces to its proposer and maintains a bounded frontier of validated skill candidates.EvoSkill samples training tasks using a round-robin category schedule and supplies a flat history of proposal outcomes. Proposed modifications become SKILL.md files, receive validation scores, and enter the frontier of top-performing programs.Source: §D.1 · p. 21
D.2. Optimizer API Call Complexity · 3. Hierarchical reduce & apply stage: The 𝑁train · E. System and Agent Prompts
WikiSkill uses full-batch optimization to keep optimizer API calls independent of training-set size, unlike the compared methods’ linear scaling under their reported settings. The supplied prompts specify task-specific inference behavior, incremental wiki maintenance, and wiki-guided skill proposals.
D.2. Optimizer API Call Complexity
- 1 + T_ReAct optimizer calls per iteration give full-batch WikiSkill O(1) call complexity with respect to training-set size.One call maintains the wiki, followed by roughly 10–20 Skill Proposer reasoning turns, each requiring one call. Constant call complexity can still incur higher inference cost on some datasets, accompanied by consistent performance gains over prior skill-evolution methods.Source: Table 7 · p. 22, §D.2 · p. 22
3. Hierarchical reduce & apply stage: The 𝑁train
- O(N_train/B) characterizes EvoSkill and SkillOpt, while Trace2Skill requires O(N_train) calls because every training trajectory receives individual analysis.EvoSkill and SkillOpt perform best with minibatches, whereas WikiSkill uses full batches and selectively reads traces on demand. Trace2Skill additionally merges local patches through a c-ary reduction tree and makes a final formatting call.Source: §D.2 · p. 23, §3 · p. 23
E. System and Agent Prompts · LiveMathematicianBench Inference Agent System Prompt · SealQA Inference Agent System Prompt
- The provided inference prompts make LiveMathematicianBench a mathematical multiple-choice task and give SealQA web_search and read_file tools for factual questions.Both require final answers inside <answer> tags: a single choice label for LiveMathematicianBench and an exact value without explanation for SealQA. The section supplies exact inference, Wiki Maintainer, and Skill Proposer system prompts.Source: E. System and Agent Prompts · p. 23, LiveMathematicianBench Inference Agent… · p. 24, SealQA Inference Agent System Prompt · p. 24
SpreadsheetBench Inference Agent System Prompt
- SpreadsheetBench’s inference prompt directs Python-based spreadsheet manipulation, restricts edits to answer_position, and confines file access to working_directory.The agent must save the modified spreadsheet to the supplied output_path using the exact provided absolute paths.Source: SpreadsheetBench Inference Agent… · pp. 24–25
OfficeQA Inference Agent System Prompt · ALFWorld Inference Agent System Prompt
- OfficeQA grounds answers in local Treasury documents, while ALFWorld receives task descriptions, recent action histories, and current observations.OfficeQA requires targeted searches and small evidence reads, prohibits invented values, and permits arithmetic only after extracting exact operands.Source: OfficeQA Inference Agent System Prompt · p. 25, ALFWorld Inference Agent System Prompt · p. 25
Wiki Maintainer Agent System Prompt
- The Wiki Maintainer records generalizable success and failure patterns with root causes, exact command sequences, and concrete solutions, updating existing patterns rather than duplicating them.The wiki includes a pattern catalog, evolution log, skill-outcome record, and pattern pages; incremental JSON updates require a complete index and appended log. Index descriptions summarize the problem, root cause, and fix.Source: Wiki Maintainer Agent System Prompt · pp. 26–27
Skill Proposer Agent System Prompt
- The Skill Proposer consults wiki patterns, rejected proposals, and execution traces to diagnose failures before creating, patching, or declining a skill change.It must read at least four execution traces, avoid previously rejected approaches, and prefer patching partially correct skills. New skills include application conditions, instructions, provenance, addressed patterns, and evolution history.Source: §3 · p. 23, Skill Proposer Agent System Prompt · pp. 27–28
Related papers
This paper cites
- Efficient Memory Management for Large Language Model Serving with PagedAttention2023
- Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces2026
- SkillOpt: Executive Strategy for Self-Evolving Agent Skills2026
- SkillOS: Learning Skill Curation for Self-Evolving Agents2026
- SkillNet: Create, Evaluate, and Connect AI Skills2026
- HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundry2026
Cited by
- Before the Next Rollout: Diagnosing Skill Utility Shifts from Policy Updates2026
- Skill Dreaming: Evolving Repository Skills through Pull Request Streams2026
- $\Delta\tau$-Bench: Pushing Agents Toward a Moving Capability2026
- HumanSkill-4D: Training-Free Language Querying of Animatable Gaussian Avatars in Space and Time2026
- Harness-Delta Attribution: Dissecting Gains From Harness Evolution2026
- VIEWCRAFT: Evolving What Multimodal Agents See and How They Read It2026