EvolvingBrain: An Adaptive Skill System for Multi-modal Agents Self-Evolution in Open Worlds

1 Introduction

• Agents’ self-evolution remains limited: supervised fine-tuning supports imitation without active experience organization, while outcome-based reinforcement learning underuses intermediate trajectory information.Existing skill methods also primarily distill successful trajectories, leaving learning from failures, feedback-driven evolution, and state-dependent granularity unresolved.Source: §1 · p. 1
• Stratified Skill Self-Evolution extracts reusable procedures and problematic patterns from successful and failed trajectories, then organizes evolved skills into Domain, Plan, Action, and Trap levels.Repeated open-world interactions evolve the skills, while shared knowledge across tasks is abstracted into Domain Skills.Source: §1 · pp. 1–2
• State-Aware Adaptive Execution summarizes long-horizon interaction histories into Structured Abstractive Memory and uses it to retrieve and integrate skills suited to the current state.The memory records confirmed progress, current obstacles, and ineffective attempts.Source: §1 · p. 2
• Across three backbones, EvolvingBrain improves success rates over direct agents by 11.64 percentage points on VAB-Minecraft and 8.44 on DiscoveryWorld, and skills transfer to held-out DiscoveryWorld domains.Further analyses examine complementary procedural and problematic knowledge, skill granularity, and feedback-driven evolution.Source: §1 · p. 2
• EvolvingBrain uses an adaptive skill system to organize, evolve, and reuse knowledge from agents’ own open-world interactions.Its two components are Stratified Skill Self-Evolution and State-Aware Adaptive Execution.Source: §1 · pp. 1–2

2.1 Problem Formulation and Method Overview

• EvolvingBrain evolves skills from model-specific trajectories, then freezes the skill architecture while memory guides state-aware retrieval and fusion during execution.The agent interacts with an open-world environment through observations, actions, and feedback toward a task goal.Source: §2.1 · pp. 2–3

2.2.1 Model-Specific Initial Trajectory Collection

• Each frozen target model builds its initial trajectory set from its own interactions, retaining both successful and failed trajectories.Skill construction uses that model’s trajectories and does not draw on a predefined trajectory pool or skill library.Source: §2.2.1 · p. 3

2.2.2 Procedural and Problematic Patterns

• Procedural patterns capture reusable task-advancing regularities, while problematic patterns record failure-related conditions and behaviors; either pattern type can occur in successful or failed trajectories.Problematic patterns include triggers, erroneous behaviors, consequences, interaction steps, and resolution evidence.Source: §2.2.2 · p. 3, Table 1 · p. 4

2.2.3 Constructing Trap Skill

• Trap Skills encode problem diagnoses, applicable conditions, avoidance steps, fallbacks, and success criteria, and are revised through retries until problems are resolved or recovery stops.Recovery targets root causes blocking progress; successful trajectories are also inspected for ineffective actions, repeated attempts, and incorrect steps.Source: §2.2.3 · p. 4

2.2.4 Constructing Plan and Action Skills

• From accepted high-quality trajectories, Plan Skills capture end-to-end task procedures and Action Skills capture reusable local procedures grounded in observable state transitions.Candidates are reused, updated, or added based on whether existing skills match and whether new information is contributed.Source: §2.2.4 · pp. 4–5

2.2.5 Constructing Domain Skills and the Stratified Skill Architecture

• Domain Skills abstract shared principles from Plan, Action, and Trap Skills, which are organized into a hierarchy linking domains to plans and plans to reusable atomic skills.The three levels increase in granularity from Action and Trap Skills, through Plan Skills, to Domain Skills.Source: §2.2.5 · p. 5

2.3.1 Structured Abstractive Memory

• Structured Abstractive Memory summarizes task-relevant state, progress, feedback, blockers, constraints, assumptions, and retry conditions instead of preserving the full interaction history.It is updated before skill retrieval using the task goal, prior memory, pending action, latest environmental feedback, and Domain Skill guidance.Source: §2.3.1 · p. 5

2.3.2 State-Aware Skill Fusion Strategy

• State-Aware Skill Fusion combines relevant skills, structured memory, and the current multimodal observation into temporary Active Guidance suited to the current execution stage.Hierarchical representation and adaptive retrieval provide skills at suitable granularities for models at different scales.Source: §2.3.2 · pp. 5–6

3.1 Experimental Setup

• The evaluation uses VAB-Minecraft’s official split and constructed training, test, and out-of-domain splits for DiscoveryWorld, comparing against direct agents and MMSkills on VAB-Minecraft.Direct agents use the same interface and interaction budget but no external skill library; MMSkills is omitted on DiscoveryWorld because its public implementation lacks a complete pipeline.Source: §3.1 · p. 6

3.2 Overall Performance

• 11.64 and 8.44 percentage-point success-rate gains over direct agents on VAB-Minecraft and DiscoveryWorld, respectively.Corresponding score gains are 0.0945 and 0.0607.Source: §3.2 · p. 6
• EvolvingBrain outperforms MMSkills on VAB-Minecraft by 6.97 percentage points in success rate and 0.0604 in average score.With Kimi-K3, it reaches 79.37% success on VAB-Minecraft and 74.07% on DiscoveryWorld.Source: §3.2 · p. 6

3.3 Ablation Studies

• 6.64 and 4.32 percentage-point losses follow coarse retrieval for Qwen3.8-27B, while fine-grained retrieval yields 5.19 and 5.55-point losses for Kimi-K3 across VAB-Minecraft and DiscoveryWorld.Adaptive retrieval achieves the best performance across both benchmarks and metrics; the figure reports fixed-granularity success-rate loss relative to adaptive retrieval.Source: §3.3 · p. 7, Figure 4 · p. 7
• 3.89 and 3.60 percentage-point average success-rate gains follow three evolution iterations rather than one on VAB-Minecraft and DiscoveryWorld, respectively.The corresponding score gains are 0.0309 and 0.0320; all other components remain unchanged in the one-iteration comparison.Source: §3.3 · pp. 7–8, Figure 5 · p. 8

3.4 Out-of-Domain Generalization

• 10.00, 11.11, and 10.00 percentage-point success-rate gains on unseen domains for Qwen3.8-27B, Kimi-K3, and GPT-5.6-Sol yield an average gain of 10.37 points.The average in-domain-to-out-of-domain success-rate degradation is 12.42 points for EvolvingBrain versus 14.36 for direct agents; no out-of-domain trajectories are used to construct or evolve the library.Source: §3.4 · p. 8, Figure 5 · p. 8

Open-World Agents

• EvolvingBrain converts a frozen multimodal agent’s own interactions into persistent skills without updating model parameters.The related-work discussion contrasts this with approaches using demonstrations, external knowledge, world models, planning, and memory.Source: §4 · p. 8

Experience-Based Skill Evolution

• MMSkills and MMG2Skill use public trajectories and external guides, respectively (Zhang et al., 2026; Che et al., 2026), while EvolvingBrain builds model-specific skills from successful and failed interactions.It validates Trap Skills through environmental retries and distills them into Domain, Plan, and Action Skills.Source: §4 · p. 8

Hierarchical and State-Aware Skills

• EvolvingBrain organizes Domain, Plan, Action, and Trap Skills, while Structured Abstractive Memory tracks progress and blockers to guide state-aware retrieval and fusion.The system retrieves and fuses skills required by the current state.Source: §4 · p. 8

• EvolvingBrain distills successful and failed interactions into a hierarchical, model-specific Skill Brain and retrieves guidance according to execution state.Across three backbones, it consistently improves performance on VAB-Minecraft and DiscoveryWorld and generalizes to held-out domains.Source: §5 · p. 8

• Broader real-world evaluation and improved efficiency of online skill evolution remain future work.Source: Limitations · p. 9

A.1 Benchmark Splits

• VAB-Minecraft uses 90 training task goals for skill construction and 116 official test tasks, with no task overlap.Source: §A.1 · p. 10
• DiscoveryWorld trains on six themes using variants 0–2 and tests in-domain on variants 3–4 across all difficulty levels.Source: §A.1 · p. 10
• Two remaining DiscoveryWorld themes, with all variants, are held out for out-of-domain evaluation.Source: §A.1 · p. 10
• Training data is used only to construct skills; model parameters and the resulting skill library remain frozen during evaluation.Source: §A.1 · p. 10

A.2 Evaluation Metrics

• Across evaluation tasks, success rate uses the environment’s task-success indicators, while average progress score aggregates each task’s benchmark-native score.Source: §A.2 · p. 10
• VAB-Minecraft success requires the specified item and quantity in the agent’s inventory; its progress score uses environment reward from task-relevant intermediate items.Source: §A.2 · pp. 10–11
• DiscoveryWorld success is determined by its completedSuccessfully flag, and its progress score is normalized.Source: §A.2 · p. 11
• Success rate is the primary metric, while progress score captures partial completion when a task is not fully solved.Source: §A.2 · pp. 10–11

B Implementation and Baseline Details

• Within each backbone, EvolvingBrain and the direct baseline share task instructions, interfaces, inference settings, and interaction budgets; only EvolvingBrain receives retrieved skills.Source: B Implementation and Baseline Details · p. 11
• Evaluation allows up to 60 rounds on VAB-Minecraft and 100 or 200 steps on DiscoveryWorld, depending on task difficulty.Source: B Implementation and Baseline Details · p. 11
• The evaluated backbones are Qwen3.8-27B, Kimi-K3, and GPT-5.6-Sol, with model parameters frozen during skill construction and evaluation.Source: B Implementation and Baseline Details · p. 11
• MMSkills is evaluated on VAB-Minecraft using released skills, but not on DiscoveryWorld because its release lacks the pipeline needed to construct skills for a new environment.Source: B Implementation and Baseline Details · p. 11

C Case Study

• Both agents use Kimi-K3, share the same initial state and 60-round budget, and differ in whether they receive the frozen skill library.Source: C Case Study · p. 11
• After repeated search reformulations fail to find iron, Structured Abstractive Memory records the bedrock blocker and retrieval supplies Trap guidance.Source: C Case Study · pp. 11–12
• EvolvingBrain completes the held-out piston task in 27 rounds, while the direct agent fails to obtain a piston within 60 rounds and scores 0.3114.Figure 6 traces state-aware retrieval and recovery; its Skill Brain remains frozen during evaluation.Source: C Case Study · pp. 11–12, Figure 6 · p. 12
• When iron is acquired and a teleport drops the redstone, the changed state prompts Plan, Action, and Trap retrieval for bounded item recovery.Source: C Case Study · p. 12
• The retrieved Trap Skill was consolidated offline from ten source trajectories across nine training tasks; the held-out trace can reuse it but cannot update it.Its safeguards include verifying position and inventory changes, stopping at bedrock, and switching prompt family after grounded evidence of no progress.Source: C Case Study · p. 12

D.1 Skill-System Ablation

• The Trap Skill ablation disables Trap Skill retrieval while retaining Domain, Plan, and Action Skills; both skill-enabled variants use adaptive retrieval.Table 4 contains the complete numerical results corresponding to Figure 3.Source: §D.1 · p. 12

D.2 Adaptive Skill Retrieval

• The adaptive retrieval analysis varies procedural retrieval depth while retaining Trap Skill retrieval in every variant.Coarse retrieval stops at Domain Skills; fine-grained retrieval descends to Plan and Action Skills, with complete results in Table 5.Source: §D.2 · p. 12, Table 5 · p. 13

D.3 Effect of Self-Evolution

• The self-evolution ablation varies the maximum number of evolution iterations while holding the hierarchy, retrieval mechanism, and evaluation settings constant.Table 6 reports the complete results.Source: §D.3 · p. 13, Table 6 · p. 15

D.4 Out-of-Domain Generalization

• The DiscoveryWorld generalization comparison evaluates all 30 tasks from two themes held out during skill construction.Table 7 reports the in-domain and out-of-domain results.Source: §D.4 · p. 13, Table 7 · p. 15

E.1 Four Skill Templates

• The four skill templates distinguish environment-level regularities, task procedures, local interaction steps, and failure conditions with recovery strategies.Domain Skills capture shared knowledge; Plan Skills specify prerequisites and ordered stages; Action Skills map observable states to verifiable outcomes; Trap Skills record recurrent failures and responses.Source: §E.1 · p. 13, Figure 7 · p. 14, Figure 8 · p. 16

E.2 Core Prompts

• The appendix presents eight core prompts spanning offline skill construction, interaction-time memory updates, and adaptive retrieval and guidance composition.The fusion prompt makes ActiveGuidance temporary rather than a new long-term Skill.Source: §E.2 · p. 13, Figure 18 · p. 26, i State-Aware Skill Fusion System Prompt · p. 26

i Trap-Skill Generation System Prompt

• Offline prompts turn successful trajectories or subtasks into Plan and Action Skills, concrete problems into Trap Skills, and validated principles into Domain Skills.Plan Skills preserve applicability conditions and procedures; Action Skills capture local procedural and perceptual knowledge; Trap Skills guide recognition, prevention, or resolution of recurring failures.Source: Figure 11 · p. 19, i Plan-Skill Generation System Prompt · p. 19, Figure 12 · p. 20

i Structured Memory Update System Prompt

• Online prompts update compact Structured Abstractive Memory, gate guidance needs, retrieve relevant skills, and fuse them into state-conditioned ActiveGuidance.Fusion starts from the agent’s actual current state; retrieved skills are selected for the current state and knowledge gap.Source: Figure 15 · p. 23, i Structured Memory Update System Prompt · p. 23, Figure 16 · p. 24