Source-linked AI summary
MineEvolve: Self-Evolution with Accumulated Knowledge for Long-Horizon Embodied Minecraft Agents
Zhengwei Xie, Zhisheng Chen, Ziyan Weng, Jinhan Li, Chenglong Li, Zikai Xiao, Jingwei Song, Jinhao Jing, Vireo Zhang, Kun Wang
TL;DR
Long-horizon embodied agents need to convert execution experience into knowledge that guides future decisions. MineEvolve structures feedback into skills and remedies, manages that knowledge, and applies it to local plan repair; experiments report consistent improvements across planners, especially for high-dependency Minecraft tasks.
Problem
Long-horizon Minecraft tasks require agents to transform past successes and failures into actionable knowledge for future planning.
Method
MineEvolve uses Monitor, Inducer, Curator, and Adaptor to structure execution feedback, generate skills and remedies, manage knowledge, and repair unfinished plans.
Results
MineEvolve consistently improves performance across multiple language-model planners, with larger gains on high-dependency task groups.
Takeaways & Limitations
Structured external behavioral knowledge provides an effective path toward self-evolving embodied agents in long-horizon environments.
Takeaways & Limitations
MineEvolve is mainly evaluated in Minecraft and relies on environment-specific execution signals, so transfer to other domains may require redesigning feedback fields and verification rules.
Abstract
from arXiv · showhide
Long-horizon embodied intelligence requires agents to improve through interaction, not merely to execute plans generated from static goals. A central challenge is therefore to transform past executions into knowledge that can shape future decisions. Minecraft provides a representative testbed for this problem, where tasks such as crafting tools, building redstone components, and obtaining diamond equipment involve long prerequisite chains and are frequently disrupted by missing tools, blocked paths, GUI failures, or stagnant execution. To this end, we propose \textbf{MineEvolve}, a knowledge-driven self-evolution framework that converts execution feedback into actionable behavioral knowledge. MineEvolve first uses \underline{\emph{\textbf{\ding{182}Monitor}}} to convert each subgoal execution into typed feedback, including state changes, inventory changes, failure types, progress signals, and stagnation indicators. \underline{\emph{\textbf{\ding{183}Inducer}}} then derives reusable skills from successful executions and remedies from failed or stagnant executions. \underline{\emph{\textbf{\ding{184}Curator}}} validates, merges, filters, and retrieves these knowledge entries, while \underline{\emph{\textbf{\ding{185}Adaptor}}} uses them to repair the unfinished part of the plan under repeated failures or stagnation. Experiments on the Minecraft MCU long-horizon task suite show that MineEvolve consistently improves performance across multiple language-model planners, with larger gains on high-dependency task groups. Ablation and knowledge-accumulation studies further demonstrate that converting execution signals into structured behavioral knowledge is an effective path toward self-evolving embodied agents in long-horizon environments. Our code is available at https://github.com/xzw-ustc/MC-MineEvolve.
1 Introduction
MineEvolve addresses the challenge of turning execution feedback into actionable knowledge for long-horizon Minecraft agents. Its Monitor, Inducer, Curator, and Adaptor components generate, manage, and apply skills and remedies to repair plans, improving performance across planners, especially on high-dependency tasks.
- Motivation: Long-horizon Minecraft tasks require agents to transform past successes and failures into knowledge that guides future behavior.Tasks such as crafting tools and obtaining diamond equipment involve extended prerequisite chains and can be disrupted during execution.
- Framework: MineEvolve converts each subgoal execution into typed feedback, including state, inventory, failure, progress, and stagnation information.These signals provide structured information for subsequent knowledge generation.
- Novelty: MineEvolve converts failures from passive unsuccessful-attempt records into actionable knowledge that guides future plan revision.Examples include clearing blocking blocks, selecting alternative routes, and repairing missing crafting prerequisites before retrying.
- Results: MineEvolve consistently outperforms static experience retrieval baselines across multiple language-model planners, with the largest improvements on high-dependency task groups.Reported high-dependency groups include Iron, Redstone, Diamond, and Armor.
- Framework: Inducer derives reusable skills from successful executions and remedies from failed or stagnant executions, with triggers, verification rules, and repair actions.Skills encode effective behaviors, whereas remedies guide revision after local failures.
- Framework: Curator validates, merges, filters, and retrieves knowledge, while Adaptor revises only the remaining subgoals after repeated failures or stagnation.This local repair reduces ineffective retries and improves execution stability.
2 Related Work
Minecraft embodied-agent research has progressed from learning from demonstrations and reinforcement learning toward hierarchical language-model planning and multimodal control. Recent systems accumulate skills, organize experience, or use external knowledge to support long-horizon planning and reflection.
- Early Minecraft Agents: Early Minecraft embodied-agent research focused mainly on reinforcement learning, imitation learning, and human demonstrations.MineRL provides large-scale human demonstration data for sample-efficient learning in complex Minecraft tasks.
- Early Minecraft Agents: MineDojo broadened Minecraft into an open-ended embodied-agent platform combining language tasks, environment interaction, and internet-scale knowledge.Its knowledge sources include videos, Wiki pages, and community content.
- Language-Model Agents: Recent Minecraft agents increasingly use hierarchical architectures with high-level language-model planners and low-level controllers.DEPS improves subgoal generation and selection, while JARVIS-1 integrates visual observations, language instructions, planning, and embodied control.
- Memory and Skill Accumulation: Voyager accumulates transferable code skills, while Optimus-1 organizes world knowledge and historical experience for long-horizon planning and reflection.Voyager uses an automatic curriculum and environment feedback; Optimus-1 uses a hierarchical knowledge graph and multimodal experience pool.
3 Method
MineEvolve converts subgoal executions into structured feedback, induces skills and remedies, and uses curated knowledge to guide planning and repair unfinished plans. Its Monitor detects progress and stagnation, while Adaptor preserves completed work during local repair.
- Feedback monitoring: Monitor represents each subgoal execution with success, state change, inventory change, failure type, progress, and stagnation indicators.These fields are derived from environment states, inventory changes, and execution monitoring rather than free-form reflection.
- Framework overview: MineEvolve models self-evolution as four sequential operations: Monitor, Inducer, Curator, and Adaptor.Monitor extracts feedback; Inducer creates knowledge; Curator manages retrieval; Adaptor repairs unfinished plans after repeated failures or stagnation.
- Feedback monitoring: Monitor combines movement, inventory change, and task-specific progress to identify low-progress executions.Task-specific progress can include crafting, GUI-state, or smelting progress; unchanged inventory and ineffective movement are examples of stagnation.
- Knowledge induction: Inducer abstracts successful feedback into skills and failed or stagnant feedback into remedies with triggers, actions, and verification or scope information.Skills encode reusable steps and verification rules, whereas remedies encode failure patterns and repair actions.
- Knowledge induction: A repeated-failure window or clear stagnation is required before feedback is stored as a remedy, limiting reactions to isolated failures.The recent execution window and failure-rate threshold prevent a single accidental failure from immediately producing a remedy.
- Knowledge curation: Curator filters knowledge for completeness, matchability, executability, specificity, and conflicts, then retrieves relevant skills and remedies within a fixed prompt budget.Retrieved skills provide positive guidance, while remedies provide risk information and repair actions.
- Local plan repair: Adaptor retains the completed plan prefix and repairs only the unfinished portion using retrieved or newly induced remedies.Examples include avoiding repeated blocked routes and inserting a missing crafting-table prerequisite before retrying a recipe.
4 Experiments
MineEvolve is evaluated on 70 Minecraft MCU tasks spanning seven technology-tree groups, with particular emphasis on difficult tasks requiring long prerequisite chains. Across controlled comparisons, its structured feedback, knowledge management, local repair, and accumulated knowledge improve performance, especially on hard groups.
- Experimental Setup: The evaluation uses 70 MCU tasks across seven technology-tree groups, including difficult Iron, Redstone, Diamond, and Armor tasks with longer prerequisite chains.Agents operate in survival-style Minecraft with first-person RGB observations, structured state inputs, and actions for navigation, mining, placing, item use, inventory control, and crafting.
- Main Results: Under a unified protocol, MineEvolve achieves the highest Overall and Hard Avg. across multiple planner backbones.The gains are more pronounced on difficult groups, where missing prerequisites, navigation stagnation, and crafting failures propagate through long dependency chains.
- Controlled Ablation Study: Removing the Curator, Adaptor, or continuous knowledge updates causes clear performance drops, showing that verification, updating, and local repair matter beyond experience storage.The targeted ablations use the same planner, low-level policy, horizon, retrieval top-K, and evaluation-time LLM-call budget.
- Controlled Ablation Study: Each MineEvolve component contributes on difficult tasks, while removing typed feedback produces the largest degradation and Binary Feedback performs worst.Free-form reflection remains below the full system; skills and remedies are complementary, with Remedies Only consistently outperforming Skills Only.
- Feedback Granularity: Typed Execution Feedback substantially outperforms binary feedback and trajectory-level reflection across all difficult task groups.The result attributes remedy effectiveness to feedback fields that are matchable, verifiable, and directly actionable during planning and local repair.
- Knowledge Accumulation: With the knowledge base frozen at checkpoints, MineEvolve shows more pronounced and sustained improvement than Static Store and Text Reflection across Iron, Redstone, Diamond, and Armor.Diamond remains the most difficult group, but MineEvolve improves steadily with accumulated experience.
5 Conclusion
MineEvolve converts execution feedback into external behavioral knowledge rather than updating model parameters, using skills for successful behavior and remedies for local plan repair. Experiments show consistent gains across planner backbones, especially on high-dependency task groups, while the current implementation remains tied to Minecraft-specific feedback and verification signals.
- Conclusion: MineEvolve continually converts execution feedback into actionable behavioral knowledge without updating language-model parameters.The framework uses Monitor, Inducer, Curator, and Adaptor to support knowledge construction and use.
- Conclusion: Successful executions become reusable skills, while failed or stagnant executions become remedies for local plan repair.This separates effective behavior reuse from failure-conditioned recovery knowledge.
- Conclusion: MineEvolve consistently improves performance across multiple planner backbones, especially on high-dependency Iron, Redstone, Diamond, and Armor tasks.Ablation and knowledge-accumulation studies also identify typed feedback, skill/remedy generation, curation, and local repair as important for sustained improvement.
- Limitations: The current implementation is mainly evaluated in Minecraft and relies on environment-specific signals such as inventory changes, GUI states, crafting progress, and failure types.Transfer to other embodied domains may require redesigning the feedback fields and verification rules.
- Conclusion: Structured external behavioral knowledge provides an effective path toward self-evolving embodied agents in long-horizon environments.This conclusion is supported within the Minecraft-based evaluation setting.
B.2 Environment, Tasks, and Protocol Control
The evaluation uses a controlled Minecraft survival environment and system-level protocol, while MineEvolve converts execution traces into structured knowledge and repairs unfinished plans locally.
- Environment: Agents operate from first-person RGB observations and structured state inputs in a survival-style Minecraft environment with empty initial inventories.Inputs include position, health, hunger, GUI status, inventory, and observable task progress.
- MineEvolve loop: MineEvolve maintains an episode feedback buffer and external knowledge store while Monitor–Inducer generates candidates and Curator–Adaptor validates, retrieves, and applies them.The overall loop combines execution feedback with knowledge-conditioned planning and repair.
- Feedback generation: Monitor extracts typed feedback from each subgoal execution, including success, state and inventory changes, failure type, progress, and stagnation.These fields support subsequent induction of skills from successful segments and remedies from failed or stagnant segments.
- Knowledge maintenance: Curator filters candidates for schema completeness, context match, executability, specificity, and conflicts before maintaining the knowledge store.This prevents non-executable, overly generic, or conflicting entries from entering retained knowledge.
- Local repair: Adaptor freezes the valid plan prefix and replans only unfinished subgoals when recent feedback indicates repair is needed.Repairs can avoid repeated paths, clear blocking blocks, choose alternative routes, or insert missing prerequisites.
D.1 MCU Tech-Tree Task Suite and Metric Definitions
The MCU evaluation suite contains 70 Minecraft tech-tree tasks spanning seven progression groups, and reports task-count-weighted success rates under controlled comparisons.
- Task suite: The 70-task suite contains Wooden, Stone, Iron, Gold, Redstone, Diamond, and Armor groups spanning basic acquisition through long dependency chains.Hard groups involve missing resources, tool dependencies, navigation bottlenecks, recovery requirements, and failure propagation.
- Task groups: Iron, Redstone, Diamond, and Armor are designated hard groups because they require longer prerequisite chains and more recovery planning.These groups more clearly expose missing prerequisites, local failure propagation, and recovery needs.
- Metrics: Success rate measures whether each task is completed within the episode horizon, with group and aggregate metrics computed from task-level outcomes.The supplied metric definition identifies binary task completion outcomes and group-level aggregation.
- Metrics: Easy Avg., Hard Avg., and Overall are task-count-weighted averages so each individual task contributes equally to its corresponding score.The hard average covers Iron, Redstone, Diamond, and Armor; Overall covers all seven groups.
- Metrics: The hard-group sizes are Iron 16, Redstone 6, Diamond 7, and Armor 13 tasks.These counts define the weighting used for Hard Avg.
D.2 Full Task-Group Results
The full-results table reports success rates across all seven MCU task groups under a common system-level evaluation setup, with Overall weighted across 70 tasks.
- Full results: The table complements compact results by showing how performance gains are distributed across stages of the Minecraft technology tree.It also includes a STEVE-1 low-level-only baseline for difficulty context.
- Full results: Table 4 reports success rate SR (%) for Wooden, Stone, Iron, Gold, Redstone, Diamond, Armor, and Overall.The table caption defines abbreviations for the seven groups and specifies task-weighted Overall aggregation.
- Protocol: All system-level methods in the full-results table use the same task set, evaluation metric, and interaction horizon.The comparison is framed as a controlled system-level evaluation.
- Comparison framing: Table 5 positions recent Minecraft agents by experimental protocol and mechanism design rather than claiming an absolute ranking under mismatched protocols.Its purpose is comparative mechanistic clarification.
D.4 Knowledge Accumulation and Runtime Overhead
The paper evaluates knowledge accumulation on held-out hard tasks and separately measures whether growing external knowledge creates retrieval or prompting overhead.
- Knowledge accumulation: Table 6 reports group-wise knowledge-accumulation results on held-out Iron, Redstone, Diamond, and Armor tasks.Hard Avg. is the task-count-weighted average over these four groups, with the knowledge base frozen before each evaluation checkpoint.
- Knowledge accumulation: The accumulation experiment evaluates performance at checkpoints while freezing the external knowledge base before evaluation.This protocol separates accumulated knowledge from knowledge changes during evaluation.
- Runtime overhead: Table 7 measures skill and remedy counts, retrieved tokens, retrieval latency, and evaluation-time LLM calls across accumulation checkpoints.The analysis tests whether Curator budget control prevents unbounded context growth.
D.5 Curriculum-style Knowledge Construction on Diamond Tasks
The curriculum-style experiment shows that how behavioral knowledge is constructed strongly affects Diamond-task learning. Lower-tier knowledge improves initialization, while continual mixing with target-task experience produces the strongest sustained performance.
- Experimental Setup: The experiment isolates four knowledge-base initialization and update strategies under a shared planner, execution policy, horizon, retrieval budget, and evaluation protocol.Strategies include Cold Start, Diamond-only Self-learning, Curriculum Pretrain→Freeze, and Mixed Sampling (1:1), evaluated at five checkpoints.
- Results: Cold Start remains below 3% across checkpoints, while Diamond-only Self-learning rises from 3.0% to 13.9%.Cold Start changes only from 2.6% to 2.9%, whereas Diamond-only Self-learning improves steadily but is sample-inefficient early in training.
- Results: Curriculum Pretrain→Freeze consistently outperforms Diamond-only Self-learning and reaches a final success rate of 16.4%.The transferred knowledge includes tool crafting, resource collection, smelting, crafting prerequisites, and common failure-recovery patterns.
- Results: Mixed Sampling (1:1) achieves the highest success rate at every checkpoint, improving from 6.2% to 17.6%.It combines transferable lower-tier knowledge with continuous induction of Diamond-specific skills and remedies, and its advantage remains stable across training.
- Overall Finding: Checkpoint averages confirm that Mixed Sampling has the strongest overall learning efficiency, followed by Curriculum Pretrain→Freeze.The experiment therefore attributes performance differences to the pathway used to construct the knowledge base, not only to episode count.
D.6 Qualitative Case Studies
The qualitative cases show MineEvolve turning execution feedback into locally applicable remedies and reusable skills. The system preserves valid plan prefixes, repairs unfinished subgoals, validates induced knowledge, and reuses it in later planning.
- Failure-Conditioned Local Replanning: A repeated navigation failure with low progress and no inventory change triggers a remedy for local replanning.The remedy inserts a corrective subgoal to break the blocking dirt block before resuming wood collection.
- Failure-Conditioned Local Replanning: The Adaptor preserves the valid plan prefix and repairs only downstream subgoals by clearing the obstacle before continuing toward the target wood.This replaces repeated direct approaches with an executable repair action and continuation step.
- Cross-Episode Knowledge Accumulation: A failed crafting attempt reveals that the target recipe requires a larger crafting grid, so the system induces a prerequisite remedy involving a crafting table.After corrected execution succeeds, the successful process is stored as a reusable skill.
- Cross-Episode Knowledge Accumulation: The Curator validates remedies for state-matchable triggers, executable repair actions, specificity, and conflicts with existing high-confidence knowledge before storage.Validated remedies can later be retrieved for similar tasks.
- Knowledge-Conditioned Planning: The planner uses retrieved skills as procedural guidance and active remedies as constraints while assigning verifiable completion checks to subgoals.Retrieved knowledge is separated into skills and remedies and injected alongside the task, environment state, and current plan prefix.
- Knowledge Representation: Skills encode reusable procedures, while remedies specify triggers, risk patterns, and repair actions for failed or stagnant execution.The Inducer generates these distinct knowledge types from successful versus failed feedback rather than producing generic reflections.