Dreaming in Code for Curriculum Learning in Open-Ended Worlds

• Open-ended agents commonly show rapid early gains followed by performance plateaus because suitable experience is lacking, making continual shaping of novel and learnable experiences necessary.These challenges motivate curriculum mechanisms for sustained improvement.Source: §1 · p. 1
• Existing UED methods often tune low-dimensional parameters under smooth design-space assumptions, limiting their ability to orchestrate structurally evolving environments with long-horizon dependencies.Programmatic environment representations relax these limits, but prior approaches focus on isolated challenges rather than coordinated curricula.Source: §1 · pp. 1–2
• DiCode’s generation cycle selects learnable archive levels, conditions synthesis on the parent level and agent competence, compiles generated code, and mixes accepted levels with target and archived environments for training.Agent performance and new levels update the archive, closing the curriculum loop.Source: §1 · p. 2
• DiCode uses a foundation model to synthesize executable environment-generation logic conditioned on current capabilities, while a fixed world engine preserves valid physics and consistent mechanics.Dreaming constructs a curriculum rather than improving sample efficiency through a learned world model.Source: §1 · p. 2
• DiCode improves mean return by 16% and achieves non-zero success on late-game combat tasks where standard RL and prior UED methods have 0% success.The framework enables long-horizon skills that remain intractable for the cited baselines.Source: §1 · p. 2
• Ablations show that feedback-guided curriculum generation is essential, because the foundation model alone fails to sustain progress.The framework is instantiated in Craftax using an open-weights foundation model, supporting non-proprietary and reproducible generation.Source: §1 · p. 2

2.1. Problem Setting

• The problem is modeled as a UPOMDP whose level-dependent rewards, transitions, and initial-state distributions define distinct POMDPs, with agents maximizing expected discounted return across training levels (Sutton & Barto, 2018; Dennis et al., 2021).Policies may use hidden state and goals to address partial observability and differing reward functions.Source: §2.1 · pp. 2–3

2.2. Unsupervised Environment Design

• UED casts curriculum generation as a teacher selecting levels against a student, while learnability prioritizes partially mastered levels using p(1 − p), unlike PVL and MaxMC.Rutherford et al. (2024) found that PVL and MaxMC correlate poorly with learnability, motivating the paper’s adoption of the learnability score.Source: §2.2 · p. 3

2.3. Prioritized Level Replay

• Prioritized Level Replay alternates between generating parameterized levels and replaying high-scoring levels from a fixed-size buffer, using score- and usage-weighted sampling.The replay scheme favors levels with high heuristic scores or long sampling staleness, controlled by temperature β and trade-off parameter τ.Source: §2.3 · p. 3

3. Dreaming in Code

• DiCode scaffolds mastery of a fixed target environment by generating executable levels tailored to the agent’s current skill frontier and integrating them into training.The generated levels bridge initial capabilities and target-environment demands in a closed-loop curriculum.Source: §3 · p. 3

3.1. Environment Search Space

• Each generated level is executable code defining a distinct POMDP with its own transition dynamics, stochastic initial-state distribution, and goal, rather than merely a fixed procedural layout.In Craftax, code can alter world topology, entities, inventories, combat and progression rules, and achievement-based goals.Source: §3.1 · p. 4

3.2. Generation Cycle

• Each generation cycle selects an archive parent, prompts the foundation model for a level description and executable Python program, and compilation-checks surplus candidates through short agent trajectories.Failed code is filtered without iterative self-correction because refinement costs outweigh its marginal yield.Source: §3.2 · pp. 4–5

3.3. Training

• Training batches combine target, newly generated, and archived levels, reserving 20% of each update’s simulation budget for the target environment while introducing new levels every 2 iterations.Archived-level replay uses PLR-based learnability, and asynchronous generation allows training to continue concurrently unless valid levels are unavailable for v −1 cycles.Source: §3.3 · p. 5

4.1. Setup

• Craftax tests sustained learning across an infinite procedurally generated world with hierarchical dependencies, using PPO-GTrXL as the shared agent across DiCode and UED baselines.Comparisons include PLR, SFL, Domain Randomization, and default-distribution PPO-GTrXL.Source: §4.1 · p. 5

4.2. Results

• DiCode achieves 11% and 9% success on Defeat Gnome Warrior and Defeat Gnome Archer, where baselines reach 0%, and doubles Make Diamond Sword success from 3% to 6%.The results show non-zero performance on late-stage combat tasks and improved learning of deep hierarchical dependencies.Source: Figure 3 · p. 6, §4.2 · p. 6
• 48.33 final mean return, versus 41.54 for the strongest baseline, gives DiCode a statistically significant lead maintained throughout training and a ∼16% relative improvement.Evaluation uses a held-out test set of 1024 unseen procedurally generated worlds, with results aggregated across 5 random seeds.Source: Figure 2 · p. 5, §4.2 · p. 6, §4.1 · p. 6
• DiCode overcomes hierarchical exploration bottlenecks, raising Make Iron Armour success from 14% to 45% and Gnomish Mines entry from 9% to 30%.These instrumental milestones improve survivability and enable deeper exploration beyond prerequisites that challenge the baselines.Source: §4.2 · p. 6
• The generated curriculum progresses from scaffolded early levels to more complex combat and deep-exploration tasks, while average success across training levels remains approximately 0.5.Global progression extends prior concepts and targets late-game bottlenecks; locally, the model removes resource and workstation scaffolding when agent competence is high.Source: Figure 4 · p. 7, §4.2 · p. 7
• 40.91 final score for DiCode-OL is ∼15% below DiCode’s 48.33 and comparable to PPO-GTrXL’s 41.54, showing that closed-loop grounding drives the gains.DiCode-OL withholds parent-level descriptions and agent-performance profiles, instead selecting bottlenecks from scratch using only the static environment description.Source: §4.2 · pp. 7–8

Automatic Curriculum Learning

• DiCode extends automatic curriculum and UED principles from fixed, low-dimensional parameter spaces to expressive, unbounded executable code for a single generally capable agent.It incorporates PLR prioritization and replay, learnability-based curation, and AC-CEL-inspired mutation while differing from POET’s co-evolution of environment-agent populations.Source: §5 · p. 8

FMs in Reinforcement Learning

• Unlike OMNI and OMNI-EPIC, which curate interesting tasks or discover isolated behaviors, DiCode synthesizes environment sequences that scaffold learning and bridge competence gaps.Its focus is sustained progression in open-ended worlds rather than isolated behavior discovery.Source: §5 · p. 8
• DiCode evolves high-level task semantics and progression logic, whereas GenSim and Eurekaverse primarily target low-level motor control and physical robustness.This positions code-level generation as a mechanism for constructing strategic curricula rather than only diverse manipulation tasks or physical terrain.Source: §5 · p. 8
• Compared with EnvGen’s structured JSON configurations and agent-optimization methods, DiCode uses foundation models as environment architects that dynamically shape level distributions for standard reinforcement learning.The code-based approach offers greater expressivity and flexibility than structured configurations while avoiding automated reward design, direct decision-making, or hierarchical skill decomposition.Source: §5 · p. 8

Contributions

• DiCode enables long-horizon skill acquisition in Craftax by generating intermediate training worlds that make otherwise unreachable behaviors learnable.These skills remain invisible to agents trained under standard regimes.Source: §6 · p. 8

Limitations

• The fixed game engine preserves physical validity but limits invention to configuring existing mechanics, while large foundation-model inference adds latency compared with simpler UED methods.Faster inference techniques and more efficient models may narrow this cost gap.Source: §6 · p. 8

Broader implications

• DiCode treats foundation models as tools for generating experience, aligning with calls for interaction-driven learning beyond the boundaries of human data (O. Stanley et al., 2017; Clune, 2020; Hughes et al., 2024; Silver & Sutton, 2025).Following Faldor et al. (2025), foundation models can generate computable environments through a Turing-complete programming language.Source: §6 · p. 8

Future challenges

• Foundation models can reduce the effective search space by sequencing environments according to learning signals, but reliably identifying useful stepping stones remains an open challenge.Solving this problem is important for realizing experience-driven open-ended learning.Source: §6 · p. 9

• Executable-code environments provide a framework for sustaining learning progress in complex open-ended domains where standard training fails.The framework enables agents to learn from automatically synthesized environments and advances scientific study of curriculum generation.Source: Impact Statement · p. 9
• Improved benchmark performance with open-weight models and reproducible tools supports transparent and accessible research.Source: Impact Statement · p. 9
• Automated environment and curriculum generation may reduce manual task design and generalize to robotics or game AI, but deployment raises concerns about unintended behavior.The work is limited to simulated environments; responsible scaling requires continued evaluation, safety, and alignment.Source: Impact Statement · p. 9

A.1. MiniCraftax API Interface

• The JAX simulator is wrapped in a standardized interface where generated classes inherit from BaseTask and define task parameters plus world generation.This interface enables foundation models to produce executable environments while preserving a shared task structure.Source: §A.1 · p. 12

MiniCraftax API Interface

• TaskParams exposes tunable mechanics including mob spawning, health and damage, pursuit distance, level-clearing kills, resource depletion, recovery, and plant growth.The LLM modifies these values to adjust game dynamics across MiniCraftax tasks.Source: MiniCraftax API Interface · p. 12
• Each generated task stores static and dynamic parameters, identifies relevant achievements, tracks initially completed achievements, and provides a descriptive label.These fields define the task’s configuration and success-relevant state.Source: MiniCraftax API Interface · p. 12
• Generated subclasses must implement generate_world, constructing a valid EnvState from a random key through the WorldBuilder API or other JAX-compatible code.World generation is therefore an explicit model-controlled component of each environment.Source: MiniCraftax API Interface · p. 12
• Episodes terminate when the timestep limit is reached, the agent dies, or all achievements designated as relevant to the task are completed.The terminal condition combines resource-independent time, death, and achievement-based completion checks.Source: MiniCraftax API Interface · pp. 12–13
• Task success is evaluated as a binary condition requiring every achievement in the task’s relevant-achievement set to be true.The check converts achievement state to booleans, indexes the designated achievements, and applies an all-condition.Source: MiniCraftax API Interface · p. 13

A.2. Seed Tasks

• Four predefined seed tasks initialize the archive around Craftax’s fundamental mechanics: survival, combat, crafting, and resource gathering.The seed tasks are designed to bootstrap the curriculum before further task generation.Source: §A.2 · p. 13

Collecting

• The collecting task targets COLLECT_COAL, starting the player with a wooden pickaxe and sword and placing 5 coal blocks 4-8 tiles away.It also provides three nearby cows and enables needs and mobs with easier settings.Source: Collecting · p. 13

Combat

• The combat task targets DEFEAT_ZOMBIE by giving the player a wooden sword and placing one zombie plus three cows 4-8 tiles away.Needs and melee spawning use default rates, ranged spawning is reduced to 0.2, and health recovery is increased to 5.0.Source: Combat · p. 15
• The combat subsection also defines a wooden-pickaxe task requiring COLLECT_WOOD, PLACE_TABLE, and MAKE_WOOD_PICKAXE.The player starts with a wooden sword and nearby cows, while needs deplete at 0.5 and melee and ranged spawn multipliers are 0.1 and 0.05.Source: Combat · pp. 16–17

Survive

• The survival task requires EAT_COW, COLLECT_DRINK, and WAKE_UP with an empty inventory and three cows placed 4-8 tiles from the player.All player needs are enabled at the default depletion rate, while melee and ranged spawning use easy settings of 0.05.Source: Survive · pp. 17–18

Common configuration

• All methods share the PPO-GTrXL architecture and parameters, differing only in the final value of the learning-rate schedule.The common configuration is documented in Table 1, with the schedule distinction described separately.Source: Table 1 · p. 19, §A.3 · p. 19

SFL

• SFL uses a separate set of method-specific hyperparameters documented independently from the common experimental configuration.These settings are provided in Table 2.Source: Table 2 · p. 19

PLR and DR

• PLR and DR share replay-buffer infrastructure, but DR sets replay probability to 0.0, disabling prioritization.Their method-specific hyperparameters are reported together in Table 3.Source: Table 3 · p. 20

DiCode

• DiCode has dedicated hyperparameters, including worker distributions that vary when newly generated environments join replayed and target environments.These settings are documented in Table 4.Source: Table 4 · p. 20

Foundation model

• The foundation model uses Qwen3-235B through the Hugging Face API with standard text-generation sampling parameters.Foundation-model hyperparameters are listed in Table 5.Source: Table 5 · p. 20

A.4. Infrastructure and Computational Cost

• DiCode requires approximately 48 hours for 2B timesteps, compared with 10.5 hours for PPO-GTrXL, DR, and PLR and approximately 8.5 hours for SFL.The longer duration is attributed to variable inference latency from a foundation model hosted on separate, lower-priority infrastructure, rather than inherent algorithmic complexity.Source: §A.4 · p. 20, Table 6 · p. 20

B.1. Description Generation Phase

• The description-generation phase uses a foundation model to synthesize a natural-language description of the next curriculum level.The generated description is then used to define the next task’s world, mechanics, achievements, and starting state.Source: §B.1 · p. 21

System Prompt: Description Generator

• The system prompt prioritizes downstream competence on original Craftax through focused, learnable tasks that apply forward pressure beyond the parent capability frontier.It uses global achievement success rates rather than local task success alone, and favors one primary change per evolution while avoiding difficulty spikes and unnecessary backtracking.Source: System Prompt: Description Generator · pp. 21–23
• The prompt directs the model to choose among persist, simplify, expand, and vary based on local and global performance, while scaffolding mastered prerequisites through the initial state.Later-context tasks receive consistent inventory, tools, resources, and completed achievements so training focuses on the new dependency rather than prerequisite backtracking.Source: System Prompt: Description Generator · pp. 23–24

B.1.2. INJECTED KNOWLEDGE BASE MODULES

• The injected Craftax knowledge base supplies immutable game rules and mutable API documentation, including achievement, block, and action definitions used to specify generated tasks.The prompt also requires precise coordinates, quantities, block types, mob identifiers, relevant achievements, completed achievements, world changes, and non-default mechanics.Source: System Prompt: Description Generator · p. 23, §B.1.2 · p. 24, Module: Core Definitions (constants.py) · pp. 24, 26, 27

Module: Mob Definitions (mobs.py)

• Mob definitions encode each creature’s name, behavior type, attributes, and floor assignments in a structured table.The table includes passive, melee, and ranged mobs with health, damage, defence, and floor(s) fields.Source: Module: Mob Definitions (mobs.py) · p. 27

Module: Game Mechanics (step fn nl.py)

• Game-mechanic logic is fixed, but TaskParams values such as mob_damage_multiplier can be changed through the API to control difficulty.The interface permits parameter changes without rewriting action-processing logic.Source: Module: Game Mechanics (step fn nl.py) · p. 28
• Dungeon progression requires descending through valid ladders and, unless god_mode is enabled, killing the required monsters; first-time deeper-level visits grant +1 player_xp.Ascending similarly requires a valid ladder or god_mode, and successful transitions reposition the player at the corresponding ladder.Source: Module: Game Mechanics (step fn nl.py) · p. 28
• Crafting depends on station proximity, resources, triggered actions, and upgrade constraints, with iron tools, weapons, and armour requiring both a crafting table and furnace.Recipes cover tools, weapons, armour, arrows, and torches, with armour filling the first available slot below the target tier.Source: Module: Game Mechanics (step fn nl.py) · pp. 28–29
• Interactions implement combat-prioritized actions, resource gathering, block placement, ranged attacks, spells, randomized potion effects, spell learning, enchantment, boss updates, and XP-based stat increases.These actions are governed by explicit prerequisites, costs, state updates, and achievement triggers, including randomized potion color-to-effect mappings.Source: Module: Game Mechanics (step fn nl.py) · pp. 29–30
• Potion mechanics randomize the mapping from potion color to effect, producing health, mana, or energy changes when the corresponding color is consumed.The six effects include +8 or -3 changes to each affected stat.Source: Module: Game Mechanics (step fn nl.py) · p. 31
• Game mechanics support attribute progression by requiring at least 1 XP and decrementing XP by 1 per valid increment, while collision-aware movement updates direction even when blocked.Source: Module: Game Mechanics (step fn nl.py) · pp. 32–33

Module: World Generation (world gen nl.py)

• World generation creates nine vertically stacked levels using fractal-noise or dungeon-room procedures, with distinct terrains, resources, lighting, mobs, ladders, and randomized potion mappings.The player starts in the Overworld with level-one attributes, full status bars, and an empty inventory unless god_mode is enabled; entities and plants begin inactive or empty.Source: Module: World Generation (world gen… · pp. 36–37

Module: API Documentation (minicraftax api.py)

• The API separates TaskParams, which modify dynamic rules, from WorldBuilder, which configures starting floors, player attributes, and inventory.TaskParams can scale spawning, mob health and damage, detection distance, level-clear requirements, needs depletion, health and mana recovery, health loss, and plant maturation; WorldBuilder also clamps starting attributes between 1 and 5.Source: Module: API Documentation (minicraftax… · pp. 37–39
• The API documentation exposes builder operations for starting equipment and spells, monster-kill progress, block placement, randomized resources, nearby mobs, and final world construction.These methods include deterministic, area-based, random, proximity-based, and adjacency-based world modifications, ending with build returning EnvState.Source: Module: API Documentation (minicraftax… · pp. 39–40

B.1.3. USER PROMPT AND PERFORMANCE FORMATTING

• The current generation prompt provides the parent level λp and agent statistics perfp and perftarget for the specific generation cycle.Listed outcomes are COLLECT_WOOD at 100.00%, PLACE_TABLE at 85.50%, MAKE_WOOD_PICKAXE at 45.00%, and EAT_COW at 12.00%.Source: §B.1.3 · p. 40, Example: Input Performance Profile · p. 41

User Prompt Template

• The user prompt instructs the model to create a meaningful task variation or extension using both training-task and global-agent performance reports.The prompt distinguishes skills learned during the specific task from the agent’s general skill set across all tasks.Source: User Prompt Template · p. 41

B.1.3. USER PROMPT AND PERFORMANCE FORMATTING

• The input-formatting layer converts raw JAX metrics into natural-language performance summaries that populate the prompt placeholders dynamically across curriculum iterations.The trained-task profile records skills achieved during that task, while the global profile records skills learned across all tasks.Source: User Prompt Template · p. 41

B.2. Code Generation Phase

• The code-generation phase uses a second foundation-model inference with a JAX Expert persona to synthesize executable JAX code from task descriptions and library context.The generated code is intended to implement the task in the MiniCraftax runtime.Source: §B.2 · p. 41

System Prompt: Craftax Coder

• The Craftax coder prompt requires a complete Python class inheriting from BaseTask, with task metadata, parameters, and a WorldBuilder-generated EnvState.It requires relevant and completed achievements, a label, TaskParams, and a generate_world method that follows the task’s world description.Source: System Prompt: Craftax Coder · pp. 41–42

B.2.2. INJECTED KNOWLEDGE BASE MODULES

• The coder receives injected source for the underlying MiniCraftax and Craftax libraries plus mob information, ensuring generated code follows the runtime’s available interfaces.The prompt explicitly forbids inventing new functions or classes and requires use of the documented APIs.Source: System Prompt: Craftax Coder · pp. 41–42, §B.2.2 · p. 43

Module: MiniCraftax Library (Injected into {MINICRAFTAX CODE})

• The injected MiniCraftax state model represents task parameters and EnvState fields covering maps, player status, mobs, projectiles, plants, spells, enchantments, boss progress, and achievements.TaskParams includes configurable spawn, combat, needs, recovery, ladder, and plant-growth parameters.Source: Module: MiniCraftax Library (Injected… · pp. 43–44
• The MiniCraftax task interface provides world generation plus termination and success logic based on timesteps, player death, boss defeat, task completion, and weighted achievements.The default achievement-based success logic compares weighted achievements attained against the relevant possible achievements.Source: Module: MiniCraftax Library (Injected… · p. 46
• Smooth-world generation combines player-proximity maps and fractal noise to assign water, coasts, mountains, paths, caves, trees, ores, lava, and optional diamonds.The routine also places the player on grass and returns block, item, lighting, and ladder maps.Source: Module: MiniCraftax Library (Injected… · pp. 47, 49
• Dungeon generation creates a single base level with a chunked occupancy map, initializes eight rooms sized between 5 and 10, and adds corner torches and chests.Source: Module: MiniCraftax Library (Injected… · pp. 49–50
• The dungeon routine connects selected rooms with horizontal and vertical path segments, updating the map and tracking included rooms.Path construction uses source and sink room positions to create connecting corridors along both axes.Source: Module: MiniCraftax Library (Injected… · pp. 51–52
• The library defines JIT-safe random placement routines that sample valid map cells, optionally constrained by Manhattan distance from a target, and batch-update selected locations.Invalid selections remain unchanged, including when the requested count exceeds available valid locations.Source: Module: MiniCraftax Library (Injected… · pp. 53–54
• The programmatic world builder initializes blank multi-level state, generates smooth terrain and dungeons, and combines them into a nine-level base world.World state includes maps, items, lighting, ladders, player state, mobs, plants, and other progression structures.Source: Module: MiniCraftax Library (Injected… · pp. 55–57
• Builder methods set the player’s starting floor and level, choose ladder or current position as appropriate, configure inventory, and clamp dexterity, strength, and intelligence to [1, 5].Source: Module: MiniCraftax Library (Injected… · p. 57
• The interface supports progression and spatial edits by setting monsters killed per level, placing individual blocks, and filling rectangular map areas with chosen block types.Monster counts are clamped non-negative and provide granular control over unlocking subsequent levels.Source: Module: MiniCraftax Library (Injected… · p. 58
• The MiniCraftax library supports JAX-compatible placement of blocks randomly, near target positions, or adjacent to existing blocks across specific levels.It enforces distance constraints, valid underlying blocks, and map updates through pure helper functions and array operations.Source: Module: MiniCraftax Library (Injected… · pp. 58, 59, 61
• MiniCraftax also provides targeted mob spawning that selects valid locations near a target while respecting available slots, mob types, and distance masks.Separate handling updates melee, ranged, and passive mob arrays and their corresponding map slices.Source: Module: MiniCraftax Library (Injected… · pp. 59–61
• The MiniCraftax builder assembles a multi-level EnvState with maps, inventory, progression variables, entities, projectiles, and initialized player and world state.The build process splits random keys, creates projectile arrays, initializes gameplay attributes, and returns the complete state object.Source: Module: MiniCraftax Library (Injected… · pp. 62–63

Module: Craftax Core (Injected into {CRAFTAX CODE})

• The Craftax core defines observation dimensions, block, item, action, and projectile enumerations, together with floor-specific mob counts and encounter parameters.These constants cover the nine-floor progression from the overworld through the boss floor.Source: Module: Craftax Core (Injected into… · pp. 63–65
• Craftax core constants further specify terrain collisions, mob damage, and floor-dependent passive, melee, and ranged statistics.The mappings encode different interactions and combat strengths across floors, including specialized aquatic, amphibian, and flying collisions.Source: Module: Craftax Core (Injected into… · pp. 66–67
• The injected Craftax core defines achievements, progression mappings, environment state, inventories, mobs, projectiles, resources, combat, and other mechanics used by generated tasks.The achievement mapping groups milestones into progression levels, while the environment state tracks player attributes, world entities, combat, spells, enchantments, boss progress, and achievements.Source: Module: Craftax Core (Injected into… · pp. 69–70

Module: Mob Definitions (Injected into {MOBS})

• Mob definitions enumerate passive, melee, and ranged entities together with their type IDs and permitted dungeon levels.The listed entities include cows, bats, snails, zombies, gnome warriors, orcs, lizards, knights, trolls, skeletons, elementals, and other level-specific mobs.Source: Module: Mob Definitions (Injected into… · p. 75

B.2.3. FEW-SHOT EXAMPLES

• Few-shot prompting supplies valid Description →Code examples to enforce the expected class structure and API usage.The examples are included as references for implementing new tasks in the required format.Source: §B.2.3 · p. 75

Few-Shot Example: Collect Coal

• The collect-coal scaffold starts the player on the overworld with a wooden pickaxe and sword, places five coal blocks 4–8 tiles away, and enables needs with slower depletion.Its objective is COLLECT_COAL, with MAKE_WOOD_PICKAXE and MAKE_WOOD_SWORD completed as prerequisites; needs_depletion_multiplier=0.5.Source: Few-Shot Example: Collect Coal · pp. 75–76

B.2.4. USER PROMPT (CODE REQUEST)

• The user-prompt stage inserts the latent task description produced earlier as the code-generation request’s task description.This trigger initiates the code-generation step before the complete Python file is requested.Source: §B.2.4 · p. 76

User Prompt: Code Generation

• The code-generation prompt asks for a complete, correct Python task file, provides the task description and reference examples, and instructs the model to follow the system prompt.The examples are explicitly supplied for style and structure before the final generation request.Source: User Prompt: Code Generation · p. 77

B.3. Open-Loop Ablation (DiCode-OL)

• DiCode-OL removes the curriculum feedback loop, so task generation uses only the static environment description without the parent level or agent performance profiles.Source: §B.3 · p. 77

System Prompt: Open-Loop Generator

• The system prompt defines the generator as an expert curriculum designer whose objective is to create challenges that improve performance on the full original Craftax game.Source: System Prompt: Open-Loop Generator · p. 77
• The prompt prioritizes focused, learnable pressure on a randomly chosen bottleneck because overly difficult tasks or tasks bundling fragile requirements may be discarded after brief training.Source: System Prompt: Open-Loop Generator · p. 77
• Initial states should compress prerequisites for later-game capabilities by providing consistent inventory, tools, and resources while avoiding backward travel unless navigation itself is being trained.Source: System Prompt: Open-Loop Generator · p. 78
• The generator is instructed to sample a specific Craftax mechanic, transition, or survival loop and isolate it as a thin-slice training task.Source: System Prompt: Open-Loop Generator · p. 78
• The prompt separates Relevant Achievements, which define episode success, from Completed Achievements, which represent goals already satisfied by the initial world state.Source: System Prompt: Open-Loop Generator · pp. 78–79
• The required reasoning output asks the generator to justify capability selection, explain start-state scaffolding and backtracking avoidance, and verify thin-slice focus and achievement consistency.Source: System Prompt: Open-Loop Generator · p. 79

User Prompt: Open-Loop Generator

• The user prompt clarifies that the model must produce a new creative task description rather than code, consisting of a reasoning section followed by a detailed task docstring.Source: User Prompt: Open-Loop Generator · p. 80

C.1. Curriculum Case Studies

• The four representative levels illustrate progressive scaffolding from basic survival toward deep exploration in the generated curriculum (Figure 4).The examples are described through executable Python-class docstrings.Source: §C.1 · p. 80
• The first level isolates iron-armor crafting and dungeon descent by guaranteeing resources, pre-placing workstations, unlocking the ladder, and reducing mob pressure.The agent starts with 3 iron, 3 coal, and a tier-2 stone pickaxe; iron is guaranteed at fixed coordinates, while melee and ranged spawn multipliers are 0.3 and 0.1.Source: §C.1 · pp. 80–81
• The second level generalizes armor crafting by requiring natural collection of coal and iron, workstation placement, and descent under moderate mob pressure.The agent begins with 5 wood, 5 stone, and a tier-1 wood pickaxe; coal and iron occur with 3% and 2% generation probabilities, while the ladder remains pre-unlocked.Source: §C.1 · p. 82
• The third level integrates armor crafting into survival by requiring eight hostile-mob kills before dungeon descent, with increased melee and ranged pressure.The task teaches the sequence gather resources, craft armor, clear the floor, then descend; melee and ranged spawn multipliers are 1.2 and 0.7.Source: §C.1 · pp. 83–84
• The fourth level targets combat-ready progression by forcing iron-sword prioritization before Gnomish Mines entry and nearby Gnome encounters on deeper floors.Starting on Floor 1 with exactly 1 iron and 1 coal makes sword and pickaxe construction mutually insufficient, while Gnome Warriors and a Gnome Archer introduce the parent task’s neglected combat dependency.Source: §C.1 · p. 85
• The task targets entering the Gnomish Mines and defeating a Gnome Warrior after collecting resources and crafting stone tools.Its achievement label includes collecting wood, stone, and coal; placing a table and furnace; making stone pickaxes and swords; entering the mines; and defeating a Gnome Warrior.Source: §C.1 · p. 86
• The generated world starts on Dungeon floor 1 with wood, stone, coal, iron, two stone pickaxes, and two stone swords, plus a crafting table and furnace at fixed positions.The setup also records eight monsters killed on floor 1, leaving the ladder down already unlocked.Source: §C.1 · p. 86
• On floor 2, two level-2 melee Gnome Warriors are placed randomly on PATH or GRASS blocks within Manhattan distance 3–6 of ladder-up position (24, 24).Source: §C.1 · pp. 86–87

D.1. Full Achievement Breakdown

• The evaluation provides a comprehensive assessment of agent performance across the Craftax achievement hierarchy.This establishes the scope of the additional quantitative results.Source: §D.1 · p. 88
• Aggregate success rates compare DiCode with baselines across all defined Craftax achievements.Figure 5 presents the final achievement success-rate comparison.Source: Figure 5 · p. 88
• The evaluation reports means and standard errors across five random seeds after 2 × 10^9 steps.These statistics define the aggregation and evaluation horizon for the reported results.Source: §D.1 · p. 88
• Achievement learning curves show the rate at which individual achievements are acquired over time.Figure 6 provides time-series performance for individual achievements.Source: Figure 6 · p. 89
• Shaded regions in the learning curves denote standard error across the five seeds.The uncertainty visualization applies to the achievement-specific time-series results.Source: §D.1 · p. 89

D.2. Ablation Analysis

• Final mean return and standard error are reported for the ablation analysis.The table specifies N = 5, 2.Source: Table 7 · p. 90
• DiCode is evaluated against the Open-Loop DiCode-OL ablation to test the importance of closed-loop curriculum control.Source: §D.2 · p. 90
• The held-out test comparison shows a performance gap favoring closed-loop feedback, highlighting the need to condition generation on current capabilities and parent levels.Mean episode return is compared across the respective seed counts, with shaded regions indicating standard error.Source: Figure 7 · p. 90