OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness with Environments Programmed in Code

1 INTRODUCTION

• Open-ended algorithms aim to generate and solve new challenges endlessly, drawing motivation from biological evolution’s role in producing general intelligence (Clune, 2019).The paper frames this as a route toward AI-generating algorithms and machine creativity.Source: §1 · p. 2
• Prior open-ended algorithms used limited world types and predefined parameters, while diverse-task data remains a bottleneck for advancing AI (Sudhakaran et al., 2024; Wang et al., 2023a; 2019; 2020; Zhang et al., 2023; Jiang et al., 2023).The introduction identifies generating any possible learning environment—Darwin Completeness—as essential, but warns that searching this space may be intractable.Source: §1 · p. 2
• OMNI-EPIC uses foundation models to select interesting, learnable tasks and generate code for their environments, reward functions, and termination functions, enabling any simulatable learning task in principle.It builds tasks within pre-existing simulators; the same simulated world can support different tasks through different reward functions.Source: §1 · p. 2
• OMNI-EPIC uses a model of interestingness to select tasks and check novelty against an archive, and a success detector to identify task completion.The proposed tasks must be novel, interesting, solvable, and matched to agents’ current capabilities.Source: §1 · p. 2
• Although Python’s Turing completeness motivates a vision of generating any computable environment, this work constrains code generation to the PyBullet simulator.The broader vision includes modifying existing simulators or writing new ones.Source: §1 · pp. 2–3

Unsupervised Environment Design

• Unsupervised curricula rely on predefined task distributions (Dennis et al., 2020; Jiang et al., 2021; Parker-Holder et al., 2022; Samvelyan et al., 2023; Wang et al., 2019; 2020), motivating OMNI-EPIC’s human-modeled interestingness (Zhang et al., 2023).Learning progress and regret can guide task selection, but learnable environments may still be uninteresting or trivial variants.Source: §2 · p. 3

Foundation Models for Environment Design

• Prior foundation-model designs generate reward code or simulations (Ma et al., 2023; Wang et al., 2023b), but OMNI-EPIC targets adaptive auto-curricula spanning environments and tasks.Other approaches focus on domain-specific levels or fixed task sets, without adapting difficulty to agent progress or supporting truly open-ended generation (Sudhakaran et al., 2024; Todd et al., 2023; Bruce et al., 2024; Zala et al., 2024; Wang et al., 2023c).Source: §2 · p. 3

Foundation Models in Open-Endedness

• FM-based open-ended approaches can be limited by fixed archives or predefined environmental conditions (Bradley et al., 2023; Ding et al., 2023; Lehman et al., 2023; Lim et al., 2024; Zhang et al., 2023; Wang et al., 2023a), whereas OMNI-EPIC generates worlds and rewards.This broader generation may expose agents to a wider range of challenges and learning opportunities.Source: §2 · p. 3

3.1 TASK ARCHIVE

• The unbounded task archive stores natural-language descriptions and executable environments for learned and failed tasks, using both as stepping stones toward novel, increasingly complex but learnable challenges.It is initialized with a few natural-language task seeds and expands as tasks are generated and learned.Source: §3.1 · p. 4

3.2 TASK GENERATOR

• An LLM proposes novel, learnable task descriptions using similar archived successes and failures as stepping stones, while a foundation-model-derived notion of interestingness guides generation.Because context is limited, the generator retrieves a predefined number of tasks similar to a randomly selected archived task rather than conditioning on the full archive.Source: §3.2 · pp. 4–5

3.3 ENVIRONMENT GENERATOR

• An LLM converts each natural-language task into Python environment code with reset, step, reward, and termination functions; compilation failures trigger up to five repair attempts before the task is discarded.If code still fails, the task generator proposes another task, potentially less complex or differently structured.Source: §3.3 · p. 5

3.4 POST-GENERATION MODEL OF INTERESTINGNESS

• A post-generation model of interestingness compares each task and its environment code with similar archived tasks, training an agent only if the new task is judged interesting.The check evaluates qualities such as novelty, surprise, diversity, and worthwhileness; tasks judged uninteresting are discarded.Source: §3.4 · p. 5

3.5 TRAINING AGENTS WITH REINFORCEMENT LEARNING

• OMNI-EPIC trains one specialist DreamerV3 agent per task in PyBullet, continuing newly generated tasks from the successfully learned policy with the closest task embedding.Agents use proprioceptive and visual input and, in this work, an R2D2 robot with six discrete actions; archive-initializing tasks are trained from scratch, and environment color variation provides domain randomization.Source: §3.5 · p. 6

3.6 SUCCESS DETECTOR

• An LLM-generated success-checking function determines whether the agent completed a task, separately from the reward function that shapes efficient learning; unlearned tasks are revised and retried before being archived as failures.The authors use generated code rather than a VLM because preliminary testing found current VLMs insufficiently accurate for success detection.Source: §3.6 · p. 6

• The long run shows 200 iterations with the R2D2 robot, excluding tasks without executable code and assuming all generated tasks can be completed rather than training RL agents.Tasks were embedded and projected into two dimensions for visualization; Figure 2 shows selected tasks and, for readability, only the closest parent connection for each child.Source: §4 · pp. 6–7, Figure 2 · p. 7
• OMNI-EPIC generated tasks far beyond the seeds, including dynamic objects and interactions such as moving platforms, buttons, levers, and obstacles.It also explored different niches and generated variations within them.Source: Figure 2 · p. 7, §4 · pp. 7–8
• Generated tasks evolve and diverge across the embedding space, from ball-kicking challenges to object delivery and navigation across platforms or varied terrain.Ball-kicking tasks also gain dynamic elements such as moving obstacles or targets.Source: §4 · p. 7
• Within task niches, OMNI-EPIC varies environments, object counts, and time limits; for example, retrieval tasks span outdoor settings, connected rooms, and multiple levels.Task 102 requires retrieving multiple objects, while task 90 requires one; tasks 27 and 24 allow 3 and 5 minutes, respectively.Source: §4 · p. 7

5 SHORT RUN WITH LEARNING

• In the Figure 3 run, agents completed 16 tasks, failed 6, and one task was deemed uninteresting; human evaluations aligned with the success detector 72.7%.The figure marks learned tasks with checks, failures with crosses, and shows task modifications made after failures.Source: §5 · p. 8, Figure 3 · p. 8
• The five short runs were initialized with 3 task-description seeds each and were kept short because of limited computational resources.The authors report that these runs still demonstrate OMNI-EPIC’s creative potential and ability to tailor tasks to agent abilities.Source: §5 · p. 8
• OMNI-EPIC built a curriculum from successes and failures: it combined learned bridge skills in task 4, omitted object-pushing after task 9 failed, and reduced obstacles in task 13.Task 4 combined gaps and moving bridge segments; after failure to push a box on a dynamic platform, subsequent platform-crossing tasks omitted object-pushing.Source: §5 · pp. 8–9

Measure of Progress

• OMNI-EPIC significantly outscored both controls on ANNECS-OMNI (p < 0.05, Mann-Whitney U test), with scores increasing consistently without observed stagnation during affordable runs.Each method was run with RL training and repeated 5 times.Source: Figure 4 · p. 9, §6 · pp. 9–10

Task Diversity

• OMNI-EPIC achieved significantly higher task-archive cell coverage than both controls (p < 0.05, Mann-Whitney U test), showing that the archive and MoI contribute to diversity.Each method was run with simulated learning and repeated 3 times.Source: Figure 4 · p. 9, §6 · p. 9

Measure of Progress

• ANNECS-OMNI extends ANNECS by requiring new tasks to be interesting relative to previously solved tasks, approximated by asking an FM to judge against the solved-task archive.The authors introduce this criterion because ANNECS does not measure whether newly generated tasks are interesting or novel.Source: §6 · p. 10

7 DISCUSSION, FUTURE WORK, CONCLUSION

• OMNI-EPIC uses foundation models to generate diverse task and environment code without handcrafted parameters or predefined task distributions, adapting environments to agents’ developing capabilities.The authors describe this as a step toward Darwin Completeness, or the ability to create any possible learning environment.Source: §7 · p. 10
• The authors cannot rule out similarity to training data, but report finding no identical environments online and hypothesize that longer runs could generate environments endlessly.Whether the task generator and model of interestingness generalize beyond human-data distributions as the discovery archive grows remains future research.Source: §7 · p. 10
• The current implementation is not Darwin Complete because its simulator constrains generated environments; arbitrary code generation, including creating or modifying simulators, could broaden its scope.Python is Turing Complete, so code generation could in principle create any computable environment, though taking advantage of this requires smarter foundation models.Source: §7 · p. 10
• The work trains specialist-agent populations and does not demonstrate cross-task generalization; future studies could train one policy across environments (Adaptive Agent Team et al., 2023) or prioritize tasks by learning progress (Baranes & Oudeyer, 2013; Kanitscheider et al., 2021).The effects of these approaches on agent performance and generalization remain open questions.Source: §7 · p. 10
• OMNI-EPIC could also create entertainment and educational resources, from math and poetry challenges to games and virtual worlds, for human and AI agents to explore.The authors frame combining foundation-model expressive power with human notions of interestingness as a promising path toward open-ended, creative AI.Source: §7 · p. 10

B GAME INTERFACE

• OMNI-EPIC’s game interface lets players control the R2D2 robot by keyboard through generated levels that adjust to player skill (Figure 5).The system suggests tasks that are not too easy or difficult, aiming to keep the experience engaging.Source: B GAME INTERFACE · p. 15, Figure 5 · p. 15

C VLM AS SUCCESS DETECTOR

• Current VLMs were not accurate enough in preliminary testing, so the authors use LLM-generated code for success detection while anticipating future VLM improvements.VLMs may cover tasks such as tidying a room or doing a backflip, which can be difficult to evaluate using only code.Source: C VLM AS SUCCESS DETECTOR · p. 16
• The proposed detector receives robot snapshots, the task description, and environment code, then describes the image and judges whether the robot solved the task.Snapshots are provided every second.Source: C VLM AS SUCCESS DETECTOR · p. 16
• The prompt directs the detector to assess whether the environment matches the task and the robot’s behavior meets its success requirements; impossible tasks count as failures.It instructs the detector to make an educated guess when unsure, but also to mark uncertain cases as failed.Source: C VLM AS SUCCESS DETECTOR · p. 16
• The prompt supplies a robot description and task description with environment code, and requests an image description, reasoning, and a Yes/No success judgment.The response format separates the image description, reasoning for success or failure, and final decision.Source: C VLM AS SUCCESS DETECTOR · p. 16

• Figure 6 maps task generations and parent-child conditioning, marking learned tasks with checks and discarded uninteresting tasks with ZZZ symbols.The dense graph’s interactive version is available to help navigate its task relationships.Source: Figure 6 · p. 17
• Task 76’s code specifies a 10-minute cleanup challenge in which a robot sorts debris into correct bins, avoids moving vehicles, and returns to its start.The reward values are 1 for picking up debris, 10 for placing it, 100 for completing the task, and a −1 collision penalty.Source: D SUPPLEMENTARY MATERIALS FOR LONG RUN… · pp. 18–19
• Another task asks a robot to explore an eight-room, two-floor building, find a target, and guide it back to the entrance while avoiding obstacles.Completion requires bringing the target within 2 m of the entrance; rewards encourage exploration, locating the target, and returning it.Source: D SUPPLEMENTARY MATERIALS FOR LONG RUN… · p. 21
• A soccer task progresses from scoring against a stationary opponent to defending against one that chases and kicks the ball.Success requires scoring while preventing the opponent from scoring.Source: D SUPPLEMENTARY MATERIALS FOR LONG RUN… · p. 22
• A further soccer challenge has the robot receive a teammate’s pass and kick the ball past moving obstacles and a goalie into the goal.After a fixed delay, the teammate passes toward a point 3 meters ahead at 3 m/s, while the main robot’s starting position is randomized.Source: D SUPPLEMENTARY MATERIALS FOR LONG RUN… · p. 22

Soccer challenge

• The soccer task requires receiving a teammate’s pass and kicking a moving ball past field obstacles and a tracking goalie without touching obstacles.Three obstacles move randomly, while the goalie tracks the ball within 3 meters.Source: D SUPPLEMENTARY MATERIALS FOR LONG RUN… · pp. 22–23

Busy Restaurant Table Service

• Restaurant service combines setting tables and bussing dishes amid moving patrons, with careful handling, collision avoidance, and sustained success required throughout a 10-minute episode.Busing includes respecting the bin’s capacity and returning it to the kitchen; collisions or broken dishes compromise success.Source: D SUPPLEMENTARY MATERIALS FOR LONG RUN… · p. 23

Bounce Shot Scoring Challenge

• In the bounce-shot challenge, the robot must intercept a ball bouncing unpredictably over ramps and kick it into the goal before it stops or leaves the field.The robot must adapt its approach to the ball’s changing trajectory and time its kick to redirect it.Source: D SUPPLEMENTARY MATERIALS FOR LONG RUN… · p. 24

Outdoor Retrieval Challenge

• Outdoor retrieval requires locating and carrying a ball across varied terrain back to the start, reaching within 2 meters with the ball in possession or balanced on the robot.If dropped or rolled more than 5 meters away, the ball is replaced at a new random location.Source: D SUPPLEMENTARY MATERIALS FOR LONG RUN… · pp. 24–25

Conveyor Belt Item Delivery

• Conveyor delivery challenges the robot to ride a moving belt, sort colored items into matching targets, and deliver at least five items within three minutes.Items appear at random intervals, and the robot returns to the left platform between deliveries.Source: D SUPPLEMENTARY MATERIALS FOR LONG RUN… · p. 25

Moving gates and targets

• A moving-gates task asks the robot to kick a rolling ball through five gates in order and stop it in each corresponding moving target zone to maximize its score.A gate pass scores two points if the ball stops in the correct zone, and one point otherwise.Source: D SUPPLEMENTARY MATERIALS FOR LONG RUN… · p. 26

Dynamic Target Pushing Challenge

• The dynamic-target pushing challenge places three moving spherical targets on linear back-and-forth paths in a flat, open area.The supplied description ends before specifying the task objective.Source: D SUPPLEMENTARY MATERIALS FOR LONG RUN… · p. 26

Moving-target delivery

• The robot must push three moving targets into their corresponding goal zones, with rewards for progress and a five-minute completion limit.The targets may be delivered in any order, but each must come to rest fully within its goal; the episode also ends if the robot flips or a target leaves the bounds.Source: D SUPPLEMENTARY MATERIALS FOR LONG RUN… · pp. 26–27

Obstacle-course ball delivery

• In a second task, the robot must push red and blue balls into their matching goals while avoiding obstacles, and success requires both balls to reach their goals without a collision.Rewards encourage approaching and directing each ball toward its goal; the task ends if the robot or a ball collides with an obstacle, or after 60 seconds.Source: D SUPPLEMENTARY MATERIALS FOR LONG RUN… · pp. 27–28

Coordinated target hitting

• A coordination task asks the robot to kick two moving balls so they hit the front of a laterally moving target simultaneously.The robot is rewarded for approaching the balls, matching their velocities, and kicking them toward the target; the target moves at 0.5 m/s and reverses after traveling 3 meters to either side.Source: D SUPPLEMENTARY MATERIALS FOR LONG RUN… · p. 28

Task graph

• Figure 7 maps task generations, learning success or failure, discarded uninteresting tasks, and which prior tasks conditioned each new-task request.Node colors indicate generation; check marks, crosses, and ZZZ symbols denote learned, failed, and discarded tasks, respectively.Source: Figure 7 · p. 30

Task 1: Moving bridge

• The bridge task asks the robot to time jumps across five independently moving segments to reach an end platform as quickly as possible.The 20 m gap is bridged by 2 m-wide segments moving sinusoidally with 1 m amplitude and 2-second period, phase-offset by 0.4 seconds; rewards encourage staying on the bridge or platforms and reducing distance to the goal.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · p. 30

Task 3: Ball and goal

• The ball task requires navigating around obstacles, reaching a ball, and kicking it between goalposts; obstacle collisions cause termination.Rewards encourage survival, approaching the ball while avoiding obstacles, and kicking toward and into the goal.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · pp. 33–34

Bridge task

• The bridge task requires timing jumps across six independently moving segments, each separated by a 1 m gap, to reach the end platform quickly.Segments move sinusoidally with a 1 m amplitude, 2-second period, and 1/3-second phase offsets; rewards encourage staying on the bridge and reducing distance to the end.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · pp. 36–37

Moving-goal task

• The task rewards robot survival each time step, decreasing distance to the ball, kicking it toward the goal, and successfully kicking the ball into the moving goal.The goal’s motion has a 2-meter amplitude and 10-second period, with its center 10 meters in front of the robot’s starting position.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · p. 40

Task 6

• Task 6 asks the robot to approach a ball and time a kick into a goal that moves side to side in a sinusoidal path.The goal motion has a 2 m amplitude and 10-second period, with its center 10 m in front of the robot.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · p. 43
• Task 6 rewards survival, approaching the ball, kicking toward the goal, and scoring; the task succeeds when the ball passes between the posts and ends if it goes out of bounds or time expires.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · p. 43

Task 7

• Task 7 requires the robot to move a ball across a bridge of six independently moving segments and gaps into a goal, then reach the end platform intact.The segments move sinusoidally with a 1 m amplitude and 2-second period, offset in phase by 1/3 seconds.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · pp. 45–46
• Task 7 rewards approaching and advancing the ball, scoring in the goal, staying on the bridge or platforms, and reaching the end platform; falling or the ball leaving the structure ends the episode.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · p. 46

Task 8

• Task 8’s reward terms track ball approach and forward motion, goal entry, bridge or platform presence, and reaching the platform end; termination checks bridge, ball, and goal conditions.The supplied code fragments define these reward components and termination checks, but do not provide the full task description.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · p. 49

Task 9

• Task 9 asks the robot to reach a target 20 meters ahead through five dynamic obstacles, with survival rewards and collision penalties; collision or target contact terminates the task.The obstacles move side-to-side, vertically, diagonally, or in circular patterns; the task also rewards reducing distance to the target.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · p. 50

Task 10

• Task 10 requires the robot to push a box across a dynamic bridge to a target on the end platform, traversing five independently moving segments.Each segment moves sinusoidally with a 1 m amplitude and 2-second period, phased 0.4 seconds after the previous segment.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · p. 53

Box-and-bridge task

• The box-and-bridge task rewards approaching the box, moving it forward across the bridge, and keeping the robot on bridge segments or platforms; the episode ends if either falls.The reward description also specifies a large reward for placing the box at its target and a reward for the robot reaching the end platform.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · p. 54

Task 10

• In a 10-by-10-meter arena, balls spawn every 2 seconds at 5 meters per second; the robot must survive 60 seconds, with a hit or timeout ending the task.The balls have a radius of 0.25 meters and spawn 1 meter above the arena.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · pp. 56–57

Task 11

• Five moving platforms connect start and end platforms 20 meters apart; the robot jumps across water, earns rewards for staying on platforms and approaching the end, and terminates upon falling or reaching the end.The moving platforms follow sinusoidal motion with a 1-meter amplitude, 3-second period, and 0.6-second phase offsets.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · pp. 58–59

E SUPPLEMENTARY MATERIALS FOR SHORT RUN WITH LEARNING

• Task 12 places start and end platforms 30 meters apart, with randomly placed rocks and trees between them that are separated by at least 1 meter.The task illustration appears in Figure 18.Source: Figure 18 · p. 61, E SUPPLEMENTARY MATERIALS FOR SHORT… · p. 62
• Task 12 rewards staying on the platforms or terrain and reducing distance to the goal, penalizes obstacle collisions, and terminates on a fall or collision.Success is reaching the end platform.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · p. 62
• Task 13 places obstacles in a grid pattern with enough space, incentivizes swift movement towards the goal, penalizes collisions with obstacles, and succeeds when the robot reaches the end platform.Figure 19 depicts Task 13 of the OMNI-EPIC run presented in Section 5.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · p. 65, Figure 19 · p. 65

Task 15

• To cross lava, the robot jumps from static platform to static platform across a series of five static platforms spaced 3 m apart above lava, reaching the end platform.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · p. 68

Task 16

• Between static platforms 3 meters apart, cylinders (radius 0.5 m, height 1 m) move sinusoidally along the y-axis with 2-meter amplitude and 4-second period.The robot must jump between platforms while avoiding the moving obstacles to reach the end platform. Collisions with moving obstacles are penalized.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · p. 71
• Task 16 sends the robot across a 50-meter bridge over water while 0.5-meter-radius logs roll toward it from 10 meters ahead.The bridge is 5 meters wide and 10 meters above the water; logs spawn 2 meters above it and move toward the robot at 5 m/s.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · p. 74
• Task 16 rewards survival on the bridge and progress toward its end, penalizes log collisions, and ends upon falling, colliding, or reaching the end.Success is reaching the bridge end.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · p. 74

Task 17

• Task 17 requires touching a button or lever to lower an upright 25-meter drawbridge, then crossing the 25-meter gap to the end platform.The start and end platforms are each 3 m × 3 m × 0.5 m; the drawbridge is 2 meters wide.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · pp. 76–77
• Task 17 rewards remaining on the platforms, activating the button or lever, and reducing distance to the end; it ends if the robot falls or reaches the end platform.Success is reaching the end platform.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · p. 77

Task 18

• The environment has a static 50-by-50-meter platform, a randomly placed 2-meter, 5-kilogram cube, and a randomly placed target zone of 3-meter radius; the robot starts at a fixed platform position.Success is defined as the cube being entirely within the target zone.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · pp. 79–80

Task 19

• The task uses a 50 x 50 meter flat arena containing a randomly placed 2-meter, 5-kg cube and randomly placed 3-meter-radius non-colliding target; the robot starts randomly, facing the positive x-axis.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · p. 82
• Unlike Task 18, Task 19 does not end when the cube reaches the target; its episode terminates only if the robot falls off the platform.Task 18 terminates when the robot falls or the cube enters the target zone.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · pp. 80, 83

E Supplementary Materials for Short Run with Learning

• The code defines rewards for survival, moving a cube toward a target zone, pushing the cube, and staying within the arena.Termination occurs if the robot leaves the arena; the code also checks whether the cube reaches the target zone.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · p. 85
• A robot must push a domino on a 1000 x 10 x 0.1-meter platform to start a chain reaction among dominos spaced 3 meters apart.Rewards encourage forward progress and falling dominos, while falling off the platform incurs a penalty and terminates the task.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · pp. 85–86
• A robot must knock over a randomly placed tower of 10 one-meter, 5-kg blocks until none remain stacked.The task rewards survival and reducing tower height, with termination when the robot falls or all blocks are knocked over.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · pp. 88–89
• The next task is to push a box next to a wall and climb on top of it.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · p. 90

E SUPPLEMENTARY MATERIALS FOR SHORT RUN WITH LEARNING

• The robot must push a box against a wall, climb onto the box, and then jump onto the wall.The wall is 5 meters from the robot, and the box is 3 meters away; success requires climbing onto the box before jumping onto the wall.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · pp. 90–91
• Rewards encourage the robot to stay on the platform, approach and push the box toward the wall, climb onto the box, and jump onto the wall.The reward description assigns a reward for each time step on the platform and additional rewards for the successive navigation and climbing actions.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · p. 91
• The episode terminates if the robot falls off the platform, but not when it successfully jumps onto the wall.Source: E SUPPLEMENTARY MATERIALS FOR SHORT… · p. 91

• Repeated runs generated significantly different environments despite sharing the same initial seed tasks, suggesting task diversity is not heavily influenced by the initial environments.Source: F ADDITIONAL SHORT RUNS WITH LEARNING · p. 94
• Four additional short runs show task progression and RL-agent learning outcomes, marking successful and failed tasks and iterations on failures.Source: Figure 29 · p. 94, Figure 30 · p. 95, Figure 31 · p. 96

G.1 WITHOUT TRANSFER LEARNING

• 63.2% median success with transfer (CI: 42.1 - 69.5) versus 36.8% from scratch (CI: 31.6 - 53.7) across 25 tasks and five replications, with 2 million steps per task.The settings differed only in whether each new task’s policy was initialized from a checkpoint or randomly.Source: §G.1 · p. 98

G.2 UNIFORM SAMPLING EXAMPLES FROM THE ARCHIVE

• Uniformly sampling archive examples led to learning only 5 generated tasks, fewer than using the most similar tasks as in-context examples in Figure 3.The ablation otherwise used the Figure 3 settings.Source: Figure 33 · p. 98

G ABLATIONS ON SHORT RUNS WITH LEARNING

• Across the short-run ablations, uniform examples, omitting failed examples, and omitting learning progress each reduced learned-task counts relative to Figure 3.The corresponding learned-task counts were 5, 7, and 0, respectively.Source: Figure 33 · p. 98, Figure 34 · p. 99, Figure 35 · p. 100

G.3 WITHOUT FAILED EXAMPLES

• Without failed-task examples, the generator regenerated similar tasks the agent had failed, and only 7 generated tasks were learned, fewer than in Figure 3.The failed tasks had been unsuccessful because of unsuitable reward functions or incorrect environment configurations.Source: Figure 34 · p. 99

G.4 WITHOUT LEARNING PROGRESS

• Without a learning-progress notion in the task-generator prompt, no tasks were learned, unlike in Figure 3, whose prompt included learning progress and interestingness.The ablation otherwise used the Figure 3 settings.Source: Figure 35 · p. 100
• The prompt still asked for interesting, novel, useful tasks with clear task, success, reward, and termination specifications.It also requested rich, diverse environments and engaging tasks.Source: §G.4 · pp. 100–101

H HUMAN EVALUATION SETUP

• Fifty participants reviewed robot-task videos alongside task descriptions to assess completion against the automated success detector.The study aimed to measure agreement between human evaluators and the detector’s assessments.Source: H HUMAN EVALUATION SETUP · p. 102
• Participants used an annotation interface and received instructions for annotating the data.Source: Figure 36 · p. 102
• The study ran on cloudresearch.com, paying participants $0.25 per response, estimated at $15.00 per hour for responses taking under one minute.Source: H HUMAN EVALUATION SETUP · p. 102

Supplementary quantitative results

• OMNI-EPIC achieves significantly higher archive cell coverage than controls across discretization levels 10, 20, 30, and 40 in long runs, with fewer tasks sharing cells.The coverage differences are significant at p < 0.05 by Mann-Whitney U test.Source: Figure 38 · p. 103, Figure 37 · p. 103
• In short runs, archive cell coverage trends are similar, but differences between methods are not always significant, as OMNI-EPIC’s effects become more pronounced over longer runs.The short-run comparison covers discretization levels 10 and 20.Source: Figure 39 · p. 103
• OMNI-EPIC significantly outperforms controls on both ANNECS and ANNECS-OMNI, with no difference between the metrics, indicating that all its learned tasks are considered interesting.Short runs with RL training were repeated five times.Source: Figure 40 · p. 104
• The percentage of tasks learned decreases over attempted tasks, while the gap versus training generated tasks from scratch widens, showing that OMNI-EPIC generates increasingly difficult tasks.Short runs with RL training were repeated five times.Source: Figure 41 · p. 104

J SELECTING THE MOST SIMILAR TASKS

• OMNI-EPIC retrieves the closest archived tasks by cosine similarity between embeddings of their natural-language descriptions and environment code, following Lewis et al. (2020).The authors note that repeated rejection of similar task types could cause cycles, though stochastic generation, environment diversity, and longer context may reduce this risk.Source: J SELECTING THE MOST SIMILAR TASKS · p. 105

K HYPERPARAMETERS

• Hyperparameters are tabulated for OMNI-EPIC and DreamerV3; each R2D2 task is trained for about 1 hour on 2 NVIDIA RTX 6000 Ada Generation GPUs and 32 CPU cores.Source: Table 1 · p. 106, Table 2 · p. 106, K HYPERPARAMETERS · p. 106

L.1 TASK GENERATOR

• The task-generator prompt asks for a realistic, simulatable task suited to the robot’s current skills that is novel, useful, creative, and not too difficult.It supplies learned and failed tasks with environment code as context and excludes tasks requiring grasping, carrying, stacking, or maze navigation.Source: §L.1 · p. 107

L.2 ENVIRONMENT GENERATOR

• The environment-generator prompt requires code aligned with the task, with dense, optimizable reward components in the range −10 to 10 and methods for reset, stepping, termination, and success.Source: §L.2 · p. 108

L.3 ENVIRONMENT GENERATOR REFLECTION

• The reflection prompt asks the model to diagnose an execution error and return improved environment code without errors.Source: §L.3 · p. 109

L.4 POST-GENERATION MODEL OF INTERESTINGNESS

• The post-generation evaluator judges a task interesting if it is novel, creative or surprising, fun to watch, useful, or appropriately challenging for the robot.Source: §L.4 · p. 110

M FEW-SHOT EXAMPLES

• A few-shot example specifies a robot crossing a pride-colored bridge between platforms 30 m apart, with rewards for remaining on the route and reducing distance to the goal.Reaching the end platform counts as success; falling off any platform or bridge segment terminates the task.Source: M FEW-SHOT EXAMPLES · p. 111
• The boat task has a robot board a button-activated boat, cross simulated lava, and enter a target zone on the far platform.The lava is a 10 x 10 m heightfield; the boat moves at 3 meters per second between platforms.Source: M FEW-SHOT EXAMPLES · p. 113
• Its rewards encourage survival, progress toward activating the boat, and progress toward the target zone, with a bonus for entering it.The task terminates if the robot falls off a platform or the boat, or touches the lava.Source: M FEW-SHOT EXAMPLES · p. 113
• Another example specifies descending 10 steps, each 1 m long, 10 m wide, and 0.2 m high, to reach the ground.The steps form a descending staircase above a ground platform.Source: M FEW-SHOT EXAMPLES · p. 117
• The stair example starts the robot at the top and rewards survival and forward velocity as it descends; success requires reaching the ground platform.Source: M FEW-SHOT EXAMPLES · pp. 117–118
• A second example requires the robot to activate a lever beside a closed, wall-flanked door and pass through it.The lever is placed to the left of the door, which is positioned ahead of the robot on the platform.Source: M FEW-SHOT EXAMPLES · p. 119
• Door-task rewards combine survival, moving closer to the lever, a bonus for activating it, and forward movement after the door opens.Source: M FEW-SHOT EXAMPLES · p. 120

N TASK DESCRIPTION SEEDS

• The bridge task asks the robot to cross differently colored segments separated by gaps to reach an end platform.The bridge is 2 m wide, with 2 m gaps between segments; success is reaching the end platform.Source: N TASK DESCRIPTION SEEDS · p. 123
• The stair task has the robot ascend 10 steps from the ground to a top platform.Each step is 1 m long, 10 m wide, and 0.2 m high.Source: N TASK DESCRIPTION SEEDS · p. 123
• The ball task requires the robot to reach a randomly placed ball and kick it between two goal posts.Success is defined as the ball passing between the posts.Source: N TASK DESCRIPTION SEEDS · pp. 123–124

O DISCUSSION ON TRAINING GENERALISTS

• The authors did not train generalist agents on OMNI-EPIC tasks because their lab lacked the resources, though they suggest its curriculum could make such training feasible.The generated curriculum is progressively complex and diverse.Source: O DISCUSSION ON TRAINING GENERALISTS · p. 125
• Adaptive Agent Team et al. (2023) showed that training on sufficiently diverse tasks can produce agents that generalize to new tasks within that distribution.Generating such a distribution is time-consuming and challenging, and automatically creating a curriculum within it is important for efficiency.Source: O DISCUSSION ON TRAINING GENERALISTS · p. 125
• Training generalist agents would remain a significant endeavor, requiring an extensive environment distribution, considerable effort, and substantial computational resources.The authors cite Adaptive Agent Team et al. (2023) as an example.Source: O DISCUSSION ON TRAINING GENERALISTS · p. 125

P EXPERIMENTS ON DIFFERENT ROBOTS AND ACTION SPACES

• The Ant is a 3D quadruped with a torso and four two-segment legs, and uses a continuous 8-dimensional action vector bounded between -1 and 1.The torso has free rotational movement.Source: P EXPERIMENTS ON DIFFERENT ROBOTS AND… · p. 126
• All settings except robot type and action space—including observation space and RL algorithm—match those used for the R2D2 robot.Source: P EXPERIMENTS ON DIFFERENT ROBOTS AND… · p. 126
• The Ant robot completed different OMNI-EPIC-generated tasks, demonstrating the approach across robot types and action spaces.Figure 42 shows examples of the Ant successfully completing generated tasks.Source: Figure 42 · p. 126, P EXPERIMENTS ON DIFFERENT ROBOTS AND… · p. 126