PaperlayerView exact revision on arXiv

Source-linked AI summary

Gym-Anything for Robotics: Scaling physical simulations with digital agents

Pranjal Aggarwal, Sean Welleck

arXiv:2610.40565v1cs.ROcs.AIcs.LG

TL;DR

Realistic robotics simulations are costly to build and cover a narrow range of real-world tasks and physical phenomena. Gym-Anything for Robotics uses coding agents, GDP-grounded selection, and creation-audit loops to build simulations and learning approaches. Frontier models complete none of RoboWorld’s test tasks, while the paper reports improved performance from several learning methods.

  • Problem

    Building realistic simulations requires substantial expert effort, while existing simulations cover a narrow range of real-world tasks and physical phenomena.

    Source: §1 · p. 2

  • Method

    Gym-Anything for Robotics has coding agents build simulations selected around economically valuable physical work and uses audit agents to check simulations and tasks.

    Source: §1 · pp. 2–3

  • Results

    Frontier models complete none of RoboWorld’s 25 test tasks, while the paper reports improved performance from in-context learning and privileged-state learning approaches.

    Source: §1 · p. 3, §6.2 · p. 11

  • Takeaways & Limitations

    The authors present Gym-Anything for Robotics and RoboWorld as a foundation for future work on realistic robot learning and broader coverage of physical work.

    Source: §8 · p. 16

    Evaluated agents use restricted access unless stated otherwise; privileged access to simulator state and task evaluation is available for learning but not on a real robot at test time.

    Source: §2.1 · p. 4

Abstract

from arXiv · show

Realistic, fast robotics simulations could enable large-scale training of embodied AI models that transfer to the real world. However, despite decades of progress, available simulations cover only a narrow range of real-world tasks and physical phenomena. The core bottleneck is that creating better simulations requires significant expert effort and therefore does not scale. To address this, we introduce Gym-Anything for Robotics, a framework in which coding agents create realistic robotics simulations. Our core insight is that simulation creation is a coding agent task, so improvements in coding agents translate directly into richer and more faithful simulations. We select simulations based on U.S. GDP data to cover economically valuable physical work, define what makes a good simulation and a good task, and develop an adversarial creation-audit loop that checks each simulation and task against these definitions. Using this pipeline, we build RoboWorld, a set of 25 simulations with 25 test and 175 training RL environments of varying difficulty. Even the best frontier models fail on every RoboWorld test task, achieving an average reward of only 0.06. Building on RoboWorld, we introduce several new ways for frontier models to improve, including in-context learning across repeated attempts, a privileged teacher that reads the simulator state and teaches a student in context, self-assisted learning from the model's own privileged experience, and code policies that generate demonstrations across varied scenes to train vision-language-action models (VLAs) such as $\pi_{0.5}$. Each of these substantially improves performance. We release our pipeline, RL environments, and benchmark to facilitate future research. Code and data are available at https://gym-anything.github.io/robotics

1 Introduction

The paper targets the effort required to build realistic simulations across a narrow range of physical work. Gym-Anything for Robotics uses coding agents, GDP-grounded selection, and creation-audit loops to build simulations and tasks, then evaluates frontier-model learning on RoboWorld.

1 Introduction

  • Gym-Anything for Robotics frames simulation creation as a coding-agent task, with improvements in coding agents expected to translate into richer, more faithful simulations.The framework has agents assemble assets into scenes, configure physics, define tasks, and implement rewards.Source: §1 · p. 2
  • Simulation selection decomposes occupational tasks into physical actions and weights them by economic value, then builds simulations for the most valuable actions and required capabilities.A coding agent assembles scenes from real assets and configures the robot and physics.Source: §1 · pp. 2–3
  • A creation agent and four audit agents check simulations against realism, solvability, speed, and exploit-robustness criteria, with auditors reporting issues for repair.The creation agent provides evidence such as renders, physical measurements, and assisted demonstrations.Source: §1 · p. 3
  • Generated tasks are audited for solvability and reward correctness, and their rewards measure both progress and the final outcome.Tasks support different robots and variations in object poses, appearance, and physics.Source: §1 · p. 3
  • Frontier models complete none of RoboWorld’s 25 test tasks, while the paper reports improved performance through in-context learning and privileged-state learning approaches.RoboWorld contains 25 test and 175 training RL environments; the paper also introduces code policies to train VLAs.Source: §1 · p. 3

2 Methodology

The methodology defines simulations and tasks as configurable physical environments with instructions, initial states, and verifiers. It selects simulations by the economic value of the physical actions and capabilities they cover.

2.1 Problem Setup

  • A simulation is a coded scene with physical models, a robot, cameras, and a physics engine; robots and scene variations can change without changing the task.Variations include object poses, appearance, and physics.Source: §2.1 · p. 4
  • A task pairs a simulation and initial state with a natural-language instruction and a verifier that assigns binary success and a reward measuring progress and outcome.Every successful completion receives reward 1, and the task can run across supported robots and scene variations.Source: §2.1 · p. 4
  • Agents receive camera images and robot feedback and act through robot commands; evaluated agents use restricted access unless otherwise stated, while privileged access adds simulator state and task evaluation.The simulation advances physics only through commands, so decision time does not change the scene.Source: §2.1 · p. 4

2.2 GDP-Grounded Simulation Selection

  • Simulation selection combines occupational tasks, their economic value, and time spent on everyday activities, then decomposes tasks into weighted atomic physical actions and groups actions by reproducible simulation capabilities.An optimization selects action groups whose required capabilities are supplied by the selected simulations; this work builds 25 of 120 selected groups.Source: §2.2 · p. 5

3 Scaling Robotics Simulations

The paper defines criteria for useful robotics simulations and uses a repeated creation-audit loop to build them. The resulting RoboWorld simulations span a range of physical-work domains and are compared with prior simulations.

3 Scaling Robotics Simulations

  • A simulation must satisfy ten criteria across four categories to be considered useful for evaluating and training robot policies.The criteria cover realism, dynamics and solvability, speed/configurability/documentation, and robustness to exploits.Source: §3 · pp. 5–6
  • The creation agent builds a simulation and evidence for its criteria; four independent auditors check it, including an adversarial agent, and the agent repairs reported issues over three rounds.Auditors check code and evidence, including videos, physical measurements, and assisted demonstrations.Source: §3 · p. 6, Figure 4 · p. 6
  • For liquids, chemistry, and cloth, RoboWorld’s specific physics and physics checks contrast with prior works’ general-purpose solvers and lack of reported quantitative physics checks.The comparison covers three RoboWorld simulations and the closest prior simulations in those domains.Source: §3 · p. 6, Table 1 · p. 7
  • The pipeline constructs 25 simulations across domains including manufacturing, construction, food service, households, laboratories, healthcare, logistics, sports, and education.Source: §3 · p. 6

4 Scaling Tasks

The section defines task quality and uses audited task creation to build varied, verifiable tasks across RoboWorld’s simulations. Tasks retain fixed simulation physics while varying instructions, initial states, and difficulty.

Task design

  • Each task combines an initial state, natural-language instruction, and verification function, while the simulation physics stays fixed across tasks.Tasks may include task-specific scene setup.Source: §4 · p. 7

Task criteria

  • Useful tasks must be meaningful, well-set, deterministically verified, physically achievable, and reward partial progress while crediting every valid solution.The verifier is based on physical simulation state and avoids false positives and false negatives.Source: §4 · p. 8
  • Difficulty coverage targets a spread of success rates for Claude Haiku 5.5 and Claude Opus 5.5, creating an informative ladder for weaker and stronger models.Tasks that are all too easy or all too hard provide little signal for evaluation or improvement.Source: §4 · p. 8

Task creation-audit loop

  • A task creation agent proposes diverse tasks, then six criterion-specific auditors identify issues for correction over at most three rounds.Recorded solver trajectories also guide the creation of tasks with targeted difficulty.Source: §4 · p. 8

RoboWorld

  • RoboWorld contains 25 simulations and 200 RL environments—25 for testing and 175 for training—with deterministic verifiers and continuous rewards.Environments support swappable robots and variations in object poses, appearance, and physics to provide varied initial states of the same task.Source: §4 · p. 9

5 Learning in Simulation with Frontier Models

The section studies how frontier models can learn through repeated simulation attempts and privileged state, and how those benefits can be passed to students and robot policies. Privileged code-policy demonstrations train π0.5 to reach 53% success.

Training policies from code

  • π0.5 success rises from 0% to 53% when trained on demonstrations from privileged code policies.The policies generate successful rollouts across varied initial states; π0.5 uses camera views and proprioception without object state.Source: §5 · p. 10

In-context learning

  • Repeated attempts with instant resets let a frontier model improve from failures in one context, with experiments comparing restricted and privileged state access.Each agent receives three attempts of three hours each.Source: §5 · p. 10

Teaching a student with privileged state

  • A privileged teacher-derived coach gives a restricted student up to ten qualitative hints after inspecting the student’s simulator state, but cannot act for it.Hints contain no positions, coordinates, or numerical robot commands.Source: §5 · p. 10

Learning from privileged experience

  • A model that succeeded with privileged state resumes its experience under restricted access on new scene variations, testing transfer against students without that experience.Source: §5 · p. 10

Training policies from code

  • Privileged code policies read camera images, robot state, and task-object state, then issue ordinary robot commands whose successful rollouts train a VLA.The policies cannot move objects, apply forces, change physics, or reset the simulation, and can generate demonstrations across thousands of varied initial states.Source: §5 · p. 10

Frontier model performance

  • All three frontier models achieve 0% success on RoboWorld’s 25 test tasks, while all complete some tasks in the easier training subset.The figure compares success rate and average reward, with all models using extra-high reasoning effort.Source: Figure 6 · p. 11

6 Results

The results establish RoboWorld as a difficult benchmark, show that privileged state supports several learning routes, and report stronger simulation quality than prior simulators in blind comparisons. The section also describes the evaluation setup and audits that detect simulation and task defects.

6.1 Experimental Setup

  • The evaluation runs each attempt from the task’s initial state on an independent simulation copy, allows three hours, and reports success rate and average continuous reward.Agents do not observe rewards during an attempt; the learning experiments use Claude Opus 5.5 in every role.Source: §6.1 · p. 11

6.2 Main Results

  • All three frontier models achieve 0% success on RoboWorld’s 25 test tasks, with average rewards of 0.002 to 0.06; Claude Opus 5.5 reaches 32% success on an easier 25-task subset.On the easier subset, Claude Opus 5.5 averages 0.48 reward, GPT-6 Astra 0.41, and GPT-6.1 Terra 0.17; at least one model completes 44% of those tasks.Source: §6.2 · pp. 11–12
  • Success declines with robot complexity: models succeed on 18–35% of fixed-base-arm tasks, at most 20% of mobile-manipulator tasks, and little or none of the humanoid tasks.On humanoids, GPT-6 Astra and GPT-6.1 Terra earn no reward; Claude Opus 5.5 succeeds only in football, while falling as it kicks the ball.Source: §6.2 · p. 12

6.3 Learning in Simulation with Frontier Models

  • With restricted access, Opus 5.5 improves from 0.14 to 0.22 reward across three attempts; privileged access raises its first-attempt reward to 0.34 and third-attempt reward to 0.41.A coached student reaches 0.29 on its first attempt using qualitative hints, while a model resuming after privileged practice reaches 0.29 by its third restricted attempt.Source: §6.3 · pp. 13–14

6.4 Simulation Quality

  • In blind pairwise comparisons across 25 interactions, RoboWorld simulations win 80% of visual comparisons and 76% of physics comparisons against selected prior simulators.The prior simulators were selected before comparison, and the blind comparisons used swapped presentation order across two runs.Source: Figure 9 · p. 14, §6.4 · p. 14

6.3 Learning in Simulation with Frontier Models

  • Code policies succeed on 96.7% of 2,048 varied gift-box states, and their demonstrations raise π0.5 success from 0% to 53%, including 55% on unseen variations.π0.5 uses camera views and proprioception, matches Opus 5.5’s success rate, and completes a task in about two minutes versus about 29 minutes for Opus 5.5.Source: Figure 7 · p. 12, §6.3 · p. 14

6.4 Simulation Quality

  • Simulation audits found physics defects, including detached-object and leaking-sauce errors, and the creation agent repaired them in the next round; adversarial audits found defects in about three quarters of simulations.The adversarial auditor used only legal robot actions, and each repair was checked by replaying the auditor’s reproduction.Source: §6.4 · p. 14
  • Task audits exposed verifier errors that could reward untouched scenes or reject genuine goals, helping ensure rewards increase with task progress.Auditors tested verifiers on constructed states such as untouched scenes and partly completed tasks.Source: §6.4 · p. 15

7 Related Work

The related work contrasts the paper’s breadth and physics implementation with existing simulation and generation approaches. It also connects frontier-model learning in context with privileged-state transfer methods in robot learning.

Simulation for robot learning

  • Existing robotics benchmarks are mostly hand-built and cover a narrow slice of physical work, while RoboWorld uses coding agents and selects simulations by the economic value of physical work.Prior benchmarks often focus on rigid or articulated objects or a single deformable material; BEHAVIOR-1K comes closest to broad coverage among the cited examples.Source: §7 · p. 15

Reliable physics in simulation

  • Prior physics-fidelity approaches require experts to model a task or material, while LLM generators use fixed engine physics and typically check output through policy success or visual inspection.Gym-Anything agents implement physics and audit it with physical measurements, assisted demonstrations, and adversarial robot actions.Source: §7 · p. 15

Frontier models learning in simulation

  • The paper brings together frontier-model learning across attempts and privileged-state robot learning, showing transfer through qualitative hints, self-experience, and code-policy demonstrations.Earlier privileged teachers typically transferred knowledge through weight updates.Source: §7 · p. 15

8 Conclusion

Gym-Anything for Robotics uses coding agents and a creation-audit-repair loop to build realistic simulations at scale, yielding RoboWorld’s 200 environments across 25 GDP-grounded simulations. The paper reports that privileged state improves frontier-model learning, while frontier models still fail every RoboWorld test task.

8 Conclusion

  • Gym-Anything for Robotics frames simulation creation as coding-agent work and checks simulations and tasks through multi-agent creation, auditing, and repair.RoboWorld contains 200 RL environments across 25 GDP-grounded simulations, including liquids, cloth, and physics unavailable in existing engines.Source: §8 · p. 15
  • Privileged simulator state more than doubles frontier models’ first-attempt reward, can teach a student through hints, and supports π0.5 training from 0% to 53% success.Despite these learning results, frontier models fail every RoboWorld test task.Source: §8 · pp. 15–16
  • The authors conclude that realistic simulations, rather than simulation itself, have constrained simulation-based robot learning, and identify coding agents as a route to building them.They present the released framework, benchmark, and code as a foundation for future work on realistic robot learning.Source: §8 · p. 16

A GDP-Grounded Simulation Selection: Full Pipeline

The pipeline weights physical actions by economic value, groups actions that one simulation can reproduce, and selects action groups to cover required robotic and physics capabilities. It draws on separate measures for paid work and everyday activities, then balances coverage across six tiers.

A GDP-Grounded Simulation Selection: Full Pipeline

  • Paid-work task values are distributed by importance and frequency, while everyday activities are valued in population-hours and kept as a separate stream.Combining dollars and population-hours would require an arbitrary exchange rate.Source: A GDP-Grounded Simulation Selection:… · p. 22
  • Each task is decomposed into open-vocabulary atomic physical actions, weighted by task value, physical-interaction share, and each action’s share of execution.Normalizing action shares conserves total task weight as the task is decomposed into more actions.Source: A GDP-Grounded Simulation Selection:… · p. 22
  • Actions are grouped only when one parameterized simulation can reproduce the same state change, contact sequence, effector, material regime, and completion condition.The resulting groups require 1,031 distinct robotic and 1,092 distinct physics capabilities.Source: A GDP-Grounded Simulation Selection:… · p. 22
  • An action counts as covered only when selected simulations supply all its required capabilities, and a simulation can therefore cover actions beyond its own group.Coverage weights actions by their value within a selected activity slice.Source: A GDP-Grounded Simulation Selection:… · p. 22
  • Selection maximizes added coverage across six tiers, from the economic core and priority domains to occupation groups, niche actions, weak capability families, and everyday activities.The three separate 120-group runs using combined, physics-only, and robotics-only coverage yielded 195 distinct action groups.Source: A GDP-Grounded Simulation Selection:… · pp. 22–23

B Simulation Creation: Full Details

Simulation creation uses coding agents for research, design, implementation, and documentation, with adversarial audits and shared lessons intended to identify and repair shortcomings. The process requires physical changes to result from robot interaction, not scripted outcomes.

B Simulation Creation: Full Details

  • Each framework agent is Claude Opus 5.5 with extra-high reasoning effort, run through a coding-agent harness and given an objective-specific prompt.Agents share a workspace and prompts describe what a good simulation is rather than prescribing its implementation.Source: B Simulation Creation: Full Details · p. 23
  • The creation agent researches an action group, designs a scenario, then builds it from real assets with task-relevant physics, a catalog robot, tasks, and evidence.Every world change must be physically caused by the robot; missing mechanics cannot be replaced by animations, attachments, or scripted outcomes.Source: B Simulation Creation: Full Details · p. 23
  • After building, a fixed blind nudge points the agent to the strongest existing simulations and calls its result a regression without diagnosing shortcomings.The agent must identify the shortcomings itself.Source: B Simulation Creation: Full Details · p. 23
  • Four fresh audit agents review dynamics and solvability, speed and configuration, and robustness, returning reports to the same creation agent for repairs.The speed audit requires simulation runs of at most 200 times slower than real time; robustness checks for non-finite values, energy blow-ups, and impossible motion.Source: B Simulation Creation: Full Details · p. 23
  • Agents accumulate shared Markdown lessons so findings from one simulation can guide later builds and audits.One recorded lesson requires a physical probe to confirm a collider acts on another body.Source: B Simulation Creation: Full Details · pp. 23–24

C Task Creation: Full Details

Task creation agents generate tasks within fixed existing physics, while audits check task validity and repair problems. Difficulty is assessed across model attempts with restricted access, bounded time, and approximate target success bands.

C Task Creation: Full Details

  • Task creation may change descriptions, verifiers, catalogs, and task-specific scene setup, but cannot change existing object physics or robot dynamics.Source: C Task Creation: Full Details · p. 24
  • Tasks represent realistic work or probe a nuanced robotic capability, while ordinary workmanship requirements remain implicit in the task description.Source: C Task Creation: Full Details · p. 24
  • Verifiers should measure the completion mechanism or sensor signal where possible, and audits can expose unstated constraints that reject valid completions.A labeling verifier rejected otherwise valid completions after 60 mm of part movement because of an unstated 50 mm limit.Source: C Task Creation: Full Details · p. 24
  • Rewards must not prescribe a method, step order, or reference trajectory, and fresh audit agents review tasks and return repairs to the same task creator.Source: C Task Creation: Full Details · p. 24
  • Difficulty evaluation uses four seeds for Claude Haiku 5.5 and two for Claude Opus 5.5, with restricted access and a six-hour budget per attempt.Target success bands are approximate across tasks, and agents receive no reward during attempts.Source: C Task Creation: Full Details · p. 24

D The Gym-Anything for Robotics Library

The library separates a shared runtime, simulation packages, and independent agents that connect through common interfaces. It supports restricted and privileged access, records attempts for inspection, and exposes robot commands through the same interface across policies.

D The Gym-Anything for Robotics Library

  • The library separates core runtime, simulation packages, and agents, with Python, command-line, and HTTP interfaces constructing simulations through the same registry.Agents can be replaced without changing simulations.Source: D The Gym-Anything for Robotics Library · p. 24
  • Simulations may use different physics engines behind a common interface, and their command-driven clock advances physics only while a robot command executes.Source: D The Gym-Anything for Robotics Library · p. 24
  • Restricted credentials expose task instructions, images, robot feedback, and permitted-command documentation, while privileged credentials also expose simulator state and evaluation.Privileged evaluation returns reward and an explicit success label without advancing time.Source: D The Gym-Anything for Robotics Library · p. 24
  • Each simulation’s task catalog maps tasks or scene variations to an instruction, initial state, and verifier, while recordings retain commands, images, configuration, source version, and final evaluation.Source: D The Gym-Anything for Robotics Library · p. 25
  • Policies use the same robot-command interface, with each simulation supporting the subset its robot accepts and documenting units and limits.Commands range from task-space targets to joint torques or motor commands; the library also provides keyboard chords for movement.Source: D The Gym-Anything for Robotics Library · p. 25, Table 4 · p. 25, Table 5 · p. 25

E Comparison with Prior Simulations

This section compares RoboWorld and Gym-Anything environments with prior task-specific simulations across a range of tasks, highlighting differences in physical modeling, task scope, and evaluation criteria. It also describes validation checks, including comparisons with published or real-world data and adversarial evaluator cases.

E Comparison with Prior Simulations

  • Table 6 compares all 25 RoboWorld simulations with two prior simulations each, reporting prior papers’ claims alongside counts of physics checks and adversarial evaluator cases.Physics checks test simulation behavior against theory, measurement, or expected behavior; adversarial cases are constructed states the verifier must reject.Source: E Comparison with Prior Simulations · p. 25, Table 6 · p. 26
  • RoboWorld’s cloth models include anisotropic membrane and bending behavior, thickness and compression, friction, self-contact, and exact intersection checks.The machine-sewing simulation uses an implicit backward-Euler cloth solve and KES-measured fabric, with 18 physics checks including fabric tests and bending comparisons.Source: E Comparison with Prior Simulations · p. 27
  • The deformable-can simulation models denting and thin-shell behavior, with up to 144 deforming can shells and a film-bag membrane.Its checks include dent recovery, drop contact time, and rolling behavior; the table also lists 13 adversarial evaluator cases.Source: E Comparison with Prior Simulations · p. 28
  • RoboWorld’s ingredient-mixing simulation models powders, batters, and pastes, including wetting, lumps, and sub-grid mixing.Its 29 physics checks include stirred-cup torque, yield-stress slump, and flour-column collapse; it also has 26 adversarial evaluator cases.Source: E Comparison with Prior Simulations · p. 28
  • The soil-digging simulation couples MPM and MuJoCo and models material-dependent friction, adhesion, moisture-dependent cohesion, and remolding.It spans eight soil materials and six yard presets, with tools and moisture states also specified.Source: E Comparison with Prior Simulations · p. 29
  • The football simulation combines pressure-shell ball contact, stick-slip shoe traction, compliant grass, and resistance to rolling and spin.It uses an IFAB-sized ball and goal, a grass-pitch sector, and 18 physics checks including rebound and spin behavior.Source: E Comparison with Prior Simulations · p. 29

Injection

  • RoboWorld models injection through coupled syringe, vial, line, and tissue physics, with dose, air, placement, and delivery-rate criteria plus 14 physics checks and 11 adversarial cases.Its setup includes four liquids, multiple syringe and needle sizes, a sealed vial, and a three-layer injection pad.Source: E Comparison with Prior Simulations · p. 30

Dishwashing

  • Where iGibson 2.0 [Li et al., 2021] models rinsing as absorbed droplets and RoboCasa365 [Nasiriany et al., 2026] uses coded water regions, RoboWorld simulates thin-film flow, splatter, pools, and transported species.RoboWorld’s dish-room evaluation includes 16 physics checks and 76 adversarial cases.Source: E Comparison with Prior Simulations · p. 30

Pouring

Lock mechanism

  • Against ManiSkill-ViTac [Li et al., 2024] sensor gels and Meta-World [Yu et al., 2020] rotating-lock simulation, RoboWorld models a five-pin lock with native mesh contact and contact-only key discrimination.The modeled lock includes round pin bores, spring-loaded drivers, and lubricated brass friction.Source: E Comparison with Prior Simulations · p. 31

Welding

  • Prior welding simulations use 2D traversal or collision detection with inverse kinematics, whereas RoboWorld models welding with a GPU enthalpy solver, metal phase changes, arc transfer, and weld-pool physics.The prior methods are Arc-Welding RL and Welding DDPG+IK [Zhong et al., 2022].Source: E Comparison with Prior Simulations · p. 31

Painting

Paper: gift bags, wrap and cartons

Paste: concrete, mortar and sealant finishing

Rigid furniture: classroom rearrangement

  • Newton 1.5 adds 56 furniture bodies with part-level mass properties, pair-specific glide friction, Stribeck transitions, tip-versus-slide behavior, and robot collision reflexes.The classroom task also models a swerve-drive base with wheel traction and drive-torque limits.Source: E Comparison with Prior Simulations · p. 34

Snap-fit: pen capping

  • Newton 1.5 simulates pen capping with native nonconvex contact, four compliant snap nubs, and contact-driven seating rather than a joint-defined cap.The model runs at 960 Hz with 30 solver iterations; the snap nubs have stiffness 30 kN/m and damping 600 Ns/m each.Source: E Comparison with Prior Simulations · p. 34

Soft tissue: fish filleting and meat trimming

  • Explicit GPU FEM models trout with 73,273 stable neo-Hookean tetrahedra per fish, anisotropic knife cutting, cohesive tissue interfaces, and rigid bones with cuttable joints.The simulator uses 1e-5 s substeps and models bending and breaking rod bones.Source: E Comparison with Prior Simulations · p. 35

Tool contact: whiteboard writing and erasing

E Comparison with Prior Simulations

  • For cutting, DiSECt [Heiden et al., 2021] models Neo-Hookean tetrahedra and crack-forming damage springs, while the comparison also examines prescribed knife motion and forces against real cuts.The tested produce includes apple, potato, and cucumber with laser-scanned meshes.Source: E Comparison with Prior Simulations · p. 36
  • For drilling and anchoring, Meta-World [Yu et al., 2020] is contrasted with models using solved-thrust bit feed, material-specific cutting laws, and push-gated hammer blows.The model also includes motor torque-speed behavior, drill walking, and concrete removal based on blow energy per volume.Source: E Comparison with Prior Simulations · p. 36
  • For screw tightening, Factory [Narang et al., 2022] uses PhysX SDF collisions and Gauss-Seidel contact solving, while the new model represents thread friction, detents, clamp load, and screw inertia.Its ISO 16047-based friction model includes self-locking threads, stick-breakaway behavior, and clamp friction across seven material pairs.Source: E Comparison with Prior Simulations · p. 36
  • For nail-driving tasks, RLBench [James et al., 2020] uses V-REP/PyRep with a nail turned on a joint, alongside a screwdriver-and-nail-in-block setup.The supplied comparison identifies these prior task and simulation details but does not provide a quantitative outcome for this row.Source: E Comparison with Prior Simulations · p. 36
  • For vehicle simulation, Forklift Sim2Real [Oishi et al., 2025] tunes Isaac Sim drivetrain response to a 1/14-scale forklift, while DARPA VRC [Ag¨uero et al., 2015] uses ODE rigid bodies and primitive collisions.The new forklift model uses MuJoCo rigid multibody dynamics via Newton and includes tyre, drivetrain, steering, hydraulic, stability, and brake mechanics.Source: E Comparison with Prior Simulations · p. 37

F Additional Details on Learning in Simulation

A coached towel-folding attempt shows how a qualitative hint changed the strategy: the student folded the towel with one gripper about 40 minutes later.

F Additional Details on Learning in Simulation

G Extended Related Work

Prior robotics simulation work spans several task and material domains, while physics generation, frontier-model learning, and economic grounding appear as distinct research threads. Gym-Anything for Robotics brings coding-agent simulation creation together with physics auditing, privileged-state learning, and GDP-grounded selection.

G Extended Related Work

  • Existing simulation benchmarks cover rigid and articulated tabletop objects, households, workplaces, and deformable materials, but are mostly hand-built and require expert effort to add new physics.BEHAVIOR-1K [Li et al., 2023] comes closest to broad coverage, with 1,000 household activities grounded in time-use surveys.Source: G Extended Related Work · pp. 37–38
  • Specialized physics methods can faithfully model specific tasks, but require experts to tune physics by hand; prior LLM-generated scenes and tasks largely use fixed engine physics and do not measure simulated physics.Gym-Anything for Robotics extends a creation-audit loop to physics implemented by coding agents and checked through physical measurements, assisted demonstrations, and adversarial robot actions.Source: G Extended Related Work · p. 38
  • Prior privileged-teacher methods transfer knowledge through weight updates, while frontier-model agents have mainly been studied in games and rigid-object scenes without privileged state as a learning signal.The paper combines these lines through privileged-state-accelerated in-context learning, qualitative coaching, self-assisted learning, and code policies for training vision-language-action models.Source: G Extended Related Work · p. 38
  • Earlier robotics benchmarks used time-use surveys, but the authors know of no prior robotics simulation benchmark based on occupation data; this work combines occupation tasks and wages with everyday-activity time-use data.Source: G Extended Related Work · pp. 38–39
Loading 2610.40565v1…