Source-linked AI summary
CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning
Ossama Ahmed, Frederik Träuble, Anirudh Goyal, Alexander Neitz, Yoshua Bengio, Bernhard Schölkopf, Manuel Wüthrich, Stefan Bauer
TL;DR
RL agents often struggle to transfer skills between related environments, motivating a benchmark with explicit shared causal structure. CausalWorld provides intervention-controlled robotic construction tasks, curricula, and evaluation protocols; baseline results verify feasibility while showing that training curricula affect generalization.
Problem
RL agents often fail to transfer learned skills to different but related environments, while existing benchmarks leave shared causal structure and expected generalization unclear.
Method
CausalWorld is a simulated TriFinger robotic benchmark where agents construct goal shapes while researchers intervene on shared environment variables to define curricula and evaluation distributions.
Results
Baseline experiments verify task feasibility and show that different learning curricula affect agents’ in-distribution and out-of-distribution generalization.
Takeaways & Limitations
CausalWorld enables controlled studies of transfer, causal structure, and generalization in robotic manipulation, with potential sim-to-real transfer through its open-source robot counterpart.
Takeaways & Limitations
Extreme domain randomization can prevent model-free agents from learning relevant skills, possibly because the setting requires more data or lacks a single robust strategy.
Abstract
from arXiv · showhide
Despite recent successes of reinforcement learning (RL), it remains a challenge for agents to transfer learned skills to related environments. To facilitate research addressing this problem, we propose CausalWorld, a benchmark for causal structure and transfer learning in a robotic manipulation environment. The environment is a simulation of an open-source robotic platform, hence offering the possibility of sim-to-real transfer. Tasks consist of constructing 3D shapes from a given set of blocks - inspired by how children learn to build complex structures. The key strength of CausalWorld is that it provides a combinatorial family of such tasks with common causal structure and underlying factors (including, e.g., robot and object masses, colors, sizes). The user (or the agent) may intervene on all causal variables, which allows for fine-grained control over how similar different tasks (or task distributions) are. One can thus easily define training and evaluation distributions of a desired difficulty level, targeting a specific form of generalization (e.g., only changes in appearance or object mass). Further, this common parametrization facilitates defining curricula by interpolating between an initial and a target task. While users may define their own task distributions, we present eight meaningful distributions as concrete benchmarks, ranging from simple to very challenging, all of which require long-horizon planning as well as precise low-level motor control. Finally, we provide baseline results for a subset of these tasks on distinct training curricula and corresponding evaluation protocols, verifying the feasibility of the tasks in this benchmark.
1 INTRODUCTION
CausalWorld addresses weakly characterized transfer in RL by providing robotic environments with shared causal structure and controllable interventions. It supports curricula and evaluation protocols that disentangle generalization across environment changes.
- 1 INTRODUCTION: Shared parameters and causal structure let researchers vary environment properties gradually, creating controllable training and test distributions.
- 1 INTRODUCTION: The benchmark targets robotic construction tasks inspired by children building complex structures with spatial reasoning and dexterous manipulation.
- 1 INTRODUCTION: CausalWorld introduces a parametrized family of robotic manipulation environments for out-of-distribution generalization and causal structure learning in RL.
- 1 INTRODUCTION: Do-interventions on exposed variables support curricula and disentangle agents’ generalization with respect to different environment changes.
- 1 INTRODUCTION: Baseline experiments verify task feasibility and show that curricula affect agents’ in-distribution and out-of-distribution generalization.
2 CAUSALWORLD BENCHMARK
CausalWorld defines robotic construction environments through shared causal variables, task generators, intervention-based curricula, and separate training and evaluation spaces. This design supports tasks ranging from simple manipulation to complex multi-block construction and enables controlled generalization studies.
- 2 CAUSALWORLD BENCHMARK: A robot builds specified goal structures from available objects in a Bullet simulation of the open-source TriFinger platform.
- 2 CAUSALWORLD BENCHMARK: The benchmark exposes variables including gravity, friction, colors, joint positions, and block size, color, mass, position, and orientation for intervention.
- 2 CAUSALWORLD BENCHMARK: Task generators sample related goal-shape families, spanning pushing, picking, pick-and-place, stacking, towers, and more complex stacked-block structures.
- 2 CAUSALWORLD BENCHMARK: The action and observation interfaces support multiple control modes plus structured or six-camera pixel observations, while a shared volumetric-overlap reward ranges from 0 to 1.
- 2 CAUSALWORLD BENCHMARK: Training and evaluation use an allowed training space and evaluation space, with interventions possible at episode resets or during episodes.
- 2 CAUSALWORLD BENCHMARK: Intervention actors specify when and how exposed variables change, allowing modular curricula composed of multiple actors.
3 RELATED WORK
Existing RL benchmarks often leave the shared causal structure between environments ambiguous and emphasize single-task learning or random parameter assignment. CausalWorld addresses these gaps with explicit interventions across environment variables, enabling more interpretable generalization and causal-robotics research.
- Prior multi-task and meta-learning benchmarks use hand-designed task variations, making the shared structure and expected generalization between tasks difficult to quantify.
- Earlier parametric generalization studies mostly assigned variables randomly, whereas CausalWorld provides full control through do-interventions.
- RL evaluation also lacks standardized reporting practices despite sensitivity to factors that affect learning curves and reproducibility.
- CausalWorld is motivated in part by the underdeveloped connection between causal learning, robotics, and reinforcement learning.
4 EXPERIMENTS
The experiments evaluate model-free RL agents on basic manipulation families under curricula that progressively expand interventions. Results show that single-object tasks are learnable, while stacking and extreme randomization remain difficult, and evaluation protocols reveal curriculum-dependent generalization.
- Setup: Curriculum 1 randomized goal position and orientation each episode, while curriculum 2 intervened simultaneously on all task variables in space A.
- Training model-free RL methods: Agents solved single-block pushing, picking, and pick-and-place tasks with enough experience, but none solved two-block stacking; scores below 0.5 reflected only pushing the lower cube.
- Benchmarking generalization capabilities along various axes: The benchmark evaluates 12 protocols per goal-shape family over 200 episodes, sampling selected variables while holding others fixed to disentangle generalization abilities.
- Benchmarking generalization capabilities along various axes: Under curriculum 0, agents performed well on the default pushing task and generalized to initial poses, but remained weak on varied goal poses.
- Benchmarking generalization capabilities along various axes: Goal-pose randomization produced robust generalization to different goal poses, unlike agents trained only on the default environment.
- Training model-free RL methods: Extreme domain randomization produced flat training curves and little success, even after 100 million timesteps, suggesting substantially greater learning difficulty.
- Generalization depends on the experience supplied by the training curriculum, while the hardest tasks appear beyond current model-free methods without additional inductive bias.
5 CONCLUSION
CausalWorld is introduced as a benchmark for causal structure and transfer learning in simulated robotic manipulation. Its intervention-based task family supports curricula and evaluation protocols that separate different generalization capabilities and can scale in difficulty.
- CausalWorld uses interventions on environment properties to create diverse tasks and define curricula and evaluation protocols that disentangle generalization capabilities.
- Its flexibility and modularity are intended to help researchers define increasingly difficult benchmarks and coordinate future research goals.
A OBSERVATIONS
CausalWorld provides structured and pixel observation modes, with pixel observations pairing three current-state views with three corresponding goal views. Structured observations concatenate block, partial-goal, and obstacle or fixed-block features in a defined order.
- A OBSERVATIONS: Pixel observations contain six RGB images: three current-state views and three matching goal views from the same cameras.The views are rendered around the TriFinger platform and show the target shape the robot must build.
- A OBSERVATIONS: Structured observations concatenate all block feature vectors, followed by partial-goal features and then obstacle or fixed-block features.
B TRIFINGER PLATFORM
CausalWorld uses the open-source TriFinger robot as its robotic platform. Its reproducible design and roughly $5000 cost support sim-to-real research.
- B TRIFINGER PLATFORM: The TriFinger platform is open-source, reproducible in research labs, and costs about $5000.The paper identifies these properties as supporting faster sim-to-real research.
C TASK GENERATORS
CausalWorld task generators produce families of manipulation goals ranging from single-block movements to complex, partially specified or arbitrary structures. The benchmark exposes task variables, spaces, and dense rewards for controlling and evaluating these tasks.
- C TASK GENERATORS: The benchmark includes pushing, picking, pick-and-place, and stacking2 tasks with progressively structured manipulation goals.Pushing targets a floor-level position and orientation, picking targets a height, pick-and-place crosses a fixed block, and stacking2 places two blocks above one another.
- C TASK GENERATORS: Towers and Stacked Blocks require building multi-block structures, with Stacked Blocks allowing arbitrary stable arrangements rather than exact vertical alignment.The latter is more challenging because the agent must find a stable structure covering the goal volume.
- C TASK GENERATORS: Creative Stacked Blocks reveals only the first and last goal levels, requiring the agent to infer a stable connecting structure.The paper describes this as the most challenging task in the listed family.
- C TASK GENERATORS: The General task creates an arbitrary goal shape by dropping blocks and using their final resting configuration as the target.
- C TASK GENERATORS: Task generators sample related goal shapes and expose configurable high-level variables, spaces, and dense rewards for different environments.The supplied passages identify task-generator variables, high-level variable spaces, and dense reward descriptions as configurable benchmark components.
D TRAINING DETAILS
Baseline training uses PPO, SAC, and TD3 with a shared two-layer MLP policy, while evaluation compares curricula and protocols across several manipulation tasks. The reported evaluation averages results over multiple episodes and random-seed models.
- D TRAINING DETAILS: PPO, SAC, and TD3 use a shared 2-layer MLP policy with architecture [256,256].PPO trains with 20 parallel workers for up to 100 million timesteps, whereas SAC and TD3 train serially for 10 million timesteps.
- D TRAINING DETAILS: Evaluation compares agents trained with different curricula using PPO across generalization axes and task protocols.
- D TRAINING DETAILS: Figure 10 reports pushing, picking, pick-and-place, and stacking2 evaluation scores averaged over 200 episodes and five random-seed models.