SIMPLE: Simulation-Based Policy Learning and Evaluation for Humanoid Loco-manipulation

• Humanoid foundation-model evaluation remains an open challenge because loco-manipulation requires coordinated locomotion, whole-body balance, and dexterous manipulation.Source: §1 · p. 2
• Existing simulation platforms are fragmented between tabletop manipulation and pure locomotion, leaving humanoid whole-body balancing and diverse object interaction insufficiently covered.Real-world evaluation is expensive, time-consuming, and difficult to reproduce, complicating fair policy comparisons.Source: §1 · p. 2
• SIMPLE couples MuJoCo contact-rich dynamics with Isaac Sim photorealistic rendering to support whole-body loco-manipulation.Source: §1 · p. 2
• The testbed provides automated motion-planner trajectory generation and low-latency VR teleoperation for human demonstrations.It also integrates lower-body controllers and supports mainstream VLA and WAM policies.Source: §1 · p. 2
• SIMPLE benchmarking reveals strong correlation between simulated and real-world policy performance and demonstrates zero-shot transfer of policies trained on SIMPLE data to physical humanoids.The reported transfer applies under similar settings.Source: §1 · p. 2
• The benchmark spans 16 whole-body tasks, 50 indoor scenes, and over 1,000 object assets, with automated and teleoperated collection yielding over 6,000 trajectories.Tasks cover rigid pick-and-place, non-prehensile interaction, and articulated-object manipulation.Source: §1 · p. 2

2.1 Robot Policy Evaluation in the Real World

• Real-world evaluation of generalist policies [3] [1] [24] [25] [4] requires complex, dynamic full-body demonstrations and rollouts, making rapid iterative humanoid policy development impractical.Physical testing also involves difficult-to-standardize resets, human supervision, operational complexity, and safety risks.Source: §2.1 · p. 3

2.2 Simulation-Based Benchmarks for Manipulation

• Existing simulation benchmarks cover tabletop manipulation, mobile manipulation, and other settings, but the cited platforms lack the physics support and integration for humanoid whole-body loco-manipulation.Simpler-Env [11] demonstrated correlation between simulated performance and real-world rankings after addressing visual and control gaps.Source: §2.2 · p. 3

2.3 Simulation for Humanoid Loco-Manipulation

• SIMPLE combines MuJoCo contact fidelity with Isaac Sim photorealistic rendering to address the gap between locomotion-focused and high-fidelity-rendering platforms.The paper describes SIMPLE as a full-stack testbed integrating data generation, policy learning, and standardized evaluation for whole-body humanoid loco-manipulation.Source: §2.3 · p. 3

3 Method

• The system pipeline generates data in MuJoCo, replays and renders it in Isaac Sim, then evaluates policies under domain-randomized settings.Source: §3 · p. 3, Figure 2 · p. 4

3.1 System Architecture

• MuJoCo handles rigid-body dynamics, contact resolution, and robot control, while synchronized states are rendered photorealistically in Isaac Sim.A high-level policy supplies upper-body trajectories and base-navigation commands to a faster lower-body tracking controller that maintains balance and executes locomotion.Source: §3.1 · p. 4, Figure 3 · p. 4

3.2 Large-Scale Asset Curation and Task Design

• SIMPLE curates objects and indoor scenes and uses them to build whole-body loco-manipulation tasks across pick-and-place, non-prehensile, and articulated-object interactions.Objects are converted for simulation and rendering, with convex decomposition used to create collision geometries.Source: §3.2 · pp. 4–5

3.3 Scalable Data Collection Pipelines

• The data pipelines combine automated motion planning with VR teleoperation to generate humanoid demonstrations.Automated trajectories use stable object poses, synthesized grasp configurations, and scripted coordination of arm motion and base movement; teleoperation streams stereo video to a headset, retargets hand motions, and records demonstrations at 50 Hz.Source: §3.3 · p. 5

3.4 Offline Rendering and Domain Randomization

• Offline Isaac Sim replay applies visual randomization to object instances and poses, textures, lighting, camera viewpoints, and language instructions to generate RGB observations.The passage states that this pipeline supports downstream policy learning and evaluation.Source: §3.4 · p. 5

3.5 Policy Integration and Evaluation Protocol

• Evaluation progresses from distractor objects to visual randomization and then spatial randomization of target and robot positions.The policy interface uses a 32-dimensional proprioceptive state and 36-dimensional action space, and the study rendered 6,000 episodes for benchmarking mainstream VLA and WAM architectures.Source: §3.5 · pp. 5–6

4.1 Benchmarking of Humanoid Policies

• Ψ0 consistently performs well across most tasks and leads the benchmark; DreamZero and π0.5 generalize strongly at Level 2 except on Task 6, while ACT also performs strongly overall.The authors attribute ACT’s performance to data efficiency on the less noisy, single-task SIMPLE demonstrations.Source: §4.1 · pp. 6–7, Table 1 · p. 6

4.2 Data Collection Efficiency in the Simulator

• Sim teleoperation is substantially faster than physical teleoperation, while motion planning has the lowest throughput but avoids operator fatigue and hardware-reset overhead.Table 2 reports demonstrations per hour and mean episode duration; simulated collection also scales across environments, lighting, and object instances through offline replay and rendering.Source: §4.2 · p. 7, Table 2 · p. 7

4.3 Ablation Studies

• Training with mixed Level 0 and Level 1 data preserves 0.80 success on Eval Set 0 and raises Eval Set 2 success from 0.50 to 0.80 versus Level 0-only training.Increasing teleoperation data significantly improves performance across all three validation sets.Source: §4.3 · p. 7, Table 3 · p. 7
• Human teleoperation data is more suitable than motion-planning-only data for learning complex tasks, which the authors attribute to its diverse, naturalistic motion profiles.Source: §4.3 · p. 8, Table 4 · p. 8

4.4 Zero-Shot Sim-to-Real Transfer

• Ψ0 trained exclusively on SIMPLE data transfers zero-shot to the real world on a subset of tasks under similar environmental settings.The evaluation uses pick-and-place and handover task observations; the authors report no real-world downstream fine-tuning.Source: §4.4 · p. 8, Table 5 · p. 8, Figure 5 · p. 8

4.5 Whole-Body Task Coverage

• SIMPLE includes whole-body tasks requiring coordinated arm–leg motion, including XMovePushBox, KickBox, and XMoveBendCarryBox.In XMoveBendCarryBox, the robot kneels, wraps both arms around a box, and lifts it onto a table.Source: §4.5 · p. 8, Figure 6 · p. 8

4.6 Physics–Rendering Consistency

• MuJoCo collision meshes and Isaac Sim visual meshes are closely aligned across frames, including at table surfaces, object edges, and the humanoid torso.Rendering runs at approximately 4 fps because of Isaac Sim’s synchronized render mode; parallelizing environments can shift the bottleneck to available compute.Source: §4.6 · pp. 8–9, Figure 7 · p. 9

4.7 Simulation Parameters in MuJoCo

• The tuned MuJoCo parameters cover actuator, solver, and contact settings, while control gains vary with the whole-body controller.The reported settings include joint force limits, friction, viscous damping, and armature.Source: §4.7 · p. 9, Table 6 · p. 9

• SIMPLE standardizes humanoid policy training and evaluation with 16 tasks, over 1,000 objects, 50 indoor scenes, and automated and VR teleoperation data pipelines.Source: §5 · p. 9
• The authors report that SIMPLE policy performance strongly correlates with real-world outcomes and that SIMPLE-generated data supports effective sim-to-real transfer.Source: §5 · p. 9
• The current pipeline renders at approximately 4 frames per second on a single GPU and models objects as rigid bodies, limiting faithful simulation of deformable and soft-body items.Source: §5 · p. 9

A.1.1 Main Class Entities

• BaseDualSim holds both simulator backends and the active task, while concrete environments add task-specific reset() and step() logic.Isaac Sim is initialized only when selected, so MuJoCo-only workflows avoid rendering overhead.Source: §A.1.1 · p. 13
• MujocoSimulator advances contact physics and manages actuation and state readout; IsaacSimSimulator consumes synchronized states to produce photorealistic RGB frames.The shared simulator interface defines layout, state, step, and render operations.Source: §A.1.1 · p. 13
• Task bundles task-specific randomization, robot, sensor, and Gym action/observation configuration, while reset() applies randomizers to build a layout for both simulators.Concrete tasks can add success checks, natural-language instructions, and automated subtask decomposition.Source: §A.1.1 · pp. 13–14
• Robot stores physics, rendering, kinematics, sensor, and controller configurations, while DRManager dispatches registered randomizers and selects among three difficulty levels.Robot passes assets and kinematics configuration to simulator and controller modules rather than executing control itself.Source: §A.1.1 · p. 14
• BaseAgent subclasses provide queued actions, including deterministic CuRobo trajectory replay for motion planning and teleoperated hand-pose commands routed through whole-body control.Remote VLA and WAM clients package observations for separate server processes and enqueue returned action chunks, decoupling inference from simulation-host memory.Source: §A.1.1 · pp. 14–15
• The three whole-body controllers share setup and action interfaces, with AMO decoupling upper- and lower-body control and SONIC variants forwarding or converting whole-body commands.The class hierarchy is shown in Figure 8.Source: §A.1.1 · pp. 13, 15

A.1.2 Sequential Diagram

• MuJoCo advances physics at 500 Hz and writes updated robot and object states to a shared buffer.This is the first stage of the env.step(action) sequence.Source: §A.1.2 · p. 15
• Isaac Sim reads the synchronized states and renders RGB frames at 50 Hz for the visual observation.Rendering follows the physics update within each step.Source: §A.1.2 · p. 15
• BaseDualSim queries the task for reward and termination before returning the step output, allowing MuJoCo to sub-step ahead of rendering.The ordering is intended to give each inference step a physically consistent, photorealistic observation.Source: §A.1.2 · p. 15

A.2 Offline Object Preprocessing Pipeline

• The preprocessing pipeline runs once per asset and produces the artifacts needed for physics, rendering, and grasp planning.No online grasp synthesis is required during training or evaluation.Source: §A.2 · p. 15
• Assets from Objaverse and GraspNet-1B are normalized and processed into MuJoCo collision geometries and Isaac Sim USD meshes with high-resolution PBR textures.The physics track uses CoACD decomposition and simulation drops to determine stable resting poses.Source: §A.2 · p. 15
• BoDex synthesizes dexterous grasps offline for stable poses, caching quality scores and approach depths for each asset.At task initialization, SpatialDR selects a stable pose and SIMPLE retrieves cached grasps for CuRobo trajectory generation.Source: §A.2 · p. 15

A.3 Baseline Implementation Details

• Ψ0 is fine-tuned for 40,000 steps on eight A100 GPUs with a 36-dimensional action output and chunk size 30.The setup uses a per-device batch size of 16, learning rate 1 × 10^-4, 1,000 warmup steps, and bf16 mixed precision.Source: §A.3 · p. 15
• DreamZero uses LoRA fine-tuning for 40,000 steps, predicting a 36-dimensional action at a horizon of 48 alongside future video.Video is rendered at 320 × 176; task context uses 17, 25, or 33 frames, corresponding to chunk sizes 2, 3, or 4.Source: §A.3 · pp. 15–16
• Object preprocessing creates collision geometry, stable resting poses, textured rendering meshes, and cached dexterous grasps for task initialization.The asset pipeline uses CoACD for collision geometry and BoDex for grasp synthesis.Source: Figure 10 · p. 16
• Cosmos3 jointly denoises video latents and action tokens, and is fully fine-tuned for 40,000 iterations on 32 H100 GPUs.The 33-frame egocentric clip is paired with a 32-step normalized-action chunk conditioned on proprioceptive state.Source: §A.3 · p. 16
• π0.5 is expanded from a 30-dimensional to a 36-dimensional action space, while GR00T N1.6 is fine-tuned from its released checkpoint.π0.5 uses a chunk size of 16; GR00T uses sequential inference because RTC inference code is unavailable.Source: §A.3 · p. 16
• Diffusion Policy fits training data reasonably but fails on most evaluation tasks, which the authors attribute to limited visual capacity under SIMPLE’s visual diversity.Inference uses 100 denoising steps.Source: §A.3 · p. 16
• ACT uses a 36-dimensional action head and chunk size 100 with a four-layer encoder and one-layer decoder.Its other hyperparameters match those used for Diffusion Policy.Source: §A.3 · p. 16

A.4 Whole-Body Controller Details

• Both whole-body controllers take upper-body targets and a four-dimensional navigation command, then output Unitree G1 joint-position commands at 500 Hz via MuJoCo.They share a teleoperation interface but use different internal control architectures.Source: §A.4 · p. 17
• AMO [20] separates upper-body target tracking from learned RL locomotion control, so arm and hand motion do not feed back into locomotion, at the cost of tighter coupling between arm posture and balance.Its lower-body policy uses IMU and proprioceptive joint-state inputs to produce 15 lower-body joint targets.Source: §A.4 · p. 17
• SONIC [21] tracks whole-body motion as a unified trajectory, producing smoother, more natural demonstrations, particularly for coordinated arm–leg tasks such as mobile pick-and-place.SIMPLE provides a direct SDK-bridge implementation and a decoupled implementation that converts upper-body targets and navigation commands into full-body position commands.Source: §A.4 · pp. 17–18

A.5 Extended Benchmark Results

• The multi-task benchmark jointly fine-tunes eligible models on all 16 tasks and reports per-task success at three domain-randomization levels; ACT and DP are excluded as single-task methods.Each task-level evaluation averages 20 trials, except DreamZero, which averages 10 due to time constraints.Source: §A.5 · p. 18, Table 8 · p. 18
• Ψ0 [4] leads the multi-task benchmark, followed by DreamZero, GR00T N1.6, and Cosmos3-Nano [42], while π0.5 lags; success generally declines as domain randomization increases.Nearly all models solve CloseDoor and PushOfficeChair, whereas OpenFaucet remains at or below 35% for every model; each task has a non-trivial success rate for at least one model.Source: §A.5 · p. 18, Table 8 · p. 18