PaperlayerView exact revision on arXiv

Source-linked AI summary

CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character control

Guy Tevet, Sigal Raab, Setareh Cohan, Daniele Reda, Zhengyi Luo, Xue Bin Peng, Amit H. Bermano, Michiel van de Panne

arXiv:2410.03441v1cs.CV

TL;DR

Physics-based RL enables plausible motion and object interaction, but scaling it to large motion datasets is difficult. CLoSD couples text- and target-conditioned diffusion planning with a fine-tuned tracking controller in a closed loop. One instance performs several interaction tasks, outperforms leading text-to-motion and multi-task controllers, and generates plans at 175× real time.

  • Problem

    Physics-based RL controllers commonly rely on individual clips or small curated datasets, and scaling them to larger human-motion datasets is difficult.

    Source: §1 · p. 1

  • Method

    CLoSD uses a real-time autoregressive diffusion planner conditioned on text and a target location, with a fine-tuned RL tracking controller executing plans and returning simulated motion as feedback.

    Source: §1 · p. 1, §4.3 · p. 7

  • Results

    One CLoSD instance performs goal-reaching, target-striking, and couch interactions, outperforms leading text-to-motion and state-of-the-art multi-task controllers, and generates 40-frame plans at 3,500 fps (175× real time).

    Source: §1 · p. 2

  • Takeaways & Limitations

    CLoSD performs text-directable multi-task sequences involving navigation, striking with specified hands or feet, sitting down, and getting up through a fast diffusion planner and a tracker fine-tuned for its plans.

    Source: §6 · p. 10

    The work focuses on mid- and low-level planning; longer-timescale planning and exteroception such as vision or height maps remain future directions.

    Source: §6 · p. 10

Abstract

from arXiv · show

Motion diffusion models and Reinforcement Learning (RL) based control for physics-based simulations have complementary strengths for human motion generation. The former is capable of generating a wide variety of motions, adhering to intuitive control such as text, while the latter offers physically plausible motion and direct interaction with the environment. In this work, we present a method that combines their respective strengths. CLoSD is a text-driven RL physics-based controller, guided by diffusion generation for various tasks. Our key insight is that motion diffusion can serve as an on-the-fly universal planner for a robust RL controller. To this end, CLoSD maintains a closed-loop interaction between two modules -- a Diffusion Planner (DiP), and a tracking controller. DiP is a fast-responding autoregressive diffusion model, controlled by textual prompts and target locations, and the controller is a simple and robust motion imitator that continuously receives motion plans from DiP and provides feedback from the environment. CLoSD is capable of seamlessly performing a sequence of different tasks, including navigation to a goal location, striking an object with a hand or foot as specified in a text prompt, sitting down, and getting up. https://guytevet.github.io/CLoSD-page/

1 INTRODUCTION

Physics-based RL can produce plausible interactions, but scaling controllers to larger motion datasets is difficult. CLoSD combines real-time diffusion planning with a tracking controller in a closed feedback loop for text-directed, multi-task control.

1 INTRODUCTION

  • Physics-based RL often uses individual clips or small curated datasets for task-specific controllers, making it difficult to scale to larger human-motion datasets.Those reference motions help RL produce human-like behavior while simplifying exploration and modeling the desired motion manifold.Source: §1 · p. 1
  • DiP is an autoregressive real-time diffusion planner conditioned on text and a target location, generating high-fidelity motions in as few as 10 diffusion steps.The target can specify an arbitrary joint, such as a hand for a hook punch or a foot for a high kick.Source: §1 · p. 1
  • A robust RL tracking controller executes DiP’s plan, and simulated motion feeds back to DiP so it can generate the next plan and respond to changing commands.The tracker is fine-tuned with DiP in the loop across multiple tasks to address its prior mismatch with object interaction and real-time planning.Source: §1 · p. 2, Figure 1 · p. 2
  • One CLoSD instance performs goal-reaching, target-striking, and couch interactions, outperforms leading text-to-motion and multi-task controllers, and generates 40-frame plans at 3,500 fps (175× real time).The cited comparison also notes that these tasks previously required dedicated policies and considerable reward engineering.Source: §1 · p. 2

2 RELATED WORK

Related work spans data-driven motion synthesis, physics-based character control, and motion imitation. CLoSD is positioned at their intersection: text-directed control with fine-grained physical interaction and a general motion-tracking foundation.

2 RELATED WORK

3 PRELIMINARIES

The preliminaries describe how CLoSD connects diffusion-generated kinematic plans to a physics-based tracking policy. They establish the motion representations and diffusion prediction setup used by that loop.

3 PRELIMINARIES

  • Diffusion motions use HumanML3D’s frame-relative representation, while the RL policy uses PHC states in world-axis global position and velocity.R2G converts relative motion to global form by accumulation, and G2R converts simulated global states back to relative motion.Source: §3 · pp. 3–4
  • DiP conditions a motion plan on a text prompt and target location; the RL tracking controller executes it, and resulting frames return to DiP as a prefix for the next plan.Figure 2 also depicts a single DiP denoising iteration.Source: Figure 2 · p. 4
  • The DDPM setup gradually noises a data motion toward a Gaussian distribution and models conditioned synthesis as the reversed process that cleans the noisy motion.The model predicts the motion sequence itself rather than the noise, following MDM.Source: §3 · p. 4
  • The simple diffusion objective trains the model to predict the clean motion sequence, a formulation reported to improve motion results and enable geometric target loss at each denoising step.The objective uses squared error between the clean motion and its prediction.Source: §3 · p. 4

4 CLOSD

CLoSD combines a text- and target-conditioned diffusion planner with a motion-tracking controller in a closed loop. Its planner supplies motion plans, the controller executes them in simulation, and task transitions can be handled by a state machine.

4 CLOSD

  • CLoSD pairs a real-time Diffusion Planner (DiP) that synthesizes kinematic plans with a robust motion-tracking controller that executes them.The modules operate in a closed loop, with realized motion returned to DiP as an input prefix.Source: §4 · pp. 4–5

4.1 MOTION DIFFUSION PLANNER

  • DiP autoregressively plans from prior motion frames, a text prompt, and target joint locations, grounding task descriptions in the physical environment.Adaptive validity signals allow the conditioned joint subset to change with the task.Source: §4.1 · p. 5
  • 11.4 ms generates 2 seconds of reference motion, or 175× real time, using shorter segments and reducing denoising steps from 50 to 10.Shorter segments reduce self-attention cost because runtime depends quadratically on sequence length.Source: §4.1 · p. 6

4.2 UNIVERSAL TRACKING POLICY

  • The tracking policy converts kinematic reference poses into simulated humanoid actions, using a single PHC primitive for lightweight fine-tuning.Its state includes the reference-versus-simulated pose difference and the reference pose.Source: §4.2 · pp. 6–7

4.3 CLOSING THE LOOP

  • Closed-loop PPO fine-tuning addresses tracking gaps around generated-motion artifacts and object interaction, training one policy across randomly selected tasks.DiP is fixed during episodes; the task, object, prompt, and target location are set for each episode.Source: §4.3 · p. 7

4.4 HIGH-LEVEL PLANNING USING A STATE MACHINE

  • A state machine advances to another task when a task-specific done signal occurs, changing the prompt and target conditions on the fly.Examples include sitting when the pelvis reaches the sofa area and striking when the target hits the ground.Source: §4.4 · p. 7

5 EXPERIMENTS

The experiments evaluate task success across goal-reaching, striking, sitting, and getting up, and text-to-motion performance on HumanML3D. They describe the implementation and compare CLoSD with single- and multi-task controllers and text-to-motion methods.

5.1 IMPLEMENTATION DETAILS

  • DiP generates 40-frame plans at 3,500 frames per second, and its ablation performs well with as few as five diffusion steps.The implementation uses 10 diffusion steps, a 20-frame prefix, and a 40-frame generation length; the tracker is trained on AMASS and fine-tuned in the closed loop.Source: §5.1 · p. 8, Table 2 · p. 8

5.2 TASK EVALUATION

  • A single closed-loop tracking controller is fine-tuned across tasks simultaneously, using text prompts and target locations without reward engineering.The evaluated tasks include pelvis-target goal-reaching, striking a bag with a sampled hand or foot, sitting on a sofa, and getting up from it.Source: §5.2 · pp. 8–9
  • CLoSD significantly excels on striking and getting up, while open-loop and untuned tracking baselines struggle on interaction tasks.Task success rates are compared over 1K episodes per task; the table caption highlights striking and getting up as requiring careful object interaction.Source: Table 1 · p. 8, §5.2 · p. 9

5.3 TEXT-TO-MOTION EVALUATION

  • Text-to-motion evaluation uses HumanML3D metrics for text-motion alignment, distribution distance, diversity, and motion-text distance, alongside physical plausibility metrics.The physical metrics cover penetration, floating, and skating.Source: §5.3 · p. 10
  • Against MoConVQ, CLoSD excels in all reported text-to-motion metrics; fine-tuning improves results, while longer plans benefit pure text-to-motion.Physics-based text-to-motion generally improves physical metrics but degrades data-distribution metrics; two-second plans are presented as a trade-off because object interaction needs closed-loop feedback.Source: §5.3 · p. 10, Table 3 · p. 10

6 CONCLUSIONS AND LIMITATIONS

CLoSD combines diffusion planning with physics-based simulation for text-directed multi-task sequences involving object interaction. Its future directions include perception, longer-horizon planning, and greater adaptivity in the loop.

6 CONCLUSIONS AND LIMITATIONS

  • CLoSD combines diffusion planning with physics-based simulation to produce text-prompt-directed sequences including navigation, striking, sitting, and getting up.A fast task-conditioned autoregressive diffusion model works with a motion-tracking controller fine-tuned to handle generated plans.Source: §6 · p. 10
  • The work focuses on mid- and low-level planning; proposed extensions include exteroception, longer-time-scale planning, and temporal adaptivity.Vision or height maps could help when planning requires more than a target location; longer tracking horizons could mitigate occasional artifacts.Source: §6 · p. 10
Loading 2410.03441v1…