Source-linked AI summary
MimicKit: A Reinforcement Learning Framework for Motion Imitation and Control
Xue Bin Peng
TL;DR
MimicKit addresses the practical complexity of building motion-imitation systems by unifying training methods and tools in a modular, extensible framework. It applies the same framework across diverse characters and challenging behaviors, with minimal modifications across morphologies.
Problem
Building effective motion-imitation systems involves implementation challenges and detailed design decisions, motivating a unified framework for experimentation and reproducible research.
Source: §1 · p. 1Method
Results
MimicKit trains policies for a wide range of challenging, highly dynamic behaviors across SMPL humanoids, Unitree G1 humanoid robots, and Unitree Go2 quadrupedal robots, with minimal framework modifications across morphologies.
Source: §7.1 · p. 11Takeaways & Limitations
The framework provides a unified, extensible platform intended to support reproducible research in motor-skill learning and progress in learning-based motion control.
Source: §8 · pp. 12–13DeepMimic’s motion-tracking objective can produce inflexible policies restricted to reference motions, limiting adaptation of dataset behaviors for new tasks.
Source: §4.1 · p. 5
Abstract
from arXiv · showhide
MimicKit is an open-source framework for training motion controllers using motion imitation and reinforcement learning. The codebase provides implementations of commonly-used motion-imitation techniques and RL algorithms. This framework is intended to support research and applications in computer graphics and robotics by providing a unified training framework, along with standardized environment, agent, and data structures. The codebase is designed to be modular and easily configurable, enabling convenient modification and extension to new characters and tasks. The open-source codebase is available at: https://github.com/xbpeng/MimicKit.
1 INTRODUCTION
MimicKit brings motion-imitation training methods and tools together in an extensible framework to lower the barrier to experimentation and reproducible research.
1 INTRODUCTION
- MimicKit unifies high-quality implementations of training methods and tools in an extensible framework.The framework is designed to lower the barrier to experimentation and reproducible research in motion imitation.Source: §1 · p. 1
- RL-based motion imitation is presented as a versatile, effective way to build agile, life-like controllers for simulated characters and real-world robots.Source: §1 · p. 1
- Building effective motion-imitation systems requires attention to numerous nuances and design decisions that can be challenging to implement.Source: §1 · p. 1
2 BACKGROUND
The background describes an agent learning a policy through interactions with an environment, with the objective of maximizing expected discounted return. The framework diagram separates learning components from task simulation.
2 BACKGROUND
- At each step, the agent receives an observation, samples an action from its policy, and the environment transitions to a new state.Source: §2 · pp. 1–2
- The framework diagram assigns learning algorithms primarily to the Agent and Model, while the Environment and Engine simulate the desired task.Source: Figure 2 · p. 2
- An agent’s objective is to learn a policy that maximizes expected discounted return over trajectories of observations, actions, and rewards.Trajectory likelihood is defined under the policy, with a finite time horizon and discount factor γ ∈ [0, 1].Source: §2 · p. 2
3 SYSTEM OVERVIEW
MimicKit separates learning, task logic, and low-level simulation into modular, composable components. This design supports parallel data collection and configuration across different characters, tasks, and simulators.
3 SYSTEM OVERVIEW
- The overview divides MimicKit into modular, composable components for learning, task simulation, and the underlying engine, supporting GPU-parallel data collection and character-agnostic configuration.The framework can be configured for humanoid and non-humanoid morphologies, including quadrupedal robots.Source: §3 · p. 2
3.1 Agent
- The Agent runs the learning procedure, processes environment interactions, stores experience, and uses collected batches to update its paired Model.Agent and model types and hyperparameters are selected through configuration files.Source: §3.1 · pp. 2–3
3.2 Model
- The Model implements neural-network architectures and can provide policy, value-function, discriminator, or other outputs required by the learning algorithm.Actor-critic methods may use separate actor and critic networks, while AMP may add a discriminator network.Source: §3.2 · p. 3
3.3 Environment
- The Environment constructs observations and processes agent actions into commands, then returns transition rewards and episode-termination information.Termination flags distinguish failure, success, and time-limit endings; time truncation can use value-function bootstrapping to estimate future returns.Source: §3.3 · pp. 3–4
3.4 Engine
- The Engine abstracts low-level simulation behind a shared API, receiving commands and returning updated states across supported simulators and control modes.Its configuration specifies the engine, control model, and control and simulation frequencies; the pd_1d mode is limited to morphologies with only 1D revolute joints.Source: §3.4 · p. 4
4 METHODS
MimicKit offers motion-imitation methods with different trade-offs, from precise reference tracking to distribution matching and reusable skills. It also includes an adaptive tracking method and both on-policy and off-policy RL algorithms.
4.1 DeepMimic
- DeepMimic trains an RL tracking controller to follow reference motions, offering a general and reliable option for precise replication but potentially inflexible policies.Its tracking objective can limit adaptation of dataset behaviors to new tasks [Peng et al. 2018].Source: §4.1 · p. 5
4.2 Adversarial Motion Priors (AMP)
- AMP matches the behavioral distribution, or style, of motion clips without tracking a specific clip, allowing behavior composition but risking local optima on challenging, highly dynamic motions [Peng et al. 2021].Compared with tracking-based methods, AMP may struggle more to replicate challenging behaviors closely.Source: §4.2 · p. 5
4.3 Adversarial Skill Embeddings (ASE)
- ASE combines adversarial imitation with mutual-information-based skill discovery to learn latent skill embeddings that a controller maps to diverse behaviors [Peng et al. 2022].Task-specific high-level controllers can later select skills from the learned latent space.Source: §4.3 · p. 6
4.4 Adversarial Differential Discriminator (ADD)
- ADD uses a differential discriminator to learn adaptive motion-tracking objectives and mitigate manual tracking-reward design and tuning across characters and motions [Zhang et al. 2025].Source: §4.4 · p. 6
- Most MimicKit methods use PPO as the underlying RL algorithm [Schulman et al. 2017], while AWR [Peng et al. 2019] is offered as an off-policy alternative to address settings requiring off-policy learning.The paper notes that PPO is on-policy and can be sample inefficient.Source: §4.4 · p. 6
5 INSTRUCTIONS
The instructions cover installing MimicKit, training and testing models, and using tools to inspect simulations and training logs. Users can configure training through command-line arguments or argument files and distribute it across devices.
5.1 Installation
- Install a supported simulator, the Python dependencies, and the framework’s assets and motion data before use.Supported simulators include Isaac Gym [Makoviychuk et al. 2021], Isaac Lab [NVIDIA et al. 2025], and Newton [Newton Contributors 2025].Source: §5.1 · pp. 6–7
5.2 Training
- Training commands select the engine, environment, and agent configurations, parallel environment count, visualization, and output directory.Settings can be supplied in an argument file and overridden with additional command-line arguments.Source: §5.2 · p. 7
5.3 Distributed Training
- Training can be distributed across multiple CPU or GPU devices, with the device list specifying the processes used.The standard command runs a single process; multiple devices can parallelize training.Source: §5.3 · pp. 7–8
5.4 Testing
- Training saves model parameters to a checkpoint, which can be loaded in test mode alongside a chosen environment count and visualization setting.Pretrained models and corresponding training logs are also provided.Source: §5.4 · p. 8
5.5 Visualizer User Interface
- A shared visualizer interface supports camera movement, pausing, and stepping, while individual engines may add simulator-specific controls.Camera controls use Alt-drag and the mouse wheel; Enter pauses or resumes, and Space advances one step.Source: §5.5 · p. 8
5.6 Visualizing Training Logs
- Training statistics can be viewed with runtime loggers or by plotting output text logs with the provided plotting script.TensorBoard events are saved in the output directory when that logger is used.Source: §5.6 · p. 8
6 MOTION DATA
MimicKit uses motion clips or datasets of clips as reference data for training and testing. Motion files encode character root and joint motion, with rotations represented according to joint dimensionality.
6 MOTION DATA
- The environment configuration’s motion_file field selects either an individual reference clip or a dataset containing multiple clips.Example clips are provided in data/motions/ and datasets in data/datasets/.Source: §6 · p. 8
- The view_motion environment can visualize motion clips, and the Motion class stores each frame as root position, root rotation, and joint rotations.The Humanoid example orders joint rotations by depth-first traversal of its kinematic tree.Source: §6 · pp. 8–9
- Three-dimensional joint rotations use exponential maps, while one-dimensional joint rotations use rotation angles.The Humanoid character illustration identifies the simulated character used in the example.Source: §6 · p. 9, Figure 3 · p. 9
7 EXPERIMENTS
MimicKit’s motion-imitation experiments evaluate tracking and motion fidelity across diverse reference behaviors using simulated humanoids. DeepMimic and ADD track reference motions more accurately than AMP, while ADD can yield more consistent results across training runs.
7 EXPERIMENTS
- The experiments assess quantitative tracking and qualitative fidelity across motions from everyday behaviors to acrobatics and martial arts.Tasks use IsaacGym, simulated humanoids, motion-capture clips, and separate policies per clip; pose-error termination is disabled for the comparison.Source: §7 · p. 9, §7.1 · p. 9
7.1 Motion Imitation
- DeepMimic and ADD more accurately reproduce reference motions than AMP, whose distribution-matching policies can still reproduce general behaviors without precise clip tracking.Table 1 reports position and DoF velocity tracking errors across five models and 4096 test episodes per model.Source: Table 1 · p. 11, §7.1 · p. 12
- Learned policies span challenging skills and different morphologies, including an SMPL-based humanoid, a Unitree G1, and a Unitree Go2, with minimal framework modifications.Figure 4 shows physically simulated characters imitating motion data recorded from real-life actors.Source: Figure 4 · p. 10, §7.1 · p. 11
- Position tracking error measures differences in global root position and relative joint positions between simulated and reference motion.The error averages over the root and the character’s joints.Source: §7.1 · p. 11
- DoF velocity tracking error measures differences between simulated and reference local angular velocities at each joint.The equation averages the joint-level velocity differences.Source: §7.1 · p. 11
- ADD’s learned adaptive reward can produce more consistent performance across diverse motions than DeepMimic’s manually designed reward without additional heuristics.Pose-error termination generally improves tracking accuracy and consistency for tracking-based methods.Source: §7.1 · p. 12
8 CONCLUSION
The conclusion presents MimicKit as an open-source reinforcement-learning framework for motion imitation and control. Its modular, extensible design is intended to support reproducible research and provide a convenient platform for motion-control research.
8 CONCLUSION
- MimicKit is an open-source reinforcement-learning framework that unifies motion-imitation methods for training motion controllers.The framework is described as modular and extensible.Source: §8 · pp. 12–13
- The framework is intended to facilitate reproducible research in motor-skill learning.This is presented as an intended outcome.Source: §8 · p. 13
- MimicKit is intended as a convenient platform to accelerate progress in learning-based motion control.The conclusion frames this as a hope for the framework’s use.Source: §8 · p. 13