PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Expressive Humanoid Locomotion from Monocular Runway Videos for Robot Fashion Shows

Kyrylo Kolesnichenko, Irvin Steve Cardenas, Jong-Hoon Kim

arXiv:2609.27003v1cs.RO

TL;DR

Runway catwalking calls for coordinated expressive motion, while typical humanoid policies emphasize balance and speed rather than garment presentation. This paper builds a video-to-policy workflow and evaluates it on Booster K1. The robot completed all physical trials without falling while exhibiting narrow foot placement and coordinated body movement.

  • Problem

    Humanoid fashion-show robots typically use policies optimized for balance, velocity tracking, and disturbance recovery, while catwalking also requires coordinated expressive motion for garment presentation.

    Source: §I · p. 1

  • Method

    The pipeline converts monocular human video into a deployable humanoid policy through motion recovery, retargeting, correction, training, simulation testing, and deployment.

    Source: §I · p. 1, §III · p. 1

  • Results

    The learned policy completed all 20 physical trials without falling and showed narrow foot placement with coordinated leg, torso, and arm movements.

    Source: A. Physical Execution · p. 2

  • Takeaways & Limitations

    The demonstrated Booster K1 workflow is described as usable for other custom motions in fashion, entertainment, and related applications.

    Source: §I · p. 1

Abstract

from arXiv · show

Runway walking requires coordinated control of posture, stride, foot placement, and whole-body motion to effectively present clothing and convey a distinctive style. However, humanoid robots used in fashion shows typically rely on locomotion policies optimized primarily for stability and walking speed, limiting their ability to reproduce expressive, human-like runway motions. In this work, we present an end-to-end framework that transforms monocular runway videos into deployable humanoid locomotion policies through motion recovery, robot retargeting, motion correction, policy training, simulation-based evaluation, and physical deployment. We evaluate the proposed framework on the Booster K1 humanoid robot using runway-style catwalk motions. The learned policy completed every physical trial without falling, while reproducing the characteristic narrow foot placement and coordinated movement of the legs, torso, and arms. The results demonstrate that our proposed training framework enables the Booster K1 to perform stable and expressive catwalk motions, highlighting its potential for humanoid robotic applications in fashion shows and other performance-oriented scenarios.

I. INTRODUCTION

Catwalking coordinates whole-body motion to present clothing and communicate a fashion show’s aesthetic, but conventional humanoid policies prioritize balance, speed, and recovery. The paper proposes an end-to-end video-to-policy workflow and demonstrates it on Booster K1 catwalk motion.

I. INTRODUCTION

  • Catwalking coordinates posture, cadence, stride, foot placement, arm motion, gaze, posing, and turning to present garments and communicate fashion-show aesthetics.Styles vary with designers, garments, footwear, and show direction.Source: §I · p. 1
  • Conventional humanoid locomotion policies prioritize balance, velocity tracking, and disturbance recovery, while a general data-driven framework for transferring multiple runway styles to physical humanoids remains absent.The paper frames catwalking as style-conditioned locomotion assessed through motion and stability measures and qualitative judgments of style, confidence, and fashion-show appropriateness.Source: §I · p. 1
  • The proposed pipeline converts a human video into a robot-ready policy through motion recovery, retargeting, correction, training, simulation testing, and deployment.The user supplies the video and selects the target robot.Source: §I · p. 1
  • The Booster K1 runway-style catwalk is the demonstration, while the workflow is also described as applicable to other custom motions in fashion, entertainment, and related applications.The paper presents video as a way to provide the desired behavior instead of manually creating robot trajectories.Source: §I · p. 1

II. RELATED WORK

Related work covers separate video recovery, retargeting, policy training, and deployment tools, alongside studies of expressive robot gaits and runway-inspired motion. The paper positions its contribution as a unified workflow for learning and transferring runway styles.

II. RELATED WORK

  • GVHMR recovers ground-aligned 3D human motion from video, and GMR [2] adapts that motion to a selected robot.Source: §II · p. 1
  • BeyondMimic [3] trains a tracking policy using the Booster training framework, while the Booster deployment framework tests and runs it; these tools do not form one video-to-robot workflow.Source: §II · p. 1
  • Prior work simulated fashion-model-like humanoid motion and generated and evaluated expressive gaits; leg crossing identified among runway styles motivates comparing step width.Source: §II · p. 1

III. METHOD

The method turns monocular human video into a policy that can be tested and run on a humanoid robot, with the user selecting the video and a GMR-supported robot.

III. METHOD

  • The pipeline converts monocular human video into a robot policy through 3D motion generation, retargeting, motion correction, policy training, and deployment.The user provides the video and selects a robot supported by GMR [2].Source: §III · p. 1
  • Fig. 1 lays out the workflow from input video through recovered human motion and simulated retargeting to physical deployment on Booster K1.Source: §III · p. 1
  • The workflow’s final product is a runnable robot policy rather than only a retargeted trajectory or training checkpoint.Source: §III · p. 1

A. Motion Recovery

Motion recovery uses GVHMR to derive 3D human motion from video, with camera-setting selection and subsequent transfer of SMPL-X motion and a global path to retargeting.

A. Motion Recovery

  • GVHMR converts the input video into 3D human motion, selecting a setting according to the video frame rate and whether the camera is static or moving.Source: A. Motion Recovery · p. 1
  • The recovery stage sends the resulting SMPL-X motion and global path to retargeting.Source: A. Motion Recovery · p. 1
  • Fig. 1 depicts motion recovery between the monocular runway video input and retargeted robot motion in simulation.Source: Figure 1 · p. 2

B. Robot Retargeting

GMR adapts human motion to a selected robot and produces a robot-specific motion trajectory.

C. Motion Correction and Conversion

The pipeline corrects and converts retargeted motion for training, then trains against that processed motion while matching hardware control settings.

C. Motion Correction and Conversion

  • Foot contacts are detected, feet are aligned with the floor, and vertical drift is removed without changing joint angles before conversion to training format.The converted motion includes robot joint states, base pose, and source-video frame timing.Source: C. Motion Correction and Conversion · p. 2

D. Policy Training

  • The processed motion trains the policy with BeyondMimic in the Booster framework, using actuator and action settings consistent with deployment.The stated consistency covers actuator limits, stiffness, damping, and action scaling.Source: D. Policy Training · p. 2

E. Simulation and Deployment

The exported policy is checked in MuJoCo before physical deployment, and the Booster K1 setup uses a marked straight runway with safety belts.

E. Simulation and Deployment

  • The exported policy is tested in MuJoCo with the deployment controller for balance, tracking, and joint behavior before it is run on the physical robot.The workflow’s final output is a runnable robot policy, not only a retargeted trajectory or training checkpoint.Source: E. Simulation and Deployment · p. 2

IV. EXPERIMENTAL SETUP

  • The catwalk policy was trained with BeyondMimic in Booster’s framework, tested in MuJoCo, and deployed on the Booster K1 using a custom script and wired Ethernet.Source: §IV · p. 2
  • Tests used a taped, straight 5 m path with start, centerline, and distance markings; safety belts were loose during normal walking.Each test contained approximately 23 steps.Source: §IV · p. 2

A. Physical Execution

The learned policy completed all physical trials without falling and used narrower foot placement than the standard K1 walk.

A. Physical Execution

  • All 20 physical trials were completed without falling, with approximately 23 steps per trial.The robot showed narrow foot placement and coordinated leg, torso, and arm movements; the first step sometimes caused a small lateral offset before a stable path approximately parallel to the runway line.Source: A. Physical Execution · p. 2

B. Foot-Placement Comparison

  • The comparison measured lateral distance between consecutive foot placements over 12 standard-walk steps and 23 catwalk-policy steps; negative values indicate crossover steps.Source: B. Foot-Placement Comparison · p. 2
  • The catwalk policy’s step widths were −0.8 to 1.8 cm, versus 5.1 to 11.4 cm for the standard K1 walk.The learned policy therefore achieved substantially narrower foot placement, with occasional crossover steps.Source: B. Foot-Placement Comparison · p. 2

VI. FUTURE WORK

Future work targets policy controllability, broader catwalk data and evaluation, and deployment beyond the Booster K1.

VI. FUTURE WORK

  • Future work includes developing a controllable policy and creating a larger catwalk motion dataset.Source: §VI · p. 2
  • The authors propose improving evaluation metrics and involving fashion experts in assessing the robot’s walking style.Source: §VI · p. 2
  • They also plan to train and deploy the pipeline on humanoid robots beyond the Booster K1.Source: §VI · p. 2
Loading 2609.27003v1…