PaperlayerView exact revision on arXiv

Source-linked AI summary

Unifying Physics-Based Humanoid Interaction with a Context-Conditioned Interaction Prior

Jianan Li, Xiao Chen, Tien-Tsin Wong

arXiv:2609.06591v1cs.ROcs.GR

TL;DR

Humanoid controllers still face the challenge of coordinating navigation and object manipulation in complex 3D settings. CHIP learns a shared context-conditioned interaction prior from heterogeneous motion data and adapts policies from it for downstream tasks. Evaluations show support for scene-aware locomotion, contact-rich manipulation, and compositional behaviors, with smooth transitions and physically plausible motion.

  • Problem

    Controlling physics-based humanoids across complex environments remains challenging because behaviors must adapt to scene geometry, object state, and task goals while satisfying spatial constraints and physical laws.

    Source: §1 · p. 1

  • Method

    CHIP acquires physics-based teacher behaviors, distills them into a shared discrete context-conditioned interaction prior, and adapts downstream policies with prior-regularized reinforcement learning.

    Source: §1 · p. 2

  • Results

    CHIP supports scene-aware locomotion, contact-rich object manipulation, and compositional environment-aware object transport, with continuous transitions between locomotion and manipulation.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The shared prior captures reusable scene-aware and object-aware behaviors and supports whole-body collision avoidance, compositional interaction, and adaptation to unseen scenes.

    Source: §5 · p. 8

    Interaction grounding depends on the quality and coverage of motion data paired with scene or object context, and current observations are relatively coarse and do not fully exploit semantic understanding.

    Source: §5 · p. 8

Abstract

from arXiv · show

Developing unified physics-based humanoid controllers that can navigate complex 3D scenes and manipulate objects remains a longstanding challenge. Existing approaches are often specialized for either locomotion or object-centric manipulation, or rely on task-specific reward engineering that does not scale well across diverse behaviors. We present CHIP, a unified, physics-grounded framework for learning reusable humanoid interaction skills from heterogeneous motion data. Central to our approach is a conditional interaction prior that models a context-dependent distribution over these skills within a shared discrete space. Our method is trained in three stages. We first learn physics-based motion-imitation policies that acquire grounded teacher behaviors from heterogeneous interaction data. We then distill these behaviors into a context-conditioned interaction prior that captures reusable motion structure across locomotion and manipulation. Finally, we initialize downstream task policies from the pretrained prior and adapt them through prior-regularized online RL post-training. Experiments on a diverse suite of humanoid interaction tasks show that our approach supports scene-aware locomotion, contact-rich object manipulation, and compositional behaviors such as environment-aware object transport and long-horizon skill sequencing, while producing smooth transitions and physically plausible motion.

1 Introduction

CHIP addresses the challenge of controlling humanoids that must coordinate context-dependent navigation and object interaction under spatial constraints and physical laws. It learns a shared, context-conditioned interaction prior and evaluates it across locomotion, manipulation, and compositional behaviors.

1 Introduction

  • Existing methods use task-specific objectives, environment-agnostic priors, or separate object interaction from broader scene-aware navigation, leaving unified physics-grounded context conditioning open.Kinematics-based approaches begin co-modeling scene and object interaction but are not grounded in physical simulation.Source: §1 · p. 2
  • CHIP distills heterogeneous human-scene and human-object interaction behaviors into a shared discrete prior conditioned on scene, object, and task context.Three stages acquire physics-based teacher behaviors, distill a reusable skill prior, and adapt it to downstream tasks through prior-regularized reinforcement learning.Source: §1 · p. 2
  • CHIP supports scene-aware locomotion, contact-rich object manipulation, and compositional environment-aware object transport, with continuous transitions between locomotion and manipulation.The evaluation covers humanoid interaction tasks in complex 3D environments.Source: §1 · p. 2

2 Related Work

Prior work progresses from reward-engineered and imitation-based character control toward reusable skill priors and unified humanoid controllers. The reviewed approaches still face limitations in flexibility, context modeling, or physical grounding.

2.1 Physics-based Character Animation

  • Reward-designed reinforcement-learning controllers can learn specified tasks, but often require substantial task-specific engineering and are difficult to scale across diverse behaviors.Adversarial imitation can produce natural motions, but may suffer unstable training and reduced behavioral diversity from mode collapse.Source: §2.1 · p. 2
  • Motion tracking scales to large skill repertoires and richer interactions, but requires accurate reference trajectories that limit downstream control and task adaptation.The cited work describes tracking as a simpler, scalable alternative to adversarial imitation.Source: §2.1 · p. 2
  • Reusable behavior priors improve scalability, yet often capture task-agnostic motion dynamics and leave task-specific objectives to downstream reward design.This makes complex interactions requiring adaptation to scene geometry or object state difficult to learn efficiently.Source: §2.1 · p. 2

2.2 Unified Humanoid Controllers

  • Unified task interfaces support multi-task execution but constrain flexibility through predefined specifications, while in-painting controllers generate behaviors from sparse kinematic constraints.These approaches are described as falling short for rich interaction in complex 3D environments.Source: §2.2 · pp. 2–3
  • CHIP contrasts with prior controllers by learning a physics-grounded prior from heterogeneous scene and object interaction data, then distilling and adapting it through three stages.The framework uses a shared discrete skill representation and context-conditioned inputs for scene, object, and task information.Source: §2.2 · p. 3, Figure 2 · p. 3

3 Method

CHIP organizes heterogeneous humanoid interaction skills around scene, object, and task context, then distills and adapts them through a shared physics-grounded pipeline. The method progresses from motion-imitation teachers to a discrete interaction prior and prior-regularized downstream RL.

3.1 Overview

  • CHIP first trains imitation experts, distills them into a context-conditioned generative prior, and adapts the resulting skills to downstream tasks through prior-regularized RL.The framework organizes reusable behaviors around interaction context rather than motion alone.Source: §3.1 · p. 3

3.2 Problem Formulation

  • The problem is modeled as a goal-conditioned MDP whose observations include proprioceptive state, multimodal interaction context, and goals.The context space captures scene and object constraints.Source: §3.2 · p. 3

3.3 Stage I: Motion Imitation from Heterogeneous Data

  • CHIP trains specialist physics-based imitation experts for locomotion and object interactions, then unifies their behaviors through generative distillation.Reference-state initialization and kinematic early termination support robust behavior acquisition.Source: §3.3 · pp. 3–4

3.4 Stage II: Distilling the Unified Interaction Prior

  • A shared VQ-VAE skill space compresses teacher behaviors into discrete tokens, and a transformer prior predicts tokens conditioned on proprioceptive history and interaction context.Modality masking supports partial conditioning; the decoder maps selected tokens to joint-space PD targets.Source: §3.4 · p. 4
  • The conditioning interface represents scene context with local heightmaps and object context with BPS shape and per-joint proximity cues, unifying heterogeneous HSI and HOI inputs.Available modalities are tokenized into a unified conditioning sequence.Source: §3.4 · p. 4

3.5 Stage III: Prior-Guided Online RL Post-training

  • Downstream policies are initialized from the pretrained prior and adapted with PPO while the prior, VQ decoder, and codebook remain frozen.A CFG-guided reference prior regularizes the task policy toward a more task-focused distribution while retaining pretrained behavioral support.Source: §3.5 · pp. 4–5

4 Experiments

The experiments evaluate CHIP on navigation, object relocation, and compositional interaction tasks, then test its components and held-out-scene performance. CHIP outperforms baselines on compositional task metrics, with additional results across scene-aware navigation, object manipulation, and skill composition.

4.1 Experimental Setup

  • The evaluation covers navigation, object relocation, and sequential composition; training uses heterogeneous HSI/HOI motions, imitation experts, a discrete context-conditioned prior, and prior-regularized PPO adaptation.The compositional rollout succeeds only when both subgoals are completed; the navigation-to-relocation transition is triggered by a scripted state machine.Source: §4.1 · pp. 5–6

4.2 Results on Unified Humanoid Interaction

  • For Scene Navigation, CHIP has the lowest collision and failure rates while remaining competitive with the strongest baseline in success rate.TokenHSI has the highest navigation success but a higher collision rate; CHIP’s coordinated obstacle avoidance is illustrated in Figure 4.Source: §4.2 · p. 6
  • For Object Relocation, CHIP achieves the best success rate and lowest final distance among the compared methods.The results also show coherent whole-body motion across evaluated object instances and interaction types, without a separate downstream low-level controller for each object category.Source: §4.2 · pp. 6–7
  • On the Compositional Task, CHIP outperforms baselines in full-task success, final distance, collision rate, and failure rate.Table 1 reports quantitative results for Scene Navigation, Object Relocation, and the Compositional Task.Source: §4.2 · p. 7, Table 1 · p. 7
  • Beyond sequential chaining, CHIP supports concurrent object manipulation and obstacle avoidance, adapting whole-body posture throughout the rollout.Figure 5 shows object transport while the humanoid remains aware of its surroundings and bypasses obstacles.Source: Figure 5 · p. 6, §4.2 · p. 7
  • CHIP maintains strong performance on ten scenes excluded from all training stages, with object instances and task and goal distributions unchanged.The held-out-scene test does not evaluate generalization to held-out object instances.Source: §4.2 · p. 7, Table 2 · p. 7

4.3 Ablation Study

  • Ablations show that post-training especially improves compositional success, prior regularization aids contact-rich manipulation, and removing context conditioning harms object skills and cluttered-scene behavior.Without scene and object conditioning, the prior does not encode adaptation to geometry, object state, or task goals.Source: §4.3 · pp. 7–8, Table 3 · p. 7

5 Conclusion

CHIP uses a shared interaction prior to consolidate specialist behaviors into physics-based humanoid control across diverse interaction tasks. The conclusion highlights collision avoidance, object manipulation, compositional skills, smooth transitions, and limitations in data coverage, perception, and task sequencing.

5 Conclusion

  • CHIP consolidates specialist behaviors into a reusable interaction prior for physics-based humanoid control across diverse interaction tasks.The conclusion describes the prior as learned from heterogeneous data and supporting downstream adaptation.Source: §5 · p. 8
  • The framework supports whole-body collision avoidance, contact-rich object manipulation, and compositional interaction skills, including environment-aware transport and multi-stage execution.The conclusion also reports smooth transitions and generalization to unseen scenes.Source: §5 · p. 8
  • Grounding breadth depends on the quality and coverage of motion data paired with scene or object context, which remain limited.The current policy also uses coarse geometric observations and does not fully exploit semantic scene or object understanding; future work could add learned planning and RGB or depth inputs.Source: §5 · p. 8

A.1 Dataset Composition

The pretraining corpus combines locomotion, box transport, human-scene interaction, and human-object interaction data. The appendix reports its composition and expert assignments, including optional text-conditioning data.

A.1 Dataset Composition

  • The filtered pretraining corpus contains 2,826 motion sequences totaling 12.17 hours across locomotion, box transport, HSI, and HOI data.The four components include standalone locomotion, box transport, a TRUMANS subset, and an OMOMO subset.Source: §A.1 · p. 10
  • TRUMANS contributes 1,486 HSI sequences and 8.71 hours, while OMOMO contributes 1,214 HOI sequences and 2.07 hours.Standalone locomotion and box-transport components contribute 126 sequences and 1.38 hours combined.Source: §A.1 · p. 10

A.2 Expert Assignment and Conditioning Modalities

  • Expert assignments divide object-centric data among trashcan, plasticbox, largetable, and woodchair, with separate locomotion/box-transport and TRUMANS idle experts.The object-centric experts contain 317, 300, 237, and 360 sequences, respectively; 1,214 sequences carry optional text annotations and CLIP embeddings.Source: §A.2 · p. 10

B.1 Training Configurations

The appendix summarizes configurations for CHIP’s three training stages and gives additional details about the Stage I imitation objective and Stage II skill distillation and prior learning.

B.1 Training Configurations

  • Tables 5–7 summarize architectures and optimization settings across the three training stages.For Stage III, the appendix specifies PPO, a pretrained interaction-prior initialization, and training the task policy while freezing the prior, decoder, and codebook embeddings.Source: §B.1 · pp. 10–11, Table 7

B.2 Stage I Motion-Imitation Objective

  • Stage I tracking terms cover the pelvis anchor and 14 equally weighted bodies, with object tracking when an object is active and wrist/ankle contacts permitted.The control step is 0.02 seconds, and reported experiments use fixed bandwidths with adaptive updates disabled.Source: §B.2 · p. 11

B.3 Stage II Skill-Distillation and Prior-Learning Objectives

  • Stage II uses a straight-through quantized representation that selects a code in the forward pass while passing decoder gradients to the encoder.The appendix gives a behavior-cloning term, a complete distillation objective with β_com = 0.25, and codebook updates by exponential moving averages with decay 0.99.Source: §B.3 · pp. 11–12
  • Prior learning uses five frames of proprioceptive history and independently masks each available context or task-goal modality with probability 0.3.The passage identifies the teacher token as q_E.Source: §B.3 · p. 12

B.4 Baseline Implementations

The appendix describes baseline implementations and downstream task specifications for navigation and object relocation. It details shared reward regularizers and task-specific observations, shaping, and training termination conditions.

B.4 Baseline Implementations

  • TokenHSI is trained as a task-oriented unified-policy baseline, while MaskedMimic is evaluated zero-shot without downstream RL adaptation.Both baselines share CHIP’s motion corpus, conditioning signals, embodiment, simulation and control frequencies, evaluation protocol, and metric definitions.Source: §B.4 · p. 12

C Downstream Tasks

  • Downstream RL tasks combine sparse success or failure signals with task-specific shaping rewards and shared penalties for action-rate smoothness, joint-limit violations, undesired contacts, and energy.Table 9 is described as summarizing task-specific observations and reward weights.Source: C Downstream Tasks · p. 12

C.1 Scene Navigation

  • Scene navigation uses a heading-aligned 16 × 16 heightmap over a 1.6 m × 1.6 m region and target displacement, with rewards for reaching and progressing toward the target.Training also penalizes scene collisions and excessive root velocity and rewards stillness near the target.Source: §C.1 · p. 12

C.2 Object Relocation

  • Object-relocation policy observations combine the scene-navigation local 16 × 16 heightmap with object-centric geometry and goal features, encoding body–object proximity via nearest surface points from 256 sampled object points to 32 robot bodies.Object geometry uses 512 fixed unit-ball basis points transformed by the humanoid root pose, with nearest distances to the object surface; the goal occupies the first three entries of the shared padded 10-D target tensor, is not distance-clipped, and is relative to the humanoid rather than an object-to-goal displacement.Source: §C.2 · p. 12
  • Object-relocation training uses distance-improvement shaping and terminates successfully when 3-D object-target distance stays below 0.15 m for 2.5 seconds.This training termination differs from the main-paper x-y-plane distance metric; failure occurs when humanoid base height falls below its threshold.Source: §C.2 · p. 13

C.3 Compositional Task

The compositional task scripts a navigation-to-relocation stage switch while leaving control in both stages to the learned task policy.

C.3 Compositional Task

  • During navigation, the policy observes the local heightmap and treats the current object position as its planar target; relative object displacement in the humanoid heading frame is clipped to a maximum norm of 4 m.Source: §C.3 · p. 13
  • After navigation succeeds, the target switches to the 3-D object goal and relocation observations and rewards replace navigation inputs.The rollout succeeds only when both stages are completed in sequence; only their transition is scripted.Source: §C.3 · p. 13
  • During optimization, each raw reward term is multiplied by its weight and the simulation step duration.Table 9 lists downstream-task observation components and reward weights, with common penalties shared across tasks.Source: §C.3 · p. 13, Table 9 · p. 13

Related papers

Loading 2609.06591v1…