PaperlayerView exact revision on arXiv

Source-linked AI summary

From Digital Human Interactions to Physics-Based Humanoid Skills: Physics-Grounded Post-Training of Interaction Generators

Kerui Chen, Jianrong Zhang, Kai Lv, Hehe Fan

arXiv:2610.10322v1cs.CV

TL;DR

Tracking-based interaction generation can fail to preserve intended contacts or physically plausible forces. DIGHT aligns a generator using simulator-grounded preferences, then adapts the tracker to its motions. It improves physical errors and contact-topology scores in simulation.

  • Problem

    Tracking-based methods can match reference poses yet miss intended contacts or produce destabilizing forces, and generated interactions lack explicit inference-time physics constraints.

    Source: §1 · pp. 1–2

  • Method

    DIGHT uses simulator rollouts to form separate preferences for executability and contact fidelity, aligns the generator with physics-decoupled diffusion DPO, and then adapts the tracker.

    Source: §1 · pp. 2–3

  • Results

    Across four dataset-backbone settings, DIGHT reduces all five physical errors; contact-topology precision rises from 38.29% to 46.77% and recall from 52.16% to 63.94% over InterGen.

    Source: §1 · p. 3

  • Takeaways & Limitations

    DIGHT improves physical executability and interaction fidelity for humanoid interactions in simulation.

    Source: §6 · p. 9

Abstract

from arXiv · show

Recent methods have made promising progress in generating interactions between two humanoids, largely relying on physics-based tracking policies to convert digital reference motions into executable trajectories. However, limited tracking capabilities restrict the range of reference motions that can be successfully executed, reducing data utilization. Moreover, even successful tracking does not guarantee physically plausible responses or faithful realization of the intended interactions. In this paper, we introduce DIGHT, a co-adaptive framework that couples a Digital human Interaction Generator with a Humanoid Tracking policy. Our DIGHT first executes multiple text-conditioned interaction candidates in simulation using a fixed tracker. It then constructs physics-grounded preferences from the resulting rollouts, covering both general executability and interaction fidelity. Rather than collapsing these signals into a single scalar reward for candidate ranking, we align the pretrained generator using physics-decoupled diffusion direct preference optimization (DPO), preserving criterion-specific supervision without differentiating through the simulator. To improve executability, preference pairs are derived from tracking error, friction, and floating. Additionally, to improve interaction fidelity, we propose to incorporate force feedback from simulator as a measure of contact fidelity and construct preferences over contact occurrence, location, duration, and force magnitude. The aligned generator then supplies reference motions for fine-tuning the tracker, improving compatibility between generation and physical execution. Extensive experiments demonstrate that our approach not only improves the physical plausibility of generated motions but also enables more reliable and faithful humanoid interactions in simulation.

1 INTRODUCTION

Two-humanoid interaction requires both executable individual motions and faithful coordination, but tracking alone can miss intended contacts or produce destabilizing forces. DIGHT uses simulation feedback to co-adapt an interaction generator and tracker, improving physical errors and contact-topology scores.

1 INTRODUCTION

  • Text-conditioned humanoid interaction is intended to support embodied coordination and physical response while producing semantically meaningful, physically realizable motion.The paper situates this capability in collaborative agents, social robotics, character animation, and embodied simulation.Source: §1 · p. 1
  • Two-humanoid control must coordinate spatial configurations, timing, and physical interaction because deviations can cause missed contacts, implausible forces, or unstable execution.Successful interaction requires both general executability and fidelity to intended contacts and coordination.Source: §1 · pp. 1–2
  • Tracking-focused approaches can match poses yet lose intended hand contact or generate excessive shoulder-patting forces, while InterAgent learns feasibility indirectly without explicit inference-time physics constraints.These failures can destabilize the receiving humanoid.Source: §1 · p. 2
  • DIGHT couples an interaction generator with a humanoid tracker, using rollout feedback to align generation and then adapting the tracker to the aligned generator's motions.Its generator preferences address executability and contact fidelity, including contact occurrence, location, duration, and force magnitude.Source: §1 · pp. 2–3
  • Across four dataset-backbone settings, DIGHT reduces all five physical errors; contact-topology precision rises from 38.29% to 46.77% and recall from 52.16% to 63.94% over InterGen.Motion quality and text alignment are largely preserved.Source: §1 · p. 3

2 RELATED WORK

Prior work spans text-conditioned interaction generation, physics-based humanoid control, and generator alignment using physical feedback. DIGHT focuses on realistic execution of generated two-humanoid interactions by coupling generation and tracking through simulator feedback and force-informed preferences.

Text-conditioned Human-human Interaction Generation

  • Text-conditioned interaction generators model coordinated motion through approaches including mutual attention, participant-specific language conditions, masked prediction, multi-person representations, and temporal or role-evolving models.Other work explores fine-grained control, autoregressive generation, arbitrary participant counts, and physics priors applied in kinematic space.Source: §2 · p. 3

Physics-based Humanoid Interaction

  • Physics-based humanoid control progressed from motion imitation to reusable motion priors and robust tracking, then extended to contact-rich human-object and multi-character interactions.These interaction-aware objectives address spatial coordination, contact relationships, and coupled dynamics.Source: §2 · p. 3

Conditioned Humanoid Generation

  • Existing conditioned humanoid generation either tracks generated digital interaction motion or trains on trajectories obtained through tracking, but tracking alone can miss contacts and produce implausible forces.DIGHT couples the generator and tracker through execution feedback and incorporates interaction forces into generator preferences.Source: §2 · pp. 3–4

Physical Feedback for Motion Model Alignment

  • Prior physical-feedback alignment uses refinement or control at inference, or frozen-tracker feedback to optimize motion generators; DIGHT adds simulator force feedback for unrealistic two-humanoid contacts.The cited approaches include PhysDiff, CLoSD, RLPF, and PhysMoDPO.Source: §2 · p. 4

3 PRELIMINARIES

The method uses reinforcement learning to train a humanoid tracker and a diffusion model to generate text-conditioned two-person motion. It also retargets motion data to a unified simulated morphology and refines contacts after retargeting.

Interaction Tracking Policy

  • The tracking policy imitates motion-capture data with a full SMPL humanoid controlled by PD controllers and adds an interaction graph reward to preserve inter-agent spatial relationships.The policy outputs target joint rotations from the current agent state and future reference state.Source: §3 · p. 4

Data Retarget

  • Motion data is retargeted to a unified humanoid morphology, then geometry-based contact refinement recovers original contact relationships while preserving overall motion structure.Retargeting can shift palm contacts to the wrist or forearm.Source: §3 · p. 4

Interaction Diffusion Model

  • The diffusion generator jointly denoises two participants' motions conditioned on text, with InterGen and TIMotion serving as backbones optimized to predict clean motion.A predefined noise schedule determines the diffusion process; iterative reverse diffusion defines the conditional motion distribution.Source: §3 · p. 4

4 METHOD

DIGHT first uses simulated rollouts to construct criterion-specific preferences and post-train the interaction generator, then adapts the tracker to the aligned generator's motion distribution. Its preferences distinguish general physical execution from contact fidelity.

4.1 OVERVIEW

  • DIGHT samples candidate motions, executes them with a frozen tracker, and uses the resulting rollouts to build preferences before fine-tuning the tracker on motions from the aligned generator.The two stages adapt the generator and tracker using information induced by each other.Source: §4.1 · p. 4, Figure 2 · p. 5

4.2 PHYSICS-GROUNDED PREFERENCE CONSTRUCTION

  • Tracking, friction, and floating preferences compare their corresponding rollout errors, and a general-physical winner must also have better motion-text alignment.The preference sets are constructed from multiple candidates executed with the same frozen tracker.Source: §4.2 · pp. 4–5
  • Contact preferences use persistent semantic body-region contacts for occurrence and compare topology, episode duration, and force fidelity for contact quality.Occurrence preferences are judged against the reference contact pattern; contact-quality winners must improve all three components and satisfy motion-text alignment constraints.Source: §4.2 · p. 5

4.3 PHYSICS-DECOUPLED DIFFUSION DPO

  • Physics-decoupled diffusion DPO optimizes a separate objective for each preference set rather than collapsing conflicting physical criteria into one scalar reward.Tracking, friction, and floating are grouped separately from occurrence and contact quality; λ_contact balances contact fidelity against general executability.Source: §4.3 · p. 6

4.4 TRACKER ADAPTATION

  • After generator alignment, its sampled motions become references for fine-tuning the pretrained tracker with its original tracking objective.At inference, the aligned generator produces the reference interaction and the adapted tracker executes it.Source: §4.4 · p. 6

5 EXPERIMENTS

Experiments evaluate DIGHT on text-conditioned interaction datasets and compare both generated motions and their simulated execution. Results cover generator quality, humanoid execution, preference ablations, and temporal and multi-humanoid extensions.

5.1 EXPERIMENT SETUP

  • Experiments use InterHuman and Inter-X with InterGen and TIMotion, assessing generated-motion quality and text alignment alongside simulated humanoid execution.The evaluation includes eight metrics covering text-motion consistency, motion quality, physical plausibility, and force error.Source: §5.1 · pp. 6–7

5.2 GENERATOR EVALUATION

  • On InterHuman, DIGHT1 and DIGHT2 reduce the five physical metrics by averages of 22.1% and 25.3%, respectively, while largely preserving motion quality and text alignment.The two generator backbones are InterGen and TIMotion.Source: §5.2 · p. 7

5.3 HUMANOID EXECUTION EVALUATION

  • DIGHT outperforms InterAgent across all evaluation metrics; qualitative comparisons with InterGen show better preservation of intended contacts and more plausible interactions.The InterGen examples include handshakes and helping a falling person stand up.Source: Table 2 · p. 7, §5.2 · p. 7, §5.3 · p. 8
  • On contact prompts, topology precision rises from 38.29% to 46.77%, recall from 52.16% to 63.94%, and contact-motion success from 84.70% to 89.23%.The authors report that these gains improve contact accuracy and reliability rather than simply increasing unnecessary contacts.Source: §5.3 · p. 8, Table 5 · p. 8
  • Generator alignment raises execution success from 88.73% to 91.45%, reduces fall rate from 9.32% to 6.28%, and lowers MBD tracking error from 10.36 cm to 8.22 cm.Tracker adaptation to the updated generator further improves all three metrics; the main experiments use one co-adaptation round, with iteration optional.Source: §5.3 · pp. 8–9, Table 4 · p. 8

5.4 DISCUSSION

  • Tracking preferences improve all five physical metrics; friction and floating preferences further reduce skating and floating artifacts, and contact preferences yield the best overall performance.The authors characterize these preferences as complementary supervision for general executability and interaction fidelity.Source: Table 3 · p. 8, §5.4 · p. 9
  • Physics-decoupled DPO outperforms standard DPO and SFT across all evaluation metrics; standard DPO yields modest physical-plausibility gains, while SFT degrades semantic alignment, motion quality, and interpenetration.Source: §5.4 · p. 9
  • DIGHT extends to temporally composed long-horizon interactions and a three-humanoid boxing scenario, with coherent progression and distinct partner-directed motions, respectively.Source: §5.4 · p. 9, Figure 4 · p. 9, Figure 5 · p. 9

6 CONCLUSION

DIGHT co-adapts an interaction generator and humanoid tracker using simulator-grounded preferences for executability and interaction fidelity. The appendix lists supplementary results and supporting materials.

6 CONCLUSION

  • DIGHT uses simulator feedback to construct preferences for executability and interaction fidelity, post-trains the generator with physics-decoupled diffusion DPO, then adapts the tracker to the aligned generator.The preferences model contact occurrence, duration, and interaction forces.Source: §6 · p. 9

APPENDIX

  • The appendix contents list additional visualization results and Inter-X results.It also lists supplementary material on tracking, text-motion evaluators, method details, metrics, datasets, contact annotations, and motion retargeting.Source: APPENDIX · p. 14

A MORE VISUALIZATION RESULTS

Additional qualitative results show DIGHT producing physically plausible interactions while preserving intended contacts and coordination across varied scenarios.

A MORE VISUALIZATION RESULTS

  • Additional comparisons show more physically plausible motions that better preserve intended interactions than the baseline.The comparisons are shown in Figs. 6, 7, and 8.Source: A MORE VISUALIZATION RESULTS · p. 14
  • A long-horizon sequence of dynamically diverse actions has coherent transitions and remains physically stable.The sequence is shown in Fig. 10.Source: A MORE VISUALIZATION RESULTS · p. 14
  • Three- and four-humanoid scenarios execute with coordinated behavior among participants.The multi-humanoid results are shown in Fig. 9.Source: A MORE VISUALIZATION RESULTS · p. 14
  • Boxing with sustained contact and dynamic kicking both remain physically stable while preserving intended interactions.The boxing involves prolonged close-range contact; the kicking involves rapid limb movements, balance shifts, and dynamic contact.Source: A MORE VISUALIZATION RESULTS · p. 14

B RESULTS ON INTER-X

On Inter-X, DIGHT improves physical metrics across two generator backbones while largely preserving motion quality and text-motion alignment.

B RESULTS ON INTER-X

  • 16.6% average relative reduction in five physical errors with InterGen, and 18.3% with TIMotion, on Inter-X.Across both backbones, DIGHT improves all five physical metrics while largely preserving motion quality and text-motion alignment; Table 6 reports the quantitative comparisons.Source: Table 6 · p. 15, B RESULTS ON INTER-X · p. 15

C ITERATIVE OPTIMIZATION STRATEGY.

Repeating the two-stage optimization three times improves success, fall rate, and MBD relative to one iteration, while four iterations slightly degrade performance.

C ITERATIVE OPTIMIZATION STRATEGY.

D ADDITIONAL METHOD DETAILS

The appendix details how DIGHT builds criterion-specific preferences from tracking and contact behavior, represents persistent contact between body regions, and parameterizes diffusion preference pairs.

D.1 GENERAL PHYSICAL PREFERENCES

  • General physical preferences use tracking deviation, tangential contact dissipation, and floating; a pair is valid only when its candidate also has better text-motion distance.The tracking error is defined as mean body deviation, and pair validity requires both a lower target-criterion error and lower text-motion distance.Source: §D.1 · pp. 15–16

D.2 CONTACT REPRESENTATION AND COMPARISON

  • Contact topology compares persistent region-pair sets, penalizes extra candidate contact edges separately, and excludes falls from preference sets other than tracking.The topology discrepancy is defined between region-pair sets; its bounded log error is given in the appendix.Source: §D.2 · pp. 16, 18
  • Contact-duration and force statistics are computed per region-pair edge, with short inactive gaps merged and force summarized over the longest contact episode.Gaps shorter than four frames are filled; the longest resulting episode supplies the mean and maximum force values.Source: §D.2 · pp. 16–17

D.3 DIFFUSION PARAMETERIZATION

  • For diffusion DPO pairs, preferred and rejected motions share the sampled timestep but are perturbed independently.The appendix describes this as part of the diffusion parameterization for the preference pair.Source: §D.3 · p. 18

E.1 MOTION QUALITY.

The appendix specifies measures of generated-motion quality and physical plausibility, including text alignment, ground clearance, foot sliding, overlap, smoothness, contact force, and tracking success.

E.1 MOTION QUALITY.

  • Motion quality is evaluated with FID for generated-motion quality and R-Precision and MM-Dist for text-motion alignment.Lower FID and MM-Dist indicate closer motion-distribution similarity and better text alignment; R-Precision checks whether the paired caption ranks within the top-k.Source: §E.1 · pp. 18–19

E.2 PHYSICAL QUALITY.

  • Float measures the lowest non-palm joint’s clearance above ground, with below-ground positions clamped to zero.The metric uses the lowest of 22 body joints per frame; lower values indicate better ground alignment.Source: §E.2 · p. 19
  • Skate measures horizontal foot displacement while feet are within 5 cm of the ground, where lower values indicate less foot sliding.The metric reports displacement in millimeters and does not divide by the frame interval.Source: §E.2 · pp. 19–20
  • Interpenetration approximates each body joint as a 0.06 m-radius sphere and measures cross-person geometric overlap.Lower values indicate less body overlap and better kinematic plausibility.Source: §E.2 · p. 20
  • Jerk measures temporal motion smoothness through changes in body acceleration, with lower values indicating smoother motion.It is averaged over valid temporal positions, both people, and 24 rigid bodies, and reported in m/s3.Source: §E.2 · pp. 20–21
  • Force Error compares generated and ground-truth mean inter-person contact forces, with lower values indicating closer contact intensity.PhysX contact magnitudes define active contacts using a threshold of 1.0; errors are averaged within prompts and then equally across prompts.Source: §E.2 · p. 21
  • Success Rate is the fraction of generated two-person motion sequences with each person’s sequence-averaged tracking error at most 0.25 m and neither pelvis below 0.3 m at any frame.Source: §E.2 · pp. 21–22

F MORE DETAILS OF DATASETS

The appendix describes the datasets and supporting procedures for contact-sensitive evaluation, contact-preserving retargeting, interaction-aware tracking, and text-motion assessment.

F MORE DETAILS OF DATASETS

  • InterHuman contains 6,022 sequences paired with 16,756 descriptions, while Inter-X contains 11,388 sequences and 34,164 descriptions across 40 categories.Inter-X also includes more than 8.1 million motion frames and more detailed body and hand motions; both datasets are retargeted to a neutral SMPL body for consistent embodiment.Source: F MORE DETAILS OF DATASETS · p. 22

G ANNOTATED CONTACT ON INTERHUMAN TESTSET

  • The strict InterHuman contact subset contains 503 captions specifying 631 required contact slots, with annotations based on caption semantics rather than ground-truth frame labels.Two-pass consensus accepted 445 annotations and blind third-pass adjudication resolved 58 more; annotations were manually reviewed.Source: G ANNOTATED CONTACT ON INTERHUMAN… · pp. 22–23

H MOTION RETARGET

  • Contact-aware retargeting detects source end-effector contacts and locally refines target limbs to restore them without shifting contacts to nearby joints.The refinement preserves unrelated GMR trajectories and restricts optimization to padded windows around contact episodes.Source: H MOTION RETARGET · p. 24

I TRACKING POLICY

  • The tracker conditions on the partner’s state and uses an interaction-graph reward to preserve inter-agent spatial and dynamic relationships.The complete partner observation lets the tracker respond to the partner’s realized state rather than tracking the two reference motions independently.Source: I TRACKING POLICY · pp. 25–26

J TEXT-MOTION EVALUATORS

  • Separate InterCLIP and InterCLIP-Phys evaluators assess digital motions and tracker-executed trajectories, respectively.InterCLIP-Phys is trained on executions from a frozen tracker distinct from DIGHT’s pretrained tracker, avoiding evaluator construction that favors DIGHT.Source: J TEXT-MOTION EVALUATORS · p. 26
Loading 2610.10322v1…