PaperlayerView exact revision on arXiv

Source-linked AI summary

CoCarry: Contact-Support-Structured Retargeting for Dual-Humanoid Cooperative Carrying

Ziyu Lin, Shihang Du, Tianhang Wang, Xiaodong Liu, Runze Zhang, Zongyou Yu, Haodong Zhang, Alois Knoll, Guang Chen

arXiv:2610.32460v1cs.RO

TL;DR

Cooperative carrying requires consistent shared grasp, relative configuration, and whole-body support across two humanoids, beyond plausible individual motions. CoCarry and Contact–Support-Structured Retargeting provide synchronized data and jointly refined references; the method preserves contact and reduces penetration on CoCarry, with additional transfer and physical-execution validation.

  • Problem

    Cooperative carrying requires two participants to maintain compatible shared-grasp, relative-configuration, and whole-body-support relations, which independent retargeting does not explicitly enforce.

    Source: §I · p. 2, B. Humanoid Motion Retargeting · p. 2

  • Method

    The paper introduces CoCarry and jointly refines nominal motions for both humanoids around a prescribed object trajectory using cooperation-aware hand anchors and complementary interaction and support structures.

    Source: §I · p. 2

  • Results

    On all 220 CoCarry sequences, the method achieves 0.9995 contact preservation, the highest among the compared methods, while also attaining the lowest reported penetration duration and maximum depth and 100% retargeting success.

    Source: B. CoCarry Retargeting Comparison · p. 6

  • Takeaways & Limitations

    The references show evidence of cross-dataset and cross-embodiment transfer and are validated in simulation and real-world cooperative carrying with physical A3 humanoids.

    Source: §VI · p. 8

Abstract

from arXiv · show

Recent advances in large-scale human motion data, motion retargeting, and motion imitation have enabled increasingly capable whole-body control for individual humanoids. Cooperative carrying is more challenging because successful coordination requires not only plausible individual motions, but also consistent cross-agent and agent-object relations, including shared grasp, relative configuration, and whole-body support. We introduce CoCarry, a cooperative-carrying dataset containing 220 synchronized two-person sequences across 21 task schemes, five shared-object geometries, four terrain types, and approximately 61 minutes of motion. We further propose Contact-Support-Structured Retargeting, which starts from nominal individual motions and jointly refines both humanoids around the same prescribed object trajectory. Cooperation-aware object-local hand anchors preserve compatible four-hand grasp relations, while complementary upper object-interaction and lower support structures retain object-relative geometry and coordinated support. Across all 220 CoCarry sequences retargeted to two A3 humanoids, our method achieves 0.9995 contact preservation, reduces penetration duration by 67.0% and 71.0% relative to GMR and OmniRetarget, respectively, and maintains a 100% retargeting success rate. On CORE4D with Unitree G1 humanoids, it achieves 0.9906 contact preservation with zero measured sliding. We further validate the generated references through physics simulation and multiple real-world cooperative carrying experiments on two physical A3 humanoids, demonstrating their usability for downstream dual-humanoid control.

I. INTRODUCTION

Cooperative carrying requires consistent shared grasp, relative configuration, and whole-body support across two humanoids, not just plausible individual motions. The paper introduces CoCarry and a joint retargeting method, then evaluates references across humanoids, datasets, simulation, and physical execution.

I. INTRODUCTION

  • CoCarry provides 220 synchronized two-person carrying sequences with shared-object trajectories and time-varying hand–object interaction priors.The sequences cover diverse task schemes, object geometries, and terrain conditions.Source: §I · p. 2
  • Independent retargeting can leave individually plausible motions with incompatible hand placements, distorted relative configurations, penetration, or inconsistent support.The shared task requires the two participants’ motions and their relations to the object to remain mutually consistent.Source: §I · p. 2
  • Contact–Support-Structured Retargeting jointly refines both humanoids around one prescribed object trajectory using hand anchors and complementary interaction and support structures.The anchors preserve shared grasp relations; the upper and lower structures preserve object-relative interaction and ground-referenced support.Source: §I · p. 2
  • The evaluations span all 220 CoCarry sequences, a matched CORE4D subset, physics simulation, and real-world cooperative carrying with physical A3 humanoids.They assess retargeting quality, cross-dataset and cross-embodiment generalization, and downstream physical usability.Source: §I · p. 2

A. Human Interaction and Cooperative Motion Datasets

  • CoCarry complements existing interaction datasets by focusing on continuous coordination between two participants around the same moving object.It preserves synchronized motion, shared-object trajectories, carrying conditions, and time-varying hand–object priors for humanoid retargeting.Source: A. Human Interaction and Cooperative… · p. 2

B. Humanoid Motion Retargeting

The paper positions cooperative carrying as a relation-preservation problem beyond individual humanoid retargeting, and builds its references from synchronized human carrying motion. CoCarry covers varied tasks and object geometries, while imitation is used to assess downstream execution.

B. Humanoid Motion Retargeting

  • Even individually plausible humanoid motions can yield mismatched hand placement, distorted relative configuration, penetration, or inconsistent support when the pair is not jointly constrained.The method uses individual retargeting as initialization and jointly refines both humanoids to preserve shared grasp, relative configuration, and support.Source: B. Humanoid Motion Retargeting · p. 2

C. Humanoid Imitation and Cooperative Control

  • The work constructs dual-humanoid cooperative references upstream of control, using motion imitation to validate whether they support downstream physical execution.The paper situates this focus alongside physics-based humanoid imitation and cooperative transportation research.Source: C. Humanoid Imitation and Cooperative… · pp. 2–3

A. Dataset Scope and Cooperative Task Coverage

  • CoCarry contains 220 synchronized carrying sequences, approximately 61.1 min of motion, and 440K raw MoCap frames recorded at 120 Hz.Its 21 schemes span five task families, with five object geometries that vary carrier separation, arm reach, grasp configuration, and support demands.Source: A. Dataset Scope and Cooperative Task… · p. 3
  • The 21 task schemes comprise 6 pickup/placement, 5 route/orientation, 4 terrain, 4 interaction-transition, and 2 height-transition schemes, with 60, 60, 40, 40, and 20 sequences, respectively.The families include routes and turns, ramps and stairs, joining or leaving, handover, temporary placement, and object height changes.Source: A. Dataset Scope and Cooperative Task… · p. 3, Figure 2 · p. 3

B. Synchronized Human–Object Representation

CoCarry represents two participants and their shared object as one synchronized interaction sequence. Its interaction priors encode hand–object contact locations, palm orientation, and changing hand activity for retargeting.

B. Synchronized Human–Object Representation

  • Each CoCarry sequence keeps both human streams and the shared-object trajectory on a common timeline with scene and task annotations.For retargeting, the streams and object trajectory are resampled to a common 50 Hz timeline while maintaining framewise correspondence.Source: B. Synchronized Human–Object… · p. 3

C. Cooperative Interaction Priors

  • For the four hands, CoCarry provides object-local anchors, surface normals, and time-varying activations describing contact location, palm orientation, and active time.The activations accommodate changing participation, including joining, leaving, handover, temporary placement, and release; these are kinematic priors, not force measurements.Source: C. Cooperative Interaction Priors · p. 3
  • The synchronized trajectories, task annotations, and interaction priors provide structured inputs to Contact–Support-Structured Retargeting.The dataset’s motion data, geometry, annotations, priors, tools, and available dual-humanoid references are planned for release.Source: C. Cooperative Interaction Priors · p. 3

IV. CONTACT–SUPPORT-STRUCTURED RETARGETING

The retargeting problem is to find a joint configuration for both humanoids that preserves cooperative relations around a moving object. GMR supplies synchronized nominal motions, while the prescribed object trajectory remains the common reference for object-related terms.

A. Problem Definition and GMR Initialization

  • The target is a stacked configuration for both humanoids that preserves cooperative relations around the shared object’s prescribed trajectory.The input includes synchronized human motions, a shared box mesh, and the object pose at each time.Source: A. Problem Definition and GMR… · p. 3
  • GMR is applied independently to produce nominal motions for the two humanoids before joint refinement.The synchronized nominal motions provide the initialization and nominal-motion prior, but do not preserve shared grasp, relative configuration, and coordinated support by themselves.Source: A. Problem Definition and GMR… · p. 3
  • Upper-body interaction quantities are evaluated in the object frame while both humanoid configurations are optimized with the prescribed object trajectory fixed.The pipeline begins with synchronized human motion and GMR nominal motions, then uses cooperation-aware anchors and upper interaction and lower support structures for joint refinement.Source: A. Problem Definition and GMR… · p. 3, Figure 3 · p. 4

B. Cooperation-Aware Hand Anchors

Cooperation-aware hand anchors encode compatible four-hand grasp geometry in the object frame and activate hand constraints only when relevant. Their selection balances demonstrated contact, temporal stability, and four-hand compatibility.

B. Cooperation-Aware Hand Anchors

  • Anchors are object-frame temporal medians of hand trajectories, projected onto assigned finite object surfaces with outward normals.This preserves demonstrated longitudinal and vertical contact locations within valid surface regions.Source: B. Cooperation-Aware Hand Anchors · p. 4
  • The prior combines demonstrated hand–surface proximity Ssurf with nominal GMR palm–anchor distance Sprior−pos as a weaker feasibility cue; stability favors locally consistent contact by penalizing tangential drift and measuring palm orientation.Source: B. Cooperation-Aware Hand Anchors · p. 4
  • Four anchors are evaluated jointly for each carrier’s two-hand layout and compatible spatial organization across both carriers.The lowest-scoring valid candidate is selected.Source: B. Cooperation-Aware Hand Anchors · p. 4
  • Framewise activation uses demonstrated-hand and nominal-GMR-palm distances to avoid constraining hands outside relevant contact intervals.Active intervals receive five-frame smoothstep weights, representing grasp establishment, carrying, transitions, and release without permanent four-hand contact.Source: B. Cooperation-Aware Hand Anchors · pp. 4–5

C. Contact–Support Structured Representation

The representation combines object-relative upper-body interaction structure with ground-referenced lower-body support structure. Both are coupled with active hand anchors in joint optimization.

C. Contact–Support Structured Representation

  • The upper object-interaction mesh links both carriers’ shoulder, elbow, and hand landmarks with shared object-surface samples in the object frame.Shared object nodes preserve arm configurations and carrier arrangement while complementing local palm constraints.Source: C. Contact–Support Structured… · p. 5
  • The lower support mesh links both carriers’ pelvis, leg, and foot landmarks to ground-reference nodes, preserving coordinated stance through varied carrying motions.The object-relative upper and ground-referenced lower structures are coupled in the same dual-humanoid optimization.Source: C. Contact–Support Structured… · p. 5
  • The relation-preservation error weights selected relations; upper-body and lower-body terms preserve cooperative object geometry and ground-relative support, alongside active hand-contact constraints.The formulation preserves these relations without explicitly modeling forces, friction, or contact dynamics.Source: C. Contact–Support Structured… · p. 5

D. Joint Dual-Humanoid Cooperative Optimization

Both humanoids are optimized together frame by frame around a fixed prescribed object pose. The objectives combine relational structures, contact and support constraints, and feasibility terms.

D. Joint Dual-Humanoid Cooperative Optimization

  • At each frame, both humanoids are jointly optimized around the fixed object pose using locally linearized geometric objectives and distance constraints.The stacked dual-humanoid update is solved frame by frame and warmstarted from the previous frame.Source: D. Joint Dual-Humanoid Cooperative… · p. 5
  • Joint limits, bounded updates, and distance constraints maintain kinematic feasibility while reducing penetration, unintended foot motion, and inter-robot proximity.Nominal and temporal terms preserve the GMR prior and continuity, while upper/lower structures and active hand/foot constraints preserve interaction and support.Source: D. Joint Dual-Humanoid Cooperative… · p. 5
  • The resulting trajectories serve as references for retargeting evaluation, physics-based control, and real-world cooperative execution.Cooperation is enforced on the stacked dual-humanoid state rather than through independent motions.Source: D. Joint Dual-Humanoid Cooperative… · p. 5

V. EXPERIMENTS

The experiments evaluate retargeting across CoCarry, CORE4D, simulation, and physical execution. The primary CoCarry-to-A3 comparison uses matched inputs and assesses interaction, geometric clearance, and lower-body support.

V. EXPERIMENTS

  • The evaluation spans CoCarry-to-A3 retargeting, CORE4D-to-G1 transfer, and downstream simulation and physical A3 execution.The experiments assess contact, geometric validity, support consistency, transfer, and downstream usability.Source: §V · p. 5

A. Experimental Setup and Metrics

  • All 220 CoCarry sequences are retargeted to two A3 humanoids at 50 Hz, comparing GMR, OmniRetarget, and the complete method with two component ablations.Methods share source motions, object trajectories, scene assets, and evaluation protocol.Source: A. Experimental Setup and Metrics · p. 5
  • Contact preservation measures retained source hand–object contact; penetration duration and depth measure overlap, while sliding duration and toe velocity measure support-motion errors.Contact preservation is interpreted jointly with penetration because penetration can also satisfy a proximity threshold.Source: A. Experimental Setup and Metrics · pp. 5–6
  • Retargeting success is the share of sequences yielding a complete valid dual-humanoid trajectory; duration metrics are averaged across synchronized two-person sequences.Fig. 4 reports mean contact loss by task family, sliding duration by terrain/support condition, and penetration duration by object geometry; lower is better.Source: A. Experimental Setup and Metrics · p. 6, Figure 4 · p. 6

B. CoCarry Retargeting Comparison

On the full CoCarry set, the method improves contact preservation and penetration metrics over GMR and OmniRetarget; its gains extend across tasks, terrains, and object geometries. Component analyses show that hand anchors and the structured representation address distinct aspects of cooperative retargeting.

B. CoCarry Retargeting Comparison

  • 0.9995 contact preservation, 0.1925 penetration duration, and 0.0451 m penetration maximum depth are the method’s best CoCarry results, with 100% retargeting success.Table I compares the method with GMR and OmniRetarget on all 220 sequences.Source: B. CoCarry Retargeting Comparison · p. 6
  • Penetration duration falls from 0.5840 for GMR and 0.6640 for OmniRetarget to 0.1925, reductions of 67.0% and 71.0%, respectively.GMR has lower sliding duration, but also lower contact preservation, deeper penetration, and larger maximum toe-sliding velocity.Source: B. CoCarry Retargeting Comparison · p. 6
  • Across five task families, contact loss stays below 0.001; across five object geometries, penetration duration remains below both generic baselines.Across flat ground, support surfaces, ramps, and stairs, the method has the lowest sliding duration among the three cooperative variants; removing anchors returns penetration to 0.6580–0.6706.Source: B. CoCarry Retargeting Comparison · p. 6

C. Component Analysis

  • Removing hand anchors leaves contact preservation nearly unchanged (0.9994 vs. 0.9995) but raises penetration duration from 0.1925 to 0.6638.The anchors add geometric guidance toward compatible four-hand grasp configurations, which a proximity-based contact score alone does not capture.Source: C. Component Analysis · p. 6
  • Replacing the upper object-interaction and lower support structures with a unified representation raises sliding duration from 0.1913 to 0.3600; the proposed decomposition reduces it by 46.9%.The complete method produces valid trajectories for all 220 CoCarry sequences.Source: C. Component Analysis · p. 6

D. Cross-Dataset and Cross-Embodiment Evaluation

On a matched 200-sequence CORE4D subset retargeted to Unitree G1 humanoids, the method preserves contact and eliminates measured sliding. It also shows a contact–clearance trade-off, while qualitative examples and downstream executions illustrate retargeted motions across settings.

D. Cross-Dataset and Cross-Embodiment Evaluation

  • Figure 5 compares OmniRetarget, Ours, and GMR across matched examples of penetration, missing contact, body distortion, and hand orientation in both settings.Its markers distinguish penetration, missing contact, misoriented contact, and correctly preserved contact.Source: Figure 5 · p. 7, D. Cross-Dataset and Cross-Embodiment… · p. 7
  • CORE4D motions, object trajectories, and scene assets are converted to the CoCarry retargeting representation, with robot-specific settings adapted to Unitree G1.The evaluation uses a matched shared-success subset covering bucket, box, desk, chair, and stick objects.Source: D. Cross-Dataset and Cross-Embodiment… · p. 7
  • 0.9906 contact preservation exceeds GMR’s 0.6908 and OmniRetarget’s 0.7337, while sliding duration and maximum toe-sliding velocity are both zero on the matched subset.The matched subset includes 200 sequences on which all three methods produce valid outputs.Source: D. Cross-Dataset and Cross-Embodiment… · p. 7
  • OmniRetarget has lower penetration duration and maximum depth, exposing a contact–clearance trade-off rather than uniform improvement across metrics.The method more strongly preserves cooperative grasp while introducing a small amount of shallow penetration; both penetration measures remain substantially below GMR’s.Source: D. Cross-Dataset and Cross-Embodiment… · p. 7
  • Figure 6 shows approach, lift, carry, and place in free-object physics simulation, alongside representative real-world cooperative carrying keyframes on two A3 humanoids.These experiments provide downstream physical validation, not a dataset-wide control benchmark.Source: Figure 6 · p. 7

E. Simulation and Real-World Execution

Simulation and physical A3 experiments validate the references for downstream dual-humanoid control, while the conclusion identifies broader tasks, platforms, and real-world evaluation as future work.

E. Simulation and Real-World Execution

  • In simulation, learned policies reproduce approach, lifting, transport, placement, and release with a free shared object.The object motion emerges from coordinated robot behavior rather than being prescribed during retargeting.Source: E. Simulation and Real-World Execution · pp. 7–8
  • Physics simulation and multiple real-world cooperative carrying motions on two physical A3 humanoids validate the references for downstream dual-humanoid control, not dataset-wide control benchmarking.The simulated policies reproduce approach, lifting, transport, placement, and release.Source: E. Simulation and Real-World Execution · p. 8, §VI · p. 8

VI. CONCLUSION

  • The conclusion presents CoCarry and Contact–Support-Structured Retargeting as a dataset and method for converting synchronized cooperative human motion into dual-humanoid references.The method jointly refines both humanoids around the same prescribed object trajectory using hand anchors and upper interaction and lower support structures.Source: §VI · p. 8
  • Future work will extend the framework to broader cooperative manipulation tasks, additional humanoid platforms, and larger-scale real-world evaluation.Source: §VI · p. 8
Loading 2610.32460v1…