Source-linked AI summary
TeamHOI: Learning a Unified Policy for Cooperative Human-Object Interactions with Any Team Size
Stefan Lionar, Gim Hee Lee
TL;DR
Cooperative HOI requires scalable coordination despite fixed-team policy designs and scarce multi-human reference motions. TeamHOI combines a Transformer policy with teammate tokens and masked AMP using single-human references, then evaluates the result on cooperative carrying. The single policy achieves coherent, stable, and diverse coordination across varying team sizes and object configurations, including tested teams of two to eight agents.
Problem
Existing cooperative HOI methods have limited team-size scalability and behavior diversity because policies use fixed-size inputs or indirect communication and multi-human reference motions are scarce.
Method
TeamHOI uses a shared decentralized Transformer policy with teammate tokens and masked AMP that lets task rewards guide object-interacting body parts from single-human reference motions.
Results
TeamHOI achieves coherent, stable, and diverse coordination across varying team sizes and object configurations, including cooperative carrying with two to eight agents.
Takeaways & Limitations
A single policy can coordinate cooperative carrying across varied team sizes and object shapes while producing physically plausible cooperative behaviors.
Abstract
from arXiv · showhide
Physics-based humanoid control has achieved remarkable progress in enabling realistic and high-performing single-agent behaviors, yet extending these capabilities to cooperative human-object interaction (HOI) remains challenging. We present TeamHOI, a framework that enables a single decentralized policy to handle cooperative HOIs across any number of cooperating agents. Each agent operates using local observations while attending to other teammates through a Transformer-based policy network with teammate tokens, allowing scalable coordination across variable team sizes. To enforce motion realism while addressing the scarcity of cooperative HOI data, we further introduce a masked Adversarial Motion Prior (AMP) strategy that uses single-human reference motions while masking object-interacting body parts during training. The masked regions are then guided through task rewards to produce diverse and physically plausible cooperative behaviors. We evaluate TeamHOI on a challenging cooperative carrying task involving two to eight humanoid agents and varied object geometries. Finally, to promote stable carrying, we design a team-size- and shape-agnostic formation reward. TeamHOI achieves high success rates and demonstrates coherent cooperation across diverse configurations with a single policy.
1. Introduction
TeamHOI targets cooperative HOI limitations in team-size scalability and motion-data diversity by combining a variable-size decentralized policy with masked motion-prior supervision. It evaluates this framework on cooperative table carrying across varied team sizes and shapes.
- Motivation: Existing cooperative HOI frameworks are constrained by fixed team sizes, absent explicit teammate communication, and limited adaptation to changing team composition.Fixed-size MLP inputs restrict multi-agent policies, while relying only on shared object dynamics does not capture continuous teammate-aware coordination.
- Motivation: Cooperative HOI reference motions are scarce, so single-human demonstrations can limit the diversity of coordination patterns available to larger groups.Full-body single-actor references tie coordination to one demonstrator’s motion, reducing flexibility for larger cooperative teams.
- TeamHOI: TeamHOI uses one decentralized policy with local observations, teammate tokens, and a Transformer architecture to generalize across varying numbers of cooperating agents.The policy shares parameters across agents and is trained with different team-size configurations without retraining or fine-tuning.
- TeamHOI: Masked AMP masks object-interacting body parts during supervision, allowing task rewards to guide those regions while single-human motions provide motion-prior information.This can repurpose a sideways walking reference for sideways lifting and broaden feasible cooperative behaviors.
- Evaluation: The evaluation uses cooperative table carrying with square, rectangular, and round objects, and a formation reward agnostic to table shape and team size.Agents must approach, form stable carrying arrangements, transport the table, and reach a target location.
3. Methodology
TeamHOI combines a Transformer-based decentralized policy with masked AMP and formation rewards to coordinate cooperative human-object interaction across variable team sizes while preserving realistic, diverse motion.
- TeamHOI Framework: TeamHOI uses local observations and teammate tokens in a Transformer policy to coordinate agents across varying team sizes.Alternating self-attention and cross-attention layers process the observing agent and variable teammate tokens.
- TeamHOI Framework: The policy is trained across diverse multi-agent configurations to learn one unified controller for different collaboration scenarios.PPO advantages are normalized separately for each team size to stabilize mixed-configuration training.
- TeamHOI Framework: Masked AMP combines full-body and masked discriminators so object-interacting body parts can follow task rewards while other motion remains realistic.The masked discriminator excludes interacting parts such as hands and forearms; a sigmoid interaction indicator blends masked and full-body style rewards.
- Cooperative Carrying Task: The cooperative carrying task requires multiple humanoids to transport tables with round, rectangular, or square geometries through physically grounded coordination.The reward includes walking, contact, lifting, transport, and put-down components.
- Formation Reward: Formation rewards promote stable lifting by encouraging even angular spacing and support coverage along the object’s principal axes.The angular spread target is 2π/m for m agents, while principal-axes coverage is computed from a convex-hull support polygon.
4. Experiment
TeamHOI is evaluated on cooperative carrying with 2–8 agents, varied table geometries, and comparisons against team-size-specific CooHOI* baselines. The experiments examine unified-policy coordination, masked AMP, and principal-axes formation rewards.
- Evaluation setup: Experiments train a unified policy for 2–8 agents and evaluate square, rectangular, and round tables in cooperative carrying scenarios.Episodes use randomized initial agent positions and target locations, with 600 simulation timesteps per evaluation episode.
- Quantitative evaluation: Table 1 reports high success, collective cooperation, and smooth motion across team sizes for TeamHOI, averaged over 10,000 simulations.CooHOI* baselines depend strongly on their training team size, while TeamHOI uses one unified policy.
- Qualitative evaluation: TeamHOI produces synchronized, globally coherent lifting and transport across 4-agent and 8-agent configurations, unlike CooHOI* baselines.The baselines show competing behaviors, unstable movements, or conflicting forces that cause contact loss and ineffective coordination.
- Ablation study: Masked AMP improves lifting-stage success and hand-object interactions by letting task rewards guide object-interacting body parts instead of over-constraining them with single-human references.It also supports varied coordination patterns, including different walking or stepping directions while carrying.
- Ablation study: Principal-axes coverage reward encourages formations aligned with the object’s rotational-stability axes, producing coordinated directions, symmetric gaits, and balanced support.Without this reward, angular-spread-only training can produce unnatural stepping patterns across scenarios and team sizes.
5. Conclusion
The paper presents TeamHOI as a unified framework for cooperative human-object interaction across varying team sizes and object configurations. Its Transformer-based teammate modeling, masked AMP, and formation reward support coherent, stable, and diverse coordination behaviors.
- Conclusion: TeamHOI enables a single decentralized policy to generalize across varying team sizes and object configurations.The framework incorporates teammate cues through a Transformer-based architecture.
- Conclusion: Masked AMP broadens motion diversity from single-human reference data, while principal-axes coverage reward encourages stable and natural formations during transport.The conclusion frames these components as supporting diverse cooperative behaviors and stable carrying.
- Conclusion: Experiments demonstrate coherent, stable, and diverse coordination behaviors across a wide range of multi-agent settings.The stated scope is cooperative carrying with varying team sizes and object configurations.
6. Training with Various Team Sizes
TeamHOI normalizes policy advantages separately for each team size during training because heterogeneous team-size data can have different reward scales and variances. This adjustment yields higher task reward and supports unified training across team sizes.
- Team-size normalization: Advantages are normalized separately for each team size n because combining heterogeneous team-size data can distort update-signal magnitudes.The separate normalization accounts for distinct reward scales and variances across team sizes.
- Training outcome: Figure 7 shows that team-size advantage normalization results in higher task reward.The passage reports the direction of the effect without specifying a numerical value.
- Training implementation: A fixed actor count in IsaacGym environments is handled by placing unused agents on a dummy ceiling plane when smaller teams are required.This workaround enables training a unified policy across different team sizes.
7. Reward Functions
The cooperative carrying task uses staged rewards for approaching, contacting, lifting, transporting, and putting down the table. Additional formation, alignment, and normalization terms shape stable multi-agent behavior.
- Advantage normalization: Team-size advantage normalization produces higher task reward than global advantage normalization.Figure 7 compares the corresponding task reward curves.
- Walking toward object: Walking rewards attract each agent toward the table while shaping position, inward directional speed, and facing.Agents target a 0.3 m gap from the nearest perimeter point and use inward-normal and table-facing directions.
- Hand contact preparation: Hand-preparation rewards encourage proximity to contact points, lower-edge alignment, target hand separation, and matched hand heights.The combined preparation reward requires all four terms to be satisfied simultaneously.
- Contact, lifting, and transport: Contact gates lifting and transport rewards, with valid contacts contributing to lifting and collective transport moving the table toward its target.Transport is activated once all agents establish contact using both hands; putdown begins when the table is within 0.03 m of the target.
- Putdown: Putdown rewards activate at the target and combine hand release with zero root velocity.The target hand height during putdown is 0.65 m, and agents are encouraged to stop moving.
- Reward aggregation: The task reward combines walking, formation, hand preparation, contact, lifting, transport, alignment, and putdown terms.The aggregation weights walking and formation, hand preparation and lifting coverage, contact, transport, alignment, and putdown rewards.
8. Generalized Principal-Axes Coverage Reward
The generalized principal-axes coverage reward represents object geometry through weighted planar points, principal axes, and convex-hull boundary extents. This construction supports irregular geometries and non-uniform mass distributions.
- Scope: The coverage reward supports irregular and concave geometries, including L-shapes, and non-uniform mass distributions.Object points may carry positive mass weights, while uniform density uses unit weights.
- Center of mass: The object’s planar center of mass is computed from sampled 2D points with optional mass weights.The sampled points can represent the tabletop surface, and uniform density corresponds to w_k = 1.
- Principal axes: Principal axes u1 and u2 are obtained from the eigenvectors of the object’s real symmetric inertia matrix.The first axis is associated with the smallest eigenvalue, and the second is the remaining orthonormal eigenvector.
- Boundary extents: Boundary extents measure maximum distances from the center of mass to the convex hull along positive and negative principal-axis directions.Using the convex hull keeps the extents well-defined for irregular and concave geometries.
9. Additional Implementation Details
Training uses staged progression toward the eight-agent task, with parallel environments and minibatch handling that excludes deactivated agents. The experiment’s key hyperparameters are summarized in Table 2.
- Training strategy: Sequential training progressively increases team size and task completion, with early termination when any agent falls or the table topples.This strategy addresses the computational inefficiency of directly training the unified policy with up to eight agents.
- Deactivated agents: Observations from deactivated agents placed above the ceiling are excluded from PPO and AMP minibatches.The environment setup also excludes these agents from observations, rewards, and gradient updates.
- Hyperparameters: Remaining hyperparameters follow CooHOI unless otherwise noted, with key values listed in Table 2.Table 2 is identified as the summary of key training hyperparameters.
10. CooHOI* Baseline
The CooHOI* baseline uses the same Transformer backbone but replaces cross-attention and teammate tokens with self-attention. Its training proceeds from single-agent skills to separately trained multi-agent models.
- Architecture: CooHOI* replaces TeamHOI’s cross-attention with self-attention and omits teammate tokens, leaving cooperation to shared object dynamics.The policy and critic retain the same Transformer-based backbone.
- Approach-angle reward: Its approach-angle reward aligns each agent toward a designated contact point using cosine similarity while avoiding table collisions.The reward equals 1 at 0° alignment and 0 at 180° opposition.
- Reward formulation: The aggregated baseline reward replaces formation, angular-spread, and coverage rewards with the approach-angle reward.The remaining reward structure follows Equation 25.
- Training strategy: CooHOI training first develops single-agent locomotion and manipulation before introducing multi-agent cooperation from that checkpoint.The first stage includes approaching, contact, lifting or tilting, and pushing or dragging the table.
- Team sizes: Separate models are trained for two, four, and eight agents, with the eight-agent model continuing from the four-agent checkpoint.Reported training durations are roughly 2 days for two agents, 6 days for four agents, and 5 additional days for eight agents.
11. More Experimental Results
Additional experiments show that one unified policy generalizes across trained and unseen team sizes and table geometries, while remaining coherent under distribution shifts. Performance becomes more difficult for heavier or larger tables, especially with very small or very large teams.
- Unified policy across all team sizes: The unified policy is evaluated across team sizes from 2 to 8 agents under normal and heavy table weights without retraining.Intermediate team sizes beyond the main configurations test smooth generalization across team sizes.
- Unified policy across all team sizes: Under 5× table mass, success rates steadily improve as team size grows because larger groups provide greater mechanical advantage.The heavier load increases the need for coordinated force generation.
- Zero-shot generalization: The policy maintains coherent cooperation across unseen small and large table geometries, with stable transport and consistently high success rates for smaller tables.Smaller tables can produce slightly stronger lift initiation and modestly higher jerk.
- Zero-shot generalization: Larger tables make lifting and stabilization harder because increased mass and longer moment arms can cause falls and early termination.Two-agent teams are particularly challenged, with less mechanical leverage and slower transport.
- Zero-shot generalization: For unseen 12-agent and 16-agent teams, the policy retains synchronized motion, relatively high success rates, and low jerk, unlike an unstable baseline.Very large teams have lower cooperative-time ratios because agents struggle to position within tight support gaps around the tabletop perimeter.
12. Multiple Affordance Behaviors
TeamHOI supports multiple affordance behaviors by changing task rewards, allowing agents to choose side-holding or edge-lifting according to feasible interaction regions. The demonstrations use the same single-human reference motions as the main experiments.
- Multiple Affordance Behaviors: Adapting the task reward enables agents to perform side-holding or edge-lifting based on their proximity to feasible affordance regions.The policy can also produce walking toward diverse directions.
- Multiple Affordance Behaviors: The multiple-affordance demonstrations are trained with the same single-human reference motions used in the main experiments.This demonstrates behavior variation through task-reward adaptation rather than a new reference-motion set.