Source-linked AI summary
Learning from Humans for Proactive Assistance in Human-Robot Collaborative Transport
Elvin Yang, Christoforos Mavrogiannis
TL;DR
Collaborative transport requires a robot to help relocate an object while remaining responsive to human input, but prior approaches often separate those capabilities. PROACT combines learned predictions of collaborative object motion with compliant whole-body control; across 108 real-world trials, it reduced interaction work and completion time relative to compliance-only and MPC baselines.
Problem
Prior work often treats efficient task progress and responsiveness to human input separately, while obstacle-constrained transport requires both.
Source: §I · p. 1Method
PROACT learns transformer-based predictions of future object motion from dyadic transport demonstrations and integrates them with compliant whole-body control.
Source: §I · p. 1, A. Learning a Model of Object Twist… · p. 2Results
Across 108 real-world trials, mean interaction work was 59.2% lower than Compliance-only and 20.4% lower than MPC, while mean completion time was 12.9% and 6.9% shorter, respectively.
Source: C. Pilot Study Results · pp. 5–6Takeaways & Limitations
PROACT demonstrated efficient collaborative transport in a constrained, obstacle-occupied workspace in terms of time, interaction forces, and interaction work.
Source: §VI · p. 7The evaluation focused on a single controlled setting and assumed that the user and robot had grasped the object and established a shared objective.
Source: §VI · p. 7
Abstract
from arXiv · showhide
We focus on human-robot collaborative transport, a challenging task of broad relevance spanning logistics, manufacturing, and the home, in which a user and a robot work together to relocate a large or heavy object. To act as an effective partner, the robot should reduce the user's effort by contributing to efficient relocation of the object while remaining physically responsive to them. Prior work often addresses these capabilities separately, producing robots that may move the object efficiently but resist user input, or accommodate the user but depend on continuous guidance. Our key insight is that obstacle-constrained collaborative transport requires integrating predictions of human collaborative behavior with compliant robot control. To this end, we introduce PROACT, a framework for human-robot collaborative transport that incorporates anticipation into compliant whole-body control through a learned model of human collaborative behavior. Trained on a large-scale, real-world dataset of dyadic human transport demonstrations, our transformer architecture distills collaborative behavior into predictions of future object motion. Across 108 real-world trials with a 9-DoF mobile manipulator, PROACT reduces mean interaction work by 59.2\% and 20.4\%, and mean completion time by 12.9\% and 6.9\%, relative to compliance-only and MPC baselines, respectively. Footage from our experiments can be found at https://youtu.be/qAGvQfVPjbk.
I. INTRODUCTION
The introduction frames collaborative transport as requiring both task progress and responsiveness to human input, capabilities prior work often studies separately. PROACT combines learned collaborative-behavior prediction with compliant whole-body control and is evaluated on a mobile manipulator.
I. INTRODUCTION
- Prior work often separates compliant response to user input from human modeling, while collaborative transport also requires progress, obstacle avoidance, and object stability.Prior evaluations often restrict tasks to obstacle-free or straight-line motion, or constrain interaction speed or object motion.Source: §I · p. 1
- PROACT combines long-horizon prediction of collaborative human behavior with compliant whole-body control for obstacle-constrained transport.The framework uses object motion to represent collaborative interaction and deploys on a 9-DoF mobile manipulator.Source: §I · p. 1
- A transformer-based model predicts human collaborative behavior from real-world dyadic transport demonstrations.Source: §I · p. 1
- The work validates the model on an independent test set and evaluates the integrated system in real-world experiments using a 9-DoF mobile manipulator.Source: §I · p. 2
II. RELATED WORK
Prior physical HRC work commonly centers on compliant following or human-behavior modeling, with limitations in task scope and continued guidance. PROACT uses object-level collaborative dynamics as predictions for proactive whole-body control.
II. RELATED WORK
- Prior mobile physical-HRC studies often use short-horizon, point-to-point, straight-line, or obstacle-free tasks, while compliant followers still rely on continued human guidance.Some methods also assume instrumented objects, increasing design complexity and limiting generalization.Source: §II · p. 2
- Collaborative-behavior models estimate future object motion, but suitable datasets have been limited; the cited dyadic dataset captures diverse 6DoF object motions.The approach builds on transformer architecture for longer-horizon predictions.Source: §II · p. 2
- PROACT uses object-motion predictions as nominal commands for compliant whole-body control on a 9-DoF mobile manipulator.This connects collaborative-dynamics modeling to proactive robot behavior generation.Source: §II · p. 2
III. PROBLEM STATEMENT
The problem is to transport a rigid object through a workspace containing obstacles while a user and robot maintain fixed grasps. The robot should help the team reach a goal while avoiding collisions, maintaining smooth motion, and reducing user effort.
III. PROBLEM STATEMENT
- The task involves a user and robot transporting a rigid object through a workspace with obstacle-occupied regions, representing object state by pose and twist.Source: §III · p. 2
- The user and robot maintain fixed grasps on opposite sides of the object, and the robot comprises a holonomic mobile base and an N-DoF arm.Source: §III · p. 2
- The team aims to reach a goal while avoiding static obstacles and the user, maintaining smooth object motion, and requiring low user effort.The robot is assumed to have force-torque sensing, object-state history, workspace layout, and a global path planner.Source: §III · p. 2
IV. PROACT: A FRAMEWORK FOR PROACTIVE ASSISTANCE IN COLLABORATIVE TRANSPORT
PROACT combines learned prediction of collaborative object motion with compliant whole-body control. The predicted motion horizon is conditioned on recent object-state history, and its first twist supplies the controller’s nominal target at each update.
IV. PROACT: A FRAMEWORK FOR PROACTIVE ASSISTANCE IN COLLABORATIVE TRANSPORT
- The framework predicts a future horizon of object twists describing how a collaborative team would realize a route, conditioned on recent object-state history.Source: §IV · p. 2
- The collaborative behavior model is learned from dyadic human transport demonstrations and integrated with compliant whole-body robot control.Source: §IV · p. 2
- At each deployment update, the first predicted twist becomes the nominal object-motion target for the whole-body controller.Source: §IV · p. 2
A. Learning a Model of Object Twist Prediction
The model predicts a finite horizon of object twists from recent object-state history and a local reference path. It uses a transformer architecture trained on dyadic transport demonstrations.
A. Learning a Model of Object Twist Prediction
- The model predicts a 6-DoF object-twist sequence along a route, conditioned on recent object-state history and a local reference path.The reference path is geometric and carries no timing, velocity, or orientation information.Source: A. Learning a Model of Object Twist… · p. 2
- A transformer encoder processes object-state history, route points, and learned future-step query tokens to produce motion predictions.The encoder uses four layers, eight self-attention heads, and 512-dimensional feedforward layers.Source: A. Learning a Model of Object Twist… · pp. 2–3
- Training uses 6-DoF trajectories from 31 human dyads transporting a 27 kg stretcher-like object across obstacle and behavioral-instruction conditions.Trajectories were downsampled from 200 Hz to 20 Hz; geometric reference paths were constructed from average demonstrated object paths.Source: A. Learning a Model of Object Twist… · p. 3
- The model is trained for 50 epochs with Smooth L1 loss and AdamW, retaining the checkpoint with the lowest validation loss.Training and validation splits are separated by participant pair in a 75/25 ratio; the initial learning rate is 10^-3.Source: A. Learning a Model of Object Twist… · p. 3
B. Low-Level Robot Control
Low-level control converts the model’s predicted object motion into a nominal end-effector twist, adapts it to measured physical input, and tracks it with a whole-body controller. The controller also enforces robot velocity and obstacle-avoidance constraints.
B. Low-Level Robot Control
- With a fixed grasp, the first predicted object twist is transformed to a nominal end-effector twist for control.An admittance layer modifies that nominal twist before a whole-body QP maps it to arm and base velocities.Source: B. Low-Level Robot Control · p. 3
- The admittance layer adjusts the nominal end-effector twist using wrench measurements to support compliant physical interaction.The wrench input is filtered and deadbanded, and the admittance velocity decays toward zero without external wrench input.Source: B. Low-Level Robot Control · p. 3
- A whole-body quadratic program tracks the target end-effector twist by optimizing arm joint and base velocities.Its objectives regularize command magnitude, command smoothing, and redundant arm posture.Source: B. Low-Level Robot Control · p. 3
- Obstacle avoidance is enforced through linear constraints on planar base velocity, with allowed speed toward nearby segments depending on clearance.The controller also bounds individual arm and base velocity components and planar base speed.Source: B. Low-Level Robot Control · p. 3
V. EVALUATION
The evaluation combines model-prediction testing with a pilot study of real-world collaborative transport. The study uses a workspace designed to require successive turns between start and goal locations.
V. EVALUATION
- The evaluation has two parts: component-level model-prediction testing and real-world human-robot transport trials.The workspace was designed around practical transport challenges such as maneuvering around corners and through constrained spaces.Source: §V · p. 4
- The pilot workspace measures 4.5 m × 5.4 m and requires successive turns between start and goal locations.An OptiTrack system provides robot and object poses at 120 Hz.Source: §V · p. 4
- The trials transport a 3.75 kg storage container measuring 0.678 m × 0.388 m × 0.464 m.The passage identifies it as a commercially available heavy-duty storage container.Source: §V · p. 4
A. Model Prediction Performance
The learned model predicts future object motion on an independent lab test set and is compared with constant-twist and MPC baselines. It achieves lower prediction error across horizons, with error rising more gradually as the horizon lengthens.
A. Model Prediction Performance
- The model is trained on Freeman et al.’s object trajectories and tested on 22 independent lab trajectories collected in a substantially different workspace.Fig. 4 shows individual paths in gray and averaged paths in black.Source: A. Model Prediction Performance · p. 4, Figure 4 · p. 4
- Across prediction lengths, the learned model achieves lower test-set MAE than constant-twist and MPC baselines, with error growing more gradually as the horizon lengthens.The constant-twist baseline propagates the most recent object twist.Source: A. Model Prediction Performance · p. 4
- The learned model’s translational twist error is similar to constant twist, while its rotational twist error is reduced more; the test paths require direction changes and straight-to-curved transitions.Constant twist cannot anticipate changes in twist, while MPC’s objective favors rapid goal progress and path tracking without encoding interaction-dependent velocity variations.Source: A. Model Prediction Performance · p. 4
B. Pilot Study Experimental Setup
The pilot setup compares three controllers on obstacle-avoidance transport trials with users positioned ahead of or behind the robot. It measures task outcomes and interaction effort while using a mobile manipulator with compliant control.
B. Pilot Study Experimental Setup
- Fig. 5 reports test-set prediction error separately for translational and rotational twist components across prediction lengths.The passage identifies the panels and comparison dimensions but does not state their plotted values.Source: Figure 5 · p. 5
- PROACT uses learned predictions for anticipatory assistance, while compliance-only sets the nominal end-effector twist to zero and MPC uses a hand-designed motion planner.All three controllers share an admittance layer and whole-body QP but differ in how they generate nominal end-effector twist.Source: B. Pilot Study Experimental Setup · pp. 4–5
- Six users each complete 18 trials, testing three controllers in two configurations that place the robot ahead of or behind the user while transporting around obstacles.Users are familiarized with compliance, complete two practice trials, and receive randomized controller order without being told which controller is active.Source: B. Pilot Study Experimental Setup · p. 5
- The objective metrics are success rate, task-completion time, average object speed, and average object acceleration.Success requires reaching the goal without collision or grasp loss.Source: B. Pilot Study Experimental Setup · p. 5
- The platform is an omnidirectional mobile base with a 6-DoF UR10e arm and suction gripper; PROACT runs at 20 Hz with a 3 s history and 1 s prediction horizon.The admittance layer and whole-body QP run at 500 Hz; base speed is limited to 0.8 m/s translation and 1.5 rad/s rotation.Source: B. Pilot Study Experimental Setup · p. 5
- The force-torque sensor signal is low-pass filtered at 0.8 Hz with 5 N force and 1 N m torque deadbands before trials.The sensor is zeroed before each trial, and admittance parameters are tuned for smoothness and responsiveness.Source: B. Pilot Study Experimental Setup · p. 5
C. Pilot Study Results
Across 108 real-world trials, PROACT combined proactive assistance with compliant transport and achieved shorter completion times and lower interaction effort than the comparison controllers. Its advantages persisted across both starting configurations, while the study also observed higher acceleration and occasional oscillation before grasp failures.
Prediction performance
- The learned model more closely matched future dyad motion than MPC or constant twist in a test-set window.The figure shows object motion from above and predictions for all six twist components, separating observed history from future ground truth and predictions.Source: Figure 6 · p. 6
Aggregate performance
- Mean completion time was 12.9% shorter than Compliance-only and 6.9% shorter than MPC; mean interaction work was 59.2% and 20.4% lower, respectively.PROACT also had the highest mean transport speed and lowest mean interaction force and torque; success rates were comparable, with one suction-grasp failure each for MPC and PROACT.Source: C. Pilot Study Results · pp. 5–6, Table I · p. 6
Impact of starting configurations
- PROACT ranked lowest in completion time, force, torque, and interaction work in both starting configurations, while Compliance-only had the lowest acceleration.Across controllers, starting behind rather than ahead was associated with 7.0% lower completion time, 17.2% lower acceleration, 5.3% lower force, 19.4% lower torque, and 4.8% lower interaction work.Source: C. Pilot Study Results · p. 6, Figure 7 · p. 7
Implications of proactive behavior
- PROACT tended to maneuver proactively through curves, whereas Compliance-only could get stuck in corners; PROACT's average acceleration was 67.4% higher than Compliance-only.The authors suggest faster directional changes may contribute to higher acceleration; pliable suction cups and filtering can also produce oscillatory responses that may be reinforced through predictive feedback.Source: C. Pilot Study Results · pp. 6–7
- Oscillatory object motion was observed before the grasp breaks in the failed MPC and PROACT trials.The authors describe a possible chain involving spurious wrench measurements from pliable silicone cups, low-pass-filter phase lag, oscillatory admittance, and reinforcement through predictive feedback.Source: C. Pilot Study Results · p. 7
VI. DISCUSSION
PROACT combines collaborative-behavior prediction with compliant whole-body control and was deployed on a mobile manipulator for transport in a constrained, obstacle-occupied workspace. The pilot did not assess subjective perceptions, and the evaluation was limited to a controlled setting with an assumed grasp and shared objective.
VI. DISCUSSION
- PROACT combines collaborative-behavior prediction with compliant whole-body control to provide proactive assistance in physical HRC.The 9-DoF mobile-manipulator deployment demonstrated transport in a constrained, obstacle-occupied workspace, assessed by time, interaction forces, and interaction work.Source: §VI · p. 7
- The pilot assessed functional performance but not subjective user perceptions, and object-motion representations omit ergonomics, verbal communication, and other coordination cues.The authors identify learning from additional signals and richer perception as future work.Source: §VI · p. 7
- The evaluation used a single controlled setting and assumed the user and robot had grasped the object and established a shared objective.The authors also target generalization across object shapes, masses, and workspace layouts, and future integration with higher-level interaction and communication frameworks.Source: §VI · p. 7