Understanding Whole-Body Robot Teleoperation Strategies Under Diverse Task Objectives and Constraints

I. INTRODUCTION

• Teleoperation supports work where human presence is challenging or autonomy is unreliable, with interfaces evolving from wearable motion tracking and vision-based control to VR systems.Source: §I · p. 1
• Coordinating perception, manipulation, and navigation remains unresolved, as does how strategies change with time limits, error tolerance, and concurrent-task demands.Prior shared-autonomy approaches automate some functions while reserving critical decisions for the operator.Source: §I · p. 1
• The study examines control strategies on TIAGo using a hybrid framework that combines free-form pose control with constrained touchpad control.The framework maps headset and handheld-controller poses to the robot’s head and arms, and touchpads to its torso and base.Source: §I · pp. 1–2

A. Whole-Body Teleoperation Interface

• Prior hybrid VR/MR systems include whole-body input strategies for mobile manipulators and humanoids,, while adaptation to real-world objectives and constraints remains unclear.Earlier VR-based work also examined manipulation and navigation performance, workload, latency, and interface design.Source: A. Whole-Body Teleoperation Interface · p. 2

B. Multi-Component Coordination in Complex Teleoperation

• Whole-body tasks coordinate perception, manipulation, locomotion, and torso positioning in real time, unlike single-arm or base-only teleoperation.The TIAGo interface maps head and hand movements to the robot’s head and arms, and controller touchpads to its torso and base.Source: B. Multi-Component Coordination in… · p. 2, Figure 2 · p. 3
• Bimanual methods blend user intent with robot policy priors or use a single master interface, while torso-arm coordination supports reach across varied surfaces [27].Without torso-arm coordination, users may interrupt manipulation to adjust posture, increasing inefficiency and mental workload.Source: B. Multi-Component Coordination in… · pp. 2–3
• Arm-base coupling can support simultaneous navigation and manipulation without added interface complexity, but depends on fluid transitions and alignment between components.Prior work reports reduced task time and physical effort under that interface condition.Source: B. Multi-Component Coordination in… · p. 3
• How users adapt subsystem coordination to time pressure, accuracy requirements, or workload remains underexplored, since much prior work studies isolated components or idealized tasks.The stated goal is to design support that adapts to users’ coordination behavior rather than requiring constant micromanagement.Source: B. Multi-Component Coordination in… · p. 3

A. Hybrid Control Framework for Whole-body Teleoperation

• The framework runs on a TIAGo OMNI++ mobile manipulator integrating perception, manipulation, locomotion, and an adjustable torso, and is designed for adaptation across robot platforms and input modalities.Source: A. Hybrid Control Framework for… · p. 3
• Free-form VR pose control operates the robot’s head and arms, while touchpads increment torso height and control base translation and rotation; triggers operate the grippers.Arm pose targets are solved with TRAC-IK.Source: A. Hybrid Control Framework for… · pp. 3–4
• A downward head-mapping offset and low-pass filtering reduce fatigue and jitter, while base speed is damped when arm extension or torso height exceeds configured thresholds.The head offset also helps users see the arms in the GUI; damping is intended to support stable operation when the center of mass is raised or arms approach their limits.Source: A. Hybrid Control Framework for… · pp. 3–4

B. Graphical User Interface (GUI)

• The GUI streams head-mounted RGB camera video and robot operational states to the VR headset, including arm status, torso height, and LiDAR-derived obstacle visualization.The streams are sent via UDP, and a mini-map updates obstacles in the robot’s workspace.Source: B. Graphical User Interface (GUI) · p. 4
• The obstacle mini-map turns red when an obstacle is less than 0.1 meters away, while the GUI displays arm-control status, torso height, and operational-limit alerts.Source: B. Graphical User Interface (GUI) · p. 4

• The study investigates control strategies for teleoperating a complex robot with multiple capabilities using a hybrid framework.Source: §IV · p. 4
• Home-care tasks provide the study setting because they represent diverse constraints and objectives.Source: §IV · p. 4
• The authors aim to understand strategies that may inform avoiding duplicated control inputs, suboptimal performance, and potentially inappropriate teleoperation assistance.Source: §IV · p. 4

A. Curriculum-Based Training

• Training progresses from basic manipulation skills to advanced coordinated control, with modules increasing in difficulty and complexity.The curriculum was designed to address variation in prior gaming, VR, and robotics experience and provide relatively uniform proficiency.Source: A. Curriculum-Based Training · p. 4
• Basic manipulation progresses from single-arm interaction to two-arm tasks involving solid and deformable targets.Unimanual modules practice translation, rotation, and full end-effector control; the bimanual basket handover requires symmetric use of both arms.Source: A. Curriculum-Based Training · p. 4
• The curriculum’s two categories are manipulation and coordination, with manipulation divided into unimanual and bimanual tasks.Source: Figure 3 · p. 5
• Advanced coordination trains users to synchronize the head, arms, torso, and base for complex teleoperation tasks.Modules cover head-arm visual tracking, torso-arm lifting across heights, base-arm search and transport through constrained paths, and combined head-torso-base-arm movement.Source: A. Curriculum-Based Training · p. 5

B. In-home Care Tasks

• Object Organization is an unconstrained baseline task in which participants gather objects from two tables and place them in a wall-mounted organizer.Objects are deposited into predefined pockets on the organizer.Source: B. In-home Care Tasks · p. 5
• Medication Preparation tests time-sensitive control by requiring delivery of two shelf-stored bottles to a marked region within five minutes.A countdown timer is displayed in the control GUI.Source: B. In-home Care Tasks · p. 5
• Hydration makes precision critical: participants pour simulated water from a bottle into a mug while avoiding spills.Source: B. In-home Care Tasks · p. 5
• Laundry combines collecting clothing and moving a basket with a secondary conversation task to simulate distraction and varying workload.The clothing is dropped into the basket before participants move it to a table.Source: B. In-home Care Tasks · pp. 5–6

C. Procedure

• Participants completed a demographic and spatial-ability assessment, learned TIAGo control with a VR headset and handheld controllers, and could practice curriculum modules for up to 30 minutes.Source: C. Procedure · p. 6
• After training, participants performed the four in-home tasks in random order and completed NASA-TLX and experience questionnaires after each task.Source: C. Procedure · p. 6

D. Measures and Data Analysis

• Measures captured component activation and coordinated-control duration, controller input speed, proximity to obstacles, joint-limit duration, task time, errors, and NASA-TLX mental demand.Errors included collisions or knocked targets, simulated-water spills, and unsuccessful grasping or releasing.Source: D. Measures and Data Analysis · p. 6
• The questionnaire also asked which task components or actions participants preferred to delegate to robot autonomy.Source: D. Measures and Data Analysis · p. 6
• Figure 4 presents coordinated-control percentages across four tasks and their correlation with task time in the time-sensitive task.Source: Figure 4 · p. 6
• The analysis used mixed regression for correlations and repeated-measures ANOVA for task comparisons, with Holm-Bonferroni correction for pairwise tests.Source: D. Measures and Data Analysis · p. 6

E. Participants

• The study recruited 18 participants aged 19–39, most reporting limited prior robot experience.The sample included 8 males and 10 females from the Johns Hopkins University Homewood campus.Source: E. Participants · p. 6

V. RESULTS AND DISCUSSION

• The results section compares no-constraint, time-sensitive, error-critical, and workload-adaptive tasks; its figures cover input speed, control discontinuation, and mental demand.The figure captions also identify a correlation between constrained torso/base control and error rate in the error-critical task.Source: §V · p. 6, Figure 5 · p. 6, Figure 6 · p. 7

A. Task Efficiency

• More coordinated control was associated with shorter task completion time, fewer arm-length-limit encounters, and less time at the head-pan limit with more head-base coordination.Participants used significantly more coordinated control in the time-sensitive task than in the other three tasks.Source: A. Task Efficiency · p. 6
• The authors recommend coordinating head and manipulator control with torso and base operations through users’ egocentric inputs.Suggested examples include rotating the base when head pan reaches its limit and adjusting torso or base when the arms reach theirs.Source: A. Task Efficiency · pp. 6–7

B. Task Precision

• Using more constrained torso/base control was associated with significantly fewer errors, while input speeds were significantly slower in the error-critical task than in the other three tasks.Near targets, defined as less than 0.35 meters from the end-effector, participants used either more free-form arm control or more constrained torso/base control.Source: B. Task Precision · p. 7
• Separating control degrees of freedom near targets can increase accuracy, and participants slowed hand movements for fine manipulation.The authors propose modeling human control behavior to guide when and what assistance to provide.Source: B. Task Precision · p. 7
• The hybrid control framework is shown applied to a Unitree G1 humanoid with a Meta Quest 3 headset and controllers.Source: Figure 7 · p. 7

C. Task Workload

• p < .001 for control discontinuation and p < .01 for mental demand: the workload-adaptive task exceeded the other three tasks on both.Control discontinuation was calculated from time when users activated no robot actions.Source: C. Task Workload · p. 7
• The authors argue that modeling workload can inform adaptive robot autonomy tailored to task contexts, human capabilities, workload, and preferences.They suggest emphasizing task-critical interface information, such as obstacle distance and grasping success rate.Source: C. Task Workload · p. 7

VI. CONCLUSION

• The user study examined how participants adapted control behaviors and strategies to different task constraints and objectives.Source: §VI · p. 7
• The authors identify intuitive control interfaces and adaptive assistance as important for improving complex teleoperation.Source: §VI · p. 7
• The framework lacked haptic feedback, and perception relied only on a camera embedded in the robot’s head.The authors identify 3D workspace reconstruction as a possible future direction.Source: §VI · p. 7