PaperlayerView exact revision on arXiv

Source-linked AI summary

Improved Trust in Human-Robot Collaboration with ChatGPT

Yang Ye, Hengxu You, Jing Du

arXiv:2304.12529v1cs.ROcs.AIcs.HC

TL;DR

The paper examines whether ChatGPT-based robot control can address trust concerns in human-robot collaboration. It presents RoboGPT for spoken tool-fetching assembly tasks, where participants reported higher trust and lower cognitive load than with fixed commands.

  • Problem

    The paper addresses the limited understanding of how integrating LLMs into HRC affects operator trust, despite their potential for natural interaction.

    Source: §I · p. 1

  • Method

    RoboGPT adapts ChatGPT to interpret task context and spoken commands, clarify instructions, and control a robot during assembly.

    Source: §I · p. 1, A. System Design · p. 2

  • Results

    Participants reported higher trust and lower cognitive load with the ChatGPT-enabled assistant than with fixed control commands.

    Source: §VII · p. 6

  • Takeaways & Limitations

    The study demonstrates the feasibility of fine-tuning ChatGPT into a robot assistant that understands task context and generates ROS messages to control an arm.

    Source: §VI · p. 5

    Text-only input can burden communication, and decisions based on the assistant’s understanding can be problematic when communication is inaccurate or misunderstood.

    Source: §VI · p. 6

Abstract

from arXiv · show

Human robot collaboration is becoming increasingly important as robots become more involved in various aspects of human life in the era of Artificial Intelligence. However, the issue of human operators trust in robots remains a significant concern, primarily due to the lack of adequate semantic understanding and communication between humans and robots. The emergence of Large Language Models (LLMs), such as ChatGPT, provides an opportunity to develop an interactive, communicative, and robust human-robot collaboration approach. This paper explores the impact of ChatGPT on trust in a human-robot collaboration assembly task. This study designs a robot control system called RoboGPT using ChatGPT to control a 7-degree-of-freedom robot arm to help human operators fetch, and place tools, while human operators can communicate with and control the robot arm using natural language. A human-subject experiment showed that incorporating ChatGPT in robots significantly increased trust in human-robot collaboration, which can be attributed to the robot's ability to communicate more effectively with humans. Furthermore, ChatGPT ability to understand the nuances of human language and respond appropriately helps to build a more natural and intuitive human-robot interaction. The findings of this study have significant implications for the development of human-robot collaboration systems.

I. INTRODUCTION

The introduction frames operator trust, communication, and mutual understanding as central challenges in human-robot collaboration (HRC), and asks whether ChatGPT can support trust through language-based robot control. It introduces RoboGPT and a 15-participant assembly experiment.

I. INTRODUCTION

  • HRC trust is challenged by limited reliability, predictability, transparency, and communication, while existing natural-language control still struggles with context, reasoning, and interactivity.Trust affects willingness to cooperate and productivity.Source: §I · p. 1
  • The study investigates whether ChatGPT-based control can affect trust in an HRC assembly task, an impact the authors say warrants investigation.The introduction notes that LLM integration methods and their effects on operator trust have not been thoroughly researched.Source: §I · p. 1
  • RoboGPT adapts ChatGPT as a robot-control assistant that interprets task context and commands, clarifies instructions, and controls the robot arm.The proposed benefit is natural bilateral communication and context understanding.Source: §I · p. 1
  • Fifteen participants worked with a collaborative robot that fetched and placed tools in response to spoken instructions during an assembly task.The introduction presents the human-subject experiment as a demonstration and validation of the HRC design.Source: §I · pp. 1–2

II. RELATED WORK

Related work connects HRC trust to transparency and handling ambiguous input, then surveys robotic interfaces and the emerging use of LLMs for natural communication. It identifies trust and collaboration effectiveness as open questions for LLM-enabled HRC.

A. Trust in Human-robot Collaboration

  • Trust affects willingness to work with and rely on robots, and limited transparency in robot decisions may reduce trust and collaboration.Trust levels can vary with interaction method, including touchscreen, joystick, and verbal control.Source: A. Trust in Human-robot Collaboration · p. 2
  • Robots that handle ambiguous commands and environmental uncertainty are more likely to be trusted, motivating study of assistants robust to varied speech styles.The cited work links adaptability and competence in uncertain situations with growing operator confidence.Source: A. Trust in Human-robot Collaboration · p. 2

B. State-of-the-art in Human-Robot Collaboration Interface

  • Prior HRC interfaces include Baxter for object manipulation and mobile robots for transport and exploration.The passage describes these mobile robots as navigating autonomously and adapting to dynamic environments.Source: B. State-of-the-art in Human-Robot… · p. 2
  • AI approaches have used deep and reinforcement learning for gesture recognition and adapting to human behavior, while LLM-based HRC remains an emerging research area.The literature review calls for evaluating LLM effects on mutual understanding, trust, efficiency, and collaboration quality.Source: B. State-of-the-art in Human-Robot… · p. 2

III. CHATGPT-ENABLED HUMAN-ROBOT COLLABORATION

RoboGPT links spoken operator input to GPT3.5, which can clarify instructions or pass adequate commands to robot-control functions. Carefully designed prompts adapt the model to the task and constrain its outputs for robotic control.

A. System Design

  • RoboGPT converts speech to text, uses GPT3.5 to assess context and ambiguity, then either clarifies with the operator or routes adequate commands through a decoder to ROS control functions.The workflow is intended to clarify both parties’ intentions and thereby increase transparency and reduce ambiguity.Source: A. System Design · p. 2, Figure 1 · p. 2

B. ChatGPT Assistant

  • The RoboGPT assistant is adapted with prompts that specify system context and sample conversations, shaping communication and mapping outputs to robotic control modules.The prompts cover task scenarios and are intended to let the model process complex instructions across robotic platforms.Source: B. ChatGPT Assistant · p. 3, Figure 2 · p. 3

C. Robot Assembly Assistant

The robot assembly assistant uses dual-stage impedance control to convert discrete target postures into smooth trajectories for a seven-degree-of-freedom arm. Its control description identifies Cartesian joint-state terms and desired inertia, damping, and stiffness.

C. Robot Assembly Assistant

  • Because ChatGPT generates discrete target postures, dual-stage impedance control maps them to continuous positions and smooth trajectories for safe robot operation.The primary controller generates an interim continuous target position; the secondary controller generates the smooth trajectory.Source: C. Robot Assembly Assistant · p. 3, Figure 3 · p. 3
  • The control description identifies joint position, velocity, and acceleration in Cartesian space, alongside desired inertia, damping, and stiffness.The robot used was a Franka Emika Panda seven-degree-of-freedom arm designed for HRC.Source: C. Robot Assembly Assistant · p. 3

D. Command Decoder

The decoder maps assistant responses to ROS control actions and can execute multi-action commands sequentially. The robot’s core functions are grab, move, and drop.

D. Command Decoder

  • A regular-expression search extracts control commands from assistant responses and maps them to ROS functions and parameters that trigger robot actions.The decoder can divide a command sequence into consecutive threads for sequential execution.Source: D. Command Decoder · p. 3
  • For multi-action commands, the decoder divides actions into consecutive threads and executes the robot’s actions sequentially.Source: D. Command Decoder · p. 3
  • Grab, move, and drop are the three control categories: optical input locates targets, impedance control moves the arm, and gripper control supports grasping.Source: D. Command Decoder · p. 3
  • The AI assistant continuously tracks, monitors, and controls the robot’s actions throughout the process.Source: D. Command Decoder · p. 3

IV. TEST CASE AND HUMAN SUBJECT EXPERIMENT

The experiment tested RoboGPT during a three-step assembly task, with a real robot arm synchronized to a digital twin in VR. A randomized within-subject design compared natural-language RoboGPT with fixed spoken commands.

A. Experiment Design

  • Participants assembled a plate onto a workpiece and fastened it with four screws, choosing the operation sequence and when to request tools; Figure 4 presents the scenario and task.Source: A. Experiment Design · p. 4, Figure 4 · p. 4
  • The experiment ran in VR while RoboGPT controlled a real 7-degree-of-freedom Franka Emika arm, synchronized with a digital twin in VR.Source: A. Experiment Design · p. 4
  • The fixed-command condition required exact spoken commands and predefined behavior, whereas RoboGPT accepted participants’ preferred natural-language instructions and supported bilateral communication.Source: A. Experiment Design · p. 4

B. Procedure and Data Collection

V. RESULTS

In interaction examples, RoboGPT tolerated speech-to-text errors and clarified a vague expression before executing robot commands.

A. Interactions

  • RoboGPT made correct decisions despite speech-to-text substitutions of “screw” and “driller” with “school” and “jeweler.”Source: A. Interactions · p. 4
  • The assistant clarified a vague expression before executing the associated robot commands.Source: A. Interactions · p. 5
  • Figure 5 provides interaction-prompt examples, including a conversation while the participant worked with the robot.Source: A. Interactions · p. 4, Figure 5 · p. 5

B. Performance

RoboGPT outperformed fixed commands on objective and self-evaluated task performance. Interviewees suggested that recalling previous patterns may have accelerated task progress.

B. Performance

  • The performance score uses completion time, with higher scores indicating quicker task completion and better performance.Source: B. Performance · p. 5
  • Objective performance scores and subjective self-evaluations from the NASA TLX questionnaire are shown in Figure 6.Source: B. Performance · p. 5, Figure 6 · p. 5
  • p < 0.001 for both performance score and self-evaluated performance in the RoboGPT versus fixed-command conditions.Source: Figure 6 · p. 5, B. Performance · p. 5
  • Post-experiment interviewees suggested that memorizing previous patterns could be a reason for faster task progress.Source: B. Performance · p. 5

C. Trust and Cognitive Load

Questionnaire results indicate that participants perceived less mental load and generally trusted the robot more when it could respond to natural communication. The discussion relates these outcomes to task competence and natural interaction, while noting risks from miscommunication and text-only input.

C. Trust and Cognitive Load

  • p < 0.001 for both trust and cognitive load; questionnaire results indicated less mental load and greater trust when the robot reacted to natural communication.Figure 7 presents the trust and cognitive-load scale results.Source: Figure 7 · p. 5, C. Trust and Cognitive Load · p. 5

VI. DISCUSSION AND LIMITATIONS

  • The assistant improved task performance and was considered more trustworthy than fixed or predefined control, with enhanced trust attributed to task competence.Communication and memory of earlier decisions, such as where to deliver tools, were also described as supporting better partner relationships and trust.Source: §VI · p. 5
  • Participants generally considered the ChatGPT-enabled assistant less mentally demanding, potentially because its autonomy and intelligence reduced the attention needed to control the robot.The authors describe natural interaction as fostering better partner relationships and trust in this task.Source: §VI · pp. 5–6
  • The assistant's scenario understanding could lead to problematic decisions after miscommunication, while text input limits descriptions of objects, robot status, and events.The discussion suggests that understanding additional modalities could improve HRC results.Source: §VI · p. 6

VII. CONCLUSIONS

The conclusion reports higher trust and lower cognitive load with ChatGPT-assisted control than with fixed commands. It also recommends studying more effective LLM-based workflows, including endorsement before ROS commands are sent.

VII. CONCLUSIONS

  • Participants reported higher trust and lower cognitive load with the ChatGPT-enabled assistant than with fixed control commands.The authors present the results as showing a significant impact of using LLMs to collaborate with robots.Source: §VII · p. 6
  • The proposed HRC design fine-tunes a ChatGPT assistant to understand task context and directly control ROS topics in response to verbal commands.The design was demonstrated in an HRC assembly task.Source: §VII · p. 6
  • Future research should investigate effective LLM-based HRC workflows, such as endorsement before commands are sent to ROS.The authors suggest that a proper workflow could support comfortable and intuitive working environments for operators.Source: §VII · p. 6
Loading 2304.12529v1…