PaperlayerView exact revision on arXiv

Source-linked AI summary

Human-robot collaboration and machine learning: a systematic review of recent research

Francesco Semeraro, Alexander Griffiths, Angelo Cangelosi

arXiv:2110.07448v4cs.ROcs.LG

TL;DR

Human-robot collaboration research increasingly uses machine learning to model collaborative tasks and cognitive variables, but the evidence base and design patterns remain fragmented. This review analyses selected HRC studies to classify tasks, metrics, cognitive variables, algorithms, sensing modalities, and cross-study trends, finding recurring associations and underused composite learning and sensor-fusion approaches.

  • Problem

    Human-robot collaboration needs cognitive and behavioural models for robots interacting with people, while the use of machine learning across HRC studies requires systematic analysis.

  • Method

    The review analyses selected HRC studies using robotic manipulators and single-user experiments, classifying collaborative tasks, metrics, cognitive variables, learning algorithms, and sensing modalities.

  • Results

    Cross-study patterns link reinforcement learning mainly with robot-related cognitive variables, unsupervised learning with object handover and human-motion variables, while vision dominates sensing and sensor fusion remains uncommon.

  • Takeaways & Limitations

    The review reports emerging task-specific trends and encourages composite machine-learning systems, especially deep reinforcement learning, alongside further development of underused sensing and learning combinations.

  • Takeaways & Limitations

    The reviewed experimental studies use low-payload robotic manipulators, with no machine-learning case study involving high-payload robots.

Abstract

from arXiv · show

Technological progress increasingly envisions the use of robots interacting with people in everyday life. Human-robot collaboration (HRC) is the approach that explores the interaction between a human and a robot, during the completion of a common objective, at the cognitive and physical level. In HRC works, a cognitive model is typically built, which collects inputs from the environment and from the user, elaborates and translates these into information that can be used by the robot itself. Machine learning is a recent approach to build the cognitive model and behavioural block, with high potential in HRC. Consequently, this paper proposes a thorough literature review of the use of machine learning techniques in the context of human-robot collaboration. 45 key papers were selected and analysed, and a clustering of works based on the type of collaborative tasks, evaluation metrics and cognitive variables modelled is proposed. Then, a deep analysis on different families of machine learning algorithms and their properties, along with the sensing modalities used, is carried out. Among the observations, it is outlined the importance of the machine learning algorithms to incorporate time dependencies. The salient features of these works are then cross-analysed to show trends in HRC and give guidelines for future works, comparing them with other aspects of HRC not appeared in the review.

1. Introduction

Human-robot collaboration studies how people and robots act as teams toward shared goals, combining cognitive and physical interaction. This review focuses on machine learning as a way to build cognitive and behavioral models for HRC.

  • 1. Introduction: HRC examines humans and robots jointly pursuing a common goal through cognitive and physical interaction.A cognitive model gathers environmental and user inputs, translates them into usable information, and helps tune robot behavior.
  • 1. Introduction: Collaborative robots share workspaces with users, especially in manufacturing, while HRC also extends to surgical assistance.Cobots are intended to improve productivity and reduce users’ physical or cognitive burden.
  • 1. Introduction: Earlier reviews addressed manufacturing, physical collaboration, human modeling, application sectors, or learning strategies, but not specifically machine learning’s contributions to HRC.The review identifies this machine-learning-focused synthesis as a gap in prior literature.
  • 1. Introduction: The paper surveys interaction types, evaluation metrics, modeled environmental variables, machine-learning algorithms, sensing modalities, and cross-feature trends.Its later analysis also offers guidance for designing HRC systems using machine learning.

2. Research methodology and selection criteria

The review searched multiple publication databases and selected studies using explicit machine-learning, physical-interaction, and behavioral-change criteria. Extracted paper features were organized across tasks, methods, cognitive abilities, results, sensing, and robotic platforms.

  • 2. Research methodology and selection criteria: The search combined HRC and learning terms across ISI Web of Knowledge, IEEE Xplore, and Scopus.The database search was performed in January 2021, and duplicate removal reduced 1053 initial papers to 627 works.
  • 2. Research methodology and selection criteria: Included studies used machine learning in a robot’s behavioral block and validated a human-robot system through joint physical experimentation.The criteria also required a tangible joint goal and an actual robot behavioral change after learning-derived information.
  • 2. Research methodology and selection criteria: Collaborative assembly studies were included despite limited mechanical-power exchange because they examined cognitive interaction and could include physical interaction.This exception was included to retain papers focused on cognitive aspects of collaboration.
  • 2. Research methodology and selection criteria: Figures report feature instances rather than unique papers because individual works can contribute multiple instances of the same feature.This means totals in the trend figures can differ from the number of reviewed papers.
  • 2. Research methodology and selection criteria: The review extracted machine-learning methods, cognitive abilities, results, robotic platforms, and degrees of freedom into a feature table.Table 1 distinguishes reinforcement, supervised, and unsupervised learning and records outcomes such as robustness, proof of concept, and workload reduction.

3. Human-robot collaboration tasks

The review groups HRC studies into cognitive and physical collaborative tasks, then compares how these works validate their systems. Collaborative assembly is increasing over time, while proof-of-concept and baseline-based performance improvement remain prominent evaluation outcomes.

  • Collaborative task categories: The selected studies cover collaborative assembly, object handover, object handling, and collaborative manufacturing using a robotic manipulator with one user.The review reports 15 assemblies, 11 handovers, 17 object-handling works, and 3 collaborative-manufacturing works.
  • Cognitive collaboration: Collaborative assembly requires the robot to assist with sequential subprocesses by understanding the user’s current or next assembly step.Examples include fetching the relevant component, preparing the scene, and predicting the human’s assistance-seeking trajectory.
  • Cognitive collaboration: Object handover studies focus on exchanging an item, especially the robot’s need to understand the user’s intentions and expectations during reception.
  • Physical collaboration: Physical HRC tasks use environmental information both to extract high-level state information and to control robot actuation while exchanging mechanical power with the user.Object handling involves jointly moving a held object, whereas collaborative manufacturing permanently alters an object during the process.
  • Time trends: Collaborative assembly experiments show a constantly increasing time trend, indicating growing attention to tasks requiring interplay between human and robot decisions.The review characterizes this as a shift from robots following user movements toward more collaborative entities.
  • Evaluation metrics: Evaluation metrics cover precision, robustness, proof of concept, performance improvement, and reduction of physical workload.Proof-of-concept studies provide qualitative validation, while performance-improvement studies compare systems against a baseline.
  • Evaluation metrics: Proof-of-concept results remain common, but performance-improvement results slightly surpass them; precision, robustness, and workload-reduction categories are less numerous but increasing.

4. Robot’s cognitive capabilities

The reviewed systems model cognitive and behavioural capabilities across robot, human, and scene factors. Robot- and human-related variables dominate, while scene-related variables are used less frequently.

  • Variable categories: The review groups modelled cognitive variables into categories describing robot aspects, human-user aspects, and the scene.Figure 4 groups 22 works under robot-related variables, 18 under human-related variables, and 7 under scene-related variables.
  • Robot-related capabilities: Robot-related variables commonly represent the robot’s next action, selected from the current interaction state and the expected return of alternative actions.This modelling approach is explicitly linked in the review to reinforcement learning.
  • Robot-related capabilities: Machine-learning outputs can also represent low-level control variables, or an intermediate robot interaction level before final actuation.
  • Human-related capabilities: Human-related variables include task intention, task-specific styles, trust in robot performance, motion, skeleton pose, hand pose, and physical effort.Human effort variables can be minimized by the control routine following the machine-learning algorithm.
  • Scene-related capabilities: Scene-related variables form a small subset of the reviewed models, which otherwise focus mainly on the human or robot components of collaboration.
  • Time trends: Scene-related variables are used less than human- and robot-related variables, with a slight predominance of robot-related variables over human-related ones.

5. Machine learning techniques

The review identifies trends across machine-learning techniques used in HRC, emphasizing diverse algorithm families and the importance of modeling temporal dependencies. Supervised learning and deep learning increased over time, while composite systems remained relatively uncommon.

  • Most reviewed HRC studies use a single machine-learning model, while only a small number combine multiple models.Composite systems increased in 2020 but remained a minority of the selected works.
  • 23 cases used supervised learning, 18 used reinforcement learning, and 12 used unsupervised learning in the selected papers.
  • Supervised-learning use increased sharply in 2018, declined in 2019, and increased again in 2020.Deep-learning models appeared in 1 case in 2018, 3 in 2019, and 8 in 2020.
  • Temporal dependencies are crucial in HRC because interaction actions are influenced by the sequentiality of preceding actions.Hidden Markov models explicitly account for time dependence during learning.
  • Variational autoencoders learn alternate input representations but cannot model time dependence, so they are used as first stages in composite systems.The review also describes GMMs and neural networks as upstream components in composite machine-learning systems.
  • Model-free reinforcement learning is common because it avoids requiring a predefined transition model of the environment.Q-learning is identified as the most frequent model-free reinforcement-learning algorithm.
  • Recurrent neural networks model time dependence by feeding previous outputs into subsequent training inputs.Reported applications include cascaded RNNs for motion-frame sequences and time-scaled RNNs for slow and fast collaboration dynamics.
  • Vision-oriented CNNs process raw images or multi-sensor measurements, but capturing temporal dependence requires concatenating multiple images.

6. Robot’s sensing modalities

Vision is the dominant sensing modality in the reviewed HRC systems, followed by accelerometry. The review also describes sensor fusion and the trade-offs between broad environmental perception, setup assumptions, and safety-relevant information.

  • Vision is the primary informative channel because it can simultaneously capture information about multiple environmental elements.It can provide user position and physical features, supporting robot end-effector positioning for the specific user.
  • 30 cases used vision, compared with 16 using accelerometry, 5 using muscular activity, and 3 using other modalities.The other modalities comprised one brain-activity case and two speech cases.
  • Sensor fusion appeared in 5 cases, combining modalities such as vision, accelerometry, muscular activity, speech, and brain activity.The reviewed combinations were reported in almost equal numbers, with no recurrent combination.
  • Accelerometry is the second most used modality and can measure signals from the user, scene, or robot.
  • Accelerometry-only systems require stronger experimental assumptions because they may lack information about an object’s position relative to its surroundings.Collision avoidance may therefore need to be ensured by presetting experimental conditions.
  • Movement measurements over time can let the system infer interaction forces, providing information relevant to safety in HRC.

7. Discussions

The review cross-analyses machine-learning use across collaborative tasks, cognitive variables, evaluation outcomes, and research directions. It finds task- and variable-specific patterns while identifying important gaps in coverage and standardization.

  • 2018 saw a high increase in papers on human-robot collaboration using machine learning, following the International Federation of Robotics’ cobot definition.
  • Machine learning for collaborative tasks and cognitive variables: Reinforcement learning is slightly predominant in collaborative assembly because it directly produces the robot policy needed for the task.
  • Machine learning for collaborative tasks and cognitive variables: Supervised learning leads object handling, while reinforcement learning is advised as researchers explore more complex task situations.
  • Machine learning for collaborative tasks and cognitive variables: Unsupervised learning predominates in object handover because it is widely used to model robot trajectories, whereas reinforcement learning is uncommon for non-continuous interaction.
  • Machine learning for collaborative tasks and cognitive variables: Reinforcement learning is highly used for robot-related cognitive variables, while unsupervised learning is most used for human motion; human effort has too few cases for a pattern.
  • Results in collaborative tasks: Most reviewed studies use task-specific success measures rather than general standards, motivating results based on standards such as NASA-TLX.
  • Results in collaborative tasks: Object handover shows greater maturity, with nearly every result reporting performance improvement against a baseline; object handling commonly targets reduced physical workload.
  • Other perspectives: Transfer learning, high-payload robots, exoskeletons, mobile manipulators, and machine-learning-enabled safety remain suggested research directions, while collaborative manufacturing lacks enough studies for a reliable pattern.

8. Conclusions

This systematic review analyzes 45 papers on machine learning in human-robot collaboration, classifying tasks, metrics, cognitive variables, algorithms, and sensing modalities. Cross-analysis identifies current trends and research directions, including time-sensitive algorithms, composite learning systems, sensor fusion, and broader robotic architectures.

  • 45 papers were systematically reviewed to identify current trends in machine learning for human-robot collaboration.The review focuses specifically on machine learning techniques used in HRC studies.
  • Collaborative assembly, object handover, object handling, and collaborative manufacturing formed the main task categories, alongside analyses of metrics and cognitive variables.
  • Supervised, unsupervised, and reinforcement learning were distinguished, while composite systems—especially deep reinforcement learning—were encouraged despite limited prevalence.
  • Vision was the most common sensing modality, followed by accelerometry, whereas muscular-activity sensing, other modalities, and sensor fusion remained uncommon.
  • Cross-feature analysis found reinforcement learning predominantly modeled robot-related cognitive variables, while unsupervised learning was more associated with object handover and variable modeling.
  • Future research should examine non-manipulator robotic architectures, safety, digital twins, and other HRC domains connected with machine learning.
Loading 2110.07448v4…