Understanding Human-Robot Collaboration through Joint Action

1 Introduction

• Joint Action theory organizes collaboration around sharing representations, predicting actions, and integrating the predicted effects of partners’ actions.Grounding accounts also describe how partners establish sufficient evidence of understanding and repair misunderstandings.Source: §1 · p. 2
• Evidence of intention recognition or information provision alone does not establish shared intention or understanding, and task success leaves compatible representations and experienced control unresolved.The review distinguishes implemented mechanisms from stronger claims about shared understanding, commitment, or agency.Source: §1 · p. 2
• Existing reviews address areas such as safety, robot learning, communication, and agency, but the review asks how distinct Joint Action constructs are instantiated and co-occur across empirical HRC studies.It also includes relevant studies that investigate collaborative mechanisms without explicitly using the term joint action.Source: §1 · p. 2
• The review synthesizes 83 empirical papers from 2021 through August 10, 2026, using theory-driven and open coding to develop nine constructs across three capacities.The framework includes 29 subcodes describing operationalizations.Source: §1 · p. 2
• The literature emphasizes partner-action processing, task sharing, and action coordination, while shared intention and cooperative commitment receive less attention.Co-occurrence patterns suggest task division, monitoring, and action prediction are structural prerequisites for many other collaborative mechanisms.Source: §1 · p. 2
• The synthesis identifies research directions concerning partner modeling, agency evaluation, learning through repair, co-training transfer, and pauses, handoffs, and withdrawal.The authors connect current gaps to questions about human understanding, agency, adaptation, and sustained joint activity.Source: §1 · p. 2

2.1 Scope

• HRC is a joint task in which at least one human and one robot both participate in task execution and contribute toward a shared goal.The paper’s robot definition includes embodied, virtually represented, and Wizard-of-Oz agents that perform task-relevant actions affecting collaboration.Source: §2.1 · pp. 2–3

2.2 Joint Action Theory

• Joint Action theory organizes collaboration into Representation, Prediction, and Integration, which the paper treats as capacities that can operate in parallel.Representation covers shared references, task representations, and mutual understanding; Prediction concerns anticipating partners’ actions; Integration concerns organizing actions, repairing breakdowns, and attributing outcomes.Source: §2.2 · p. 3

2.3 Existing Reviews and the Scope of This Survey

• This review distinguishes mechanisms that broader or single-capability reviews group together and examines how Joint Action constructs combine in HRC studies, including studies not labeled as joint action.It preserves distinctions such as intent recognition versus shared intention and information provision versus grounding.Source: §2.3 · pp. 3–4

3 Literature Search and Study Selection

• The review followed PRISMA 2020 and selected studies required a shared task goal with interdependent or coordinated human–robot actions.Studies centered on one-way control, demonstration, information transfer, or other topics rather than collaboration mechanisms were excluded.Source: §3 · p. 4, §3.2.1 · pp. 4–5

3.1 Search Strategy

• The search used collaboration, cooperation, communication, and coordination terms because relevant papers often did not explicitly use Joint Action or related theoretical terms.The database search combined ACM Digital Library with RO-MAN and four journals to broaden disciplinary coverage.Source: §3.1.1 · p. 4, §3.1.2 · p. 4

3.2.2 Resolution of Disagreements

• Reviewers agreed on 177 of 195 full-text screening decisions, yielding 90.77% agreement and Cohen’s 𝜅 of 0.811.The 18 disputed papers were adjudicated, with nine included and nine excluded.Source: §3.2.2 · p. 5

3.3.1 Paper Selection Process

• The final corpus contains 83 studies after 1,523 records were screened and 112 of 195 full-text papers were excluded.The largest stated exclusion reason was that collaborative mechanisms were not a central study focus (72 studies).Source: §3.3.1 · pp. 5–6

3.3.2 Overview of Included Papers

• The 83 papers appeared across five leading venues that together accounted for 91.6% of the corpus, with 30 papers published in 2025.The 2026 count covers publications available through August 10.Source: §3.3.2 · p. 6

3.4.2 Iterative Codebook Development

• The framework was refined through theory-driven pilot coding, including clarifying construct boundaries and adding Breakdown Detection and Repair after failure-and-recovery cases exposed a coverage gap.The initial 11 candidate constructs were refined; Joint Action Goal was not retained as an independent primary code because it could not be consistently distinguished from the study’s shared task goal.Source: §3.4.2 · p. 6

3.4.3 Held-out Test and Final Framework

• Two coders applied the release-candidate manual to 12 held-out papers before freezing the nine-code framework organized across Representation, Prediction, and Integration.Only wording clarifications that did not alter construct structure were allowed during the held-out test.Source: §3.4.3 · p. 6

3.4.4 Full-Corpus Coding and Reliability

• Full-corpus coding recorded construct evidence at passage and paper levels, then used open coding to characterize operationalizations and calculate inter-rater reliability.JA2 disagreements often involved inferring shared intention, while JA8 required clear mutual constraint between partners’ actions during execution.Source: §3.4.4 · pp. 6–7

4.1.1 Joint Attention and Shared Reference.

• Shared reference is constructed through language, deictic cues, and persistent spatial anchors that help partners attend to the same referent and prepare subsequent action.HRC systems often externalize reference in the shared environment; language interpretation can also draw on task history and perspective.Source: §4.1.1 · pp. 7–8

4.1.2 Common Ground and Grounding.

• Grounding relies on shared information, confirmation checkpoints, and clarification that repairs ambiguity and can establish partner-specific conventions.Explicit confirmation can improve grounding but may add delay and frustration; dynamic confirmation remains an open problem.Source: §4.1.2 · pp. 8–9

4.1.3 Task Co-representation.

• Task co-representation encodes task dependencies, roles, partner models, and action relations, usually in robot models, while few studies check human compatibility.These representations specify units and constraints, responsibilities and authority, partner behavior or mental states, and how actions shape each other’s possibilities.Source: §4.1.3 · p. 9

4.1.4 Shared Intention and Cooperative Commitment.

• Shared intention appears as strategy negotiation, mutual responsiveness, and sustained commitment, but evidence is mainly behavioral and rarely tests shared intentional states.Studies describe strategy inference or co-creation, contingent action responses, and persistence through difficulty; observed behavior may also reflect constraints, obligations, or lack of alternatives.Source: §4.1.4 · pp. 9–10

4.2.1 Task Sharing and Complementary Contribution.

• Task sharing makes contributions more predictable through division of labor, leadership relations, and assistance that adapts when partners’ capabilities or conditions change.Division narrows expected contributions, leadership identifies whose behavior guides coordination, and assistance makes the allocation conditional on evolving needs.Source: §4.2.1 · p. 10

4.2.2 Partner-Action Processing.

• Partner-action processing links observation, action prediction, and monitoring, selecting evidence by coordination context and prediction scale while keeping expectations revisable.Explicit communication and predictable action structures can reduce reliance on inference, and the appropriate prediction horizon depends on the decision it must support.Source: §4.2.2 · pp. 10–11

4.3.1 Action Coordination and Coordination Smoothers.

• Action coordination reduces uncertainty through readable behavior and structured transitions, then uses online coupling for residual uncertainty, though rigid protocols and mistimed signals can hinder coordination.Field studies also reveal that apparently fluent operation may depend on workers’ and specialists’ troubleshooting, preparation, and continuous adjustment.Source: §4.3.1 · p. 11

4.3.2 Breakdown Detection and Repair.

• Breakdown repair targets the failed relation: shared understanding is restored through communication, plans or contribution structures are reorganized, and failed execution is repeated or recalibrated.The same observable failure can require different recovery depending on whether understanding, organization, or execution has broken down.Source: §4.3.2 · pp. 11–12
• Breakdown and repair make Integration recursive: partners may revisit shared references, update common ground, or revise task structure, while representational mismatches may surface only during action.The open problem is diagnosing which joint relation failed and confirming that the repaired state is mutually usable; this diagnosis is rarely treated as a collaborative process in its own right.Source: §4.3.2 · p. 12

4.3.3 Sense of Individual and Joint Agency.

• Experienced agency concerns perceived control, authorship, and ownership, which can diverge from objective control allocation in human-robot collaboration.Formal human decision authority does not ensure experienced authorship, and substantial robot contributions do not ensure that the robot is treated as a co-author.Source: §4.3.3 · p. 12
• Responsibility varies with robot status, cooperative tasks can organize human behavior around robots as co-agents, and salient robot contributions can shift ownership from “my drawing” to “our creation”.The same observable contribution may be interpreted as autonomous initiative, compliant execution, or part of a jointly produced action.Source: §4.3.3 · p. 12
• Although many studies examine control allocation, autonomy, or leader-follower relations, substantially fewer directly measure self-agency, shared agency, or ownership of joint outcomes.Future work should pair objective coordination and control measures with direct measures of experienced agency, especially when robot initiative and adaptation make outcome attribution difficult.Source: §4.3.3 · p. 12

5.1 Analysis Method and Result

• Representation covers 72.3% of papers, less than other capacities; JA6–JA8 are more frequent than other categories, JA2 is rarer, and 3.2 Role/7.3 Assistance trail their parent JAs.Source: §5.1 · p. 12, Table 3 · p. 13
• Lift compares observed co-occurrence with independence, while independent expectation provides the expected count; filtering emphasizes combinations with expected counts of at least 4 and |Observed − Expected| ≥4.Source: §5.1 · p. 12
• Conditional proportions flag pairs when at least one exceeds 60% and triplets when at least one exceeds 30%.Source: §5.1 · p. 12
• The threshold is ⌊83×0.05⌋ because small absolute differences can inflate lift when expected co-occurrence counts are low.Source: §5.1 · p. 12
• Three dominating subcodes show strong unidirectional dependencies with multiple other subcodes.Source: §5.1 · pp. 12–13

6.1 Preference vs. Accommodation in Partner Modeling

• Observed adaptation can be mistaken for preference, and retaining that inference after robot limits lift may keep recoverable work assigned to humans.Source: §6.1 · pp. 13–14

6.2 Sense of Agency Beyond Action-Level Feedback

• At unchanged automation, auditory feedback alone raised agency ratings, so assessment should pair agency with task understanding and timely intervention.Source: §6.2 · pp. 15–16

6.3 Self-Improving Robots through Repair

• Current repair resolves breakdowns without changing future behavior, so human troubleshooting recurs; repair records could encode causes, conditions, actions, and verified outcomes.Evaluation should track accumulated extra human effort, avoidable recurrence, and knowledge-maintenance costs.Source: §6.3 · pp. 16–17

6.4 Co-Training Transfer on Task Dependencies

• Co-training gains have not been tested under changed task dependencies; transfer tests should keep actions familiar while rearranging preconditions, separating operation history from dependency knowledge [69].Source: §6.4 · p. 17

6.5 Pause, Handoff, and Withdrawal

• Explicit handoff records can be tested for effects on omissions and duplicate work by tracking receiver confirmation, pending work, and resumption conditions.Evaluation should also track residual human burden, alignment with both parties’ intentions, and subsequent re-engagement choices.Source: §6.5 · pp. 17–18

6.6 Asymmetric Dependence Among HRC Subcodes

• Division, Monitoring, and Prediction are the most depended-on subcodes; several communicative mechanisms depend on Division in 100% of co-occurring cases, while Communication and Confirmation do so in 94%.These are asymmetric associations, not causal links; Division structures roles and handoffs, Monitoring informs repair, and Prediction connects partner information to subsequent action.Source: §6.6 · p. 18

6.7 Common Execution Architecture for HRC

• JA6, JA7, and JA8 appear in 79, 77, and 73 papers, respectively, forming a frequent execution triad of partner processing, task division, and coordination.This pattern is especially apparent in short, task-oriented experiments; sustained commitment, plan authority, and breakdown responsibility often remain outside this execution chain.Source: §6.7 · p. 18

• Selected databases and venues may underrepresent interdisciplinary HRC work, especially studies using different terminology, affecting judgments about which mechanisms are well studied.Source: §7 · p. 18
• Researcher interpretation was required near inclusion boundaries and for implicit constructs, particularly shared intention and cooperative commitment, which were often inferred from behavior.Source: §7 · p. 18
• Phenomena that do not map clearly onto existing Joint Action constructs may be overlooked by the review’s theoretical framing.Source: §7 · p. 18

• Across 83 papers, the review identified nine constructs and 29 subcodes spanning Representation, Prediction, and Integration, finding emphasis on partner-action prediction, task division, and coordination.Shared intention, commitment, and some representational processes received less attention; the framework supports comparison of HRC systems and identification of underexplored directions.Source: §8 · p. 18

• The appendix reports the pilot sampling design, development and held-out samples, and operational coding rules used in the analysis.Source: B Pilot Sampling and Coding Framework · p. 22

B.1 Pilot Sampling Design

• Table 8 lists the six sampling dimensions and their purposes.Source: §B.1 · p. 22
• Table 9 details the boundary-pressure pairs used in the design.Source: §B.1 · p. 22
• Table 10 summarizes three development rounds and the held-out stage.Source: §B.1 · p. 22
• Tables 11 and 12 report the resulting development and held-out samples.Source: §B.1 · p. 22

B.2 Coding Manual and Boundary Rules

• Table 13 presents the operational codebook for the nine primary Joint Action codes.Source: §B.2 · p. 22
• Table 14 lists the high-risk discriminant boundaries applied during full-text coding.Source: §B.2 · p. 22

C Details of the Survey Literature Statistics

• Table 15 lists references for the Joint Action dimensions and subdimensions discussed in the main text.Source: C Details of the Survey Literature… · p. 22, Table 15 · p. 26
• Table 7 covers databases, search queries, and screening results for literature published between 2021 and 2026; no duplicates were identified after merging records.Source: Table 7 · p. 23
• Tables 8 and 9 describe pilot-sampling dimensions and boundary-pressure cases, while Table 10 outlines the development and held-out sampling design.Source: Table 8 · p. 23, Table 9 · p. 23, Table 10 · p. 23
• Tables 11 and 12 report development and held-out samples, and Tables 13 and 14 cover the codebook and high-risk coding boundaries.Source: Table 11 · p. 24, Table 12 · p. 24, Table 13 · p. 25