PaperlayerView exact revision on arXiv

Source-linked AI summary

Working with AI: A Design Framework for Human-AI Collaboration

Yuqian Lu, Regina Lee, Rui Zhou, Lixin Jiang, Andrew McDaid, Amy Lawrence

arXiv:2609.31793v1cs.AI

TL;DR

Favourable human-AI outcomes depend on deliberate design, not technical capability alone. This white paper offers an iterative framework for designing and evaluating collaboration, with success spanning human, task, organisational, and societal outcomes.

  • Problem

    Favourable outcomes depend on deliberate design, not technical capability alone.

    Source: Top Takeaways · p. 6

  • Method

    The framework iteratively connects desired outcomes and requirements with context, design decisions, and evaluation.

    Source: Top Takeaways · p. 8, IN THIS CHAPTER · p. 10

  • Results

    Effective collaboration should support positive human experiences, strong collaboration and task performance, and sustainable organisational and societal outcomes.

    Source: Top Takeaways · p. 6

  • Takeaways & Limitations

    Fairness, transparency, safety, privacy, security, accountability, training, and ongoing support should be built in from the outset.

    Source: Top Takeaways · p. 6

    The case study used a single session with students, did not evaluate organisational or societal outcomes, and acknowledged but did not evaluate responsible-deployment requirements.

    Source: §4.1.2 · p. 29

Abstract

from arXiv · show

Artificial Intelligence (AI), particularly GenAI, is becoming an increasingly important part of modern work. In industrial settings, AI can support decision-making, automate routine activities, assist humans, and improve productivity. However, successful AI adoption depends on more than what the technology can do. It also depends on how people experience and work with it. This raises an important question: how should human-AI collaboration be designed so that it works well for both people and organisations? This white paper addresses that question by presenting a practical framework for designing human-AI collaboration. The framework considers the human, the AI system, the task, the organisation, and the wider societal environment. It explains what effective collaboration looks like, what conditions influence it, what requirements should be met, and what design decisions organisations should consider. The report also includes a human-AI collaborative assembly system with cobot use case to demonstrate how the framework can be applied in practice. The use case shows how design requirements can be translated into specific collaboration features and evaluated through a case study. The aim of this white paper is to provide a clear and practical guide for designing human-AI collaboration that is effective, human-centred, and responsible.

04 Use Case: Designing and Evaluating a Human-AI Collaborative Assembly

This section presents a framework for designing human–AI collaboration as a context-dependent socio-technical teamwork and responsible-deployment challenge across five interconnected domains. It connects domain-specific preconditions to six design principles and an iterative process that evaluates outcomes against desired outcomes to guide further changes.

IN THIS CHAPTER

  • The framework links desired outcomes and requirements, contextual preconditions, and design decisions in an iterative process tailored to each setting.Evaluation results inform further changes; the framework does not prescribe one configuration for all workplaces.Source: IN THIS CHAPTER · p. 10

What does effective human-AI collaboration look like?

  • Effective collaboration must support positive human experiences, collaboration and task performance, and sustainable organisational and societal outcomes—not task performance alone.These three outcome levels provide the basis for evaluating collaboration effectiveness.Source: What does effective human-AI… · p. 12

How does the human-AI collaboration impact the human?

  • Human outcomes include psychological, professional, physical, and cognitive effects, which can be assessed with physiological, behavioural, and subjective measures.Examples include heart-rate variability, interaction behaviour, surveys, interviews, and think-aloud protocols.Source: How does the human-AI collaboration… · p. 12

How well do the human and AI work together and perform the task successfully?

  • Collaboration quality and task performance are distinct: strong performance alone does not establish effective collaboration, so both dimensions should be evaluated.A system may complete work quickly while producing poor coordination, excessive reliance, or an inappropriate distribution of control.Source: How well do the human and AI work… · p. 14

What are the broader and longer-term consequences of human-AI collaboration?

  • Organisational and societal effects extend beyond individuals and immediate tasks, often emerging over time and across deployments.They can be assessed using organisational metrics such as adoption, staff retention, and ROI, alongside societal research such as public surveys and labour-market data.Source: What are the broader and longer-term… · pp. 15–16

What must be satisfied to achieve the desired outcomes of human-AI collaboration?

  • Requirements for desired outcomes fall into three layers: human needs, effective human-AI teamwork, and responsible deployment.Teamwork requirements include shared goals and understanding, coordination, communication, adaptability, calibrated trust, and appropriate control; responsible deployment should remain beneficial, safe, fair, transparent, accountable, and sustainable.Source: What must be satisfied to achieve the… · p. 17

What conditions must be satisfied for individual users?

  • Human needs are interconnected, and productivity gains may still accompany a poor collaboration experience if autonomy, job security, or privacy concerns are undermined.The section identifies three categories of human needs but does not specify them in the supplied passage.Source: What conditions must be satisfied for… · p. 17

What conditions must be satisfied for good teamwork?

  • Human–AI collaboration should be designed as teamwork, with clear roles, intentions, responsibilities, and control so people and AI can coordinate and recover when collaboration breaks down.Human teammates should retain a more authoritative role over the AI for safety.Source: What conditions must be satisfied for… · p. 18

What conditions must be satisfied for human-AI collaboration deployment to be beneficial long-term?

What influences the human-AI collaboration?

  • Human, AI, task, organisation, and society preconditions shape collaboration requirements and reveal strengths, constraints, and risks for design to address.Assessment is intended to guide design responses rather than label collaboration simply ready or not ready.Source: What influences the human-AI… · p. 20

IN THIS CHAPTER

  • The chapter groups design decisions into six principles that guide organisations in developing effective human–AI collaboration systems.Source: IN THIS CHAPTER · p. 24

3.1 The Design Principles

  • The principles translate the framework into coherent design choices, not a universal checklist; their relative importance depends on each use case’s requirements and preconditions.Safety-critical applications may need greater attention to human control, transparency, and safeguards.Source: §3.1 · pp. 24–25

3.2 Applying the framework and design principles

  • The framework and design principles are applied through a five-step iterative process that moves from understanding context to evaluating outcomes.Source: §3.2 · p. 25, Figure 6 · p. 25

01 Understand the use case

Identify preconditions across the human, AI, task, organisation, and societal domains to establish the collaboration’s context.

01 Understand the use case

  • Identify the collaboration’s preconditions across the human, AI, task, organisation, and societal domains.This establishes the context for collaboration.Source: §01 · p. 26
  • Assess what the human and AI are capable of and what the task requires.Source: §01 · p. 26
  • Consider the organisational support available and the external expectations or constraints that apply.Source: §01 · p. 26

02 Identify the desired outcomes and requirements

Determine which human needs, teamwork requirements, and responsible deployment requirements matter most for the specific use case.

02 Identify the desired outcomes and requirements

  • Prioritise the human needs, teamwork requirements, and responsible deployment requirements for the specific use case.Source: §02 · p. 26
  • Use the preconditions identified in Step 1 to determine which requirements deserve the greatest attention.Source: §02 · p. 26

03 Apply the design principles

Apply the six human-AI collaboration design principles to make decisions suited to the use case’s context and requirements.

03 Apply the design principles

  • Use the six human-AI collaboration design principles to make context-specific design decisions.Source: §03 · p. 26
  • Let preconditions guide which design options are feasible and appropriate.Source: §03 · p. 26
  • Let requirements guide where design attention is most needed.Source: §03 · p. 26

04 Implement the design decisions

Translate the selected design decisions into the human-AI collaboration system, including relevant technical and organisational features.

04 Implement the design decisions

  • Translate selected design decisions into the human-AI collaboration system.Source: §04 · p. 26
  • Implementation may include task allocation, interface design, communication protocols, personalisation options, and safety mechanisms.Source: §04 · p. 26
  • Supporting organisational processes may also be part of implementation.Source: §04 · p. 26

05 Evaluate the outcomes

The section describes evaluating outcomes and revisiting the collaboration design as circumstances change.

05 Evaluate the outcomes

  • Evaluate the system against human, collaboration and task-performance, and organisational and societal outcomes.Use the findings to refine the system and inform the next design cycle.Source: §05 · p. 26
  • Treat the five-step framework as iterative because users, tasks, AI capabilities, and organisational conditions can change.Changing conditions may alter preconditions and requirements, prompting continuous improvement.Source: §05 · p. 26

IN THIS CHAPTER

  • The section introduces a cobot assembly use case and its application of the framework.It also includes a section on what the use case demonstrates.Source: §05 · p. 27, IN THIS CHAPTER · p. 27

4.1 Applying the Human-AI Collaboration Framework to the Use Case

The use case applies the collaboration framework to a cobot-assisted valve assembly task and evaluates human and task outcomes. Its findings illustrate trade-offs and the need to balance outcomes rather than seek one universally optimal configuration.

4.2 What the Use Case Demonstrates

  • The use case translates framework requirements and design principles into configurable collaboration features and evaluates them with human-experience and task-performance measures.The framework is applied from understanding the use case through evaluating outcomes.Source: CHAPTER 04 · p. 28, §4.1.3 · p. 30, §4.1.4 · p. 31

4.1.5 Step 5: Evaluate the outcomes

  • The study used 30 student participants completing six runs, with assigned configurations first and self-selected configurations in the final three.It evaluated human and collaboration/task-performance outcomes, but a single lab-based session could not assess organisational or societal outcomes or validate every framework element.Source: CHAPTER 04 · p. 28, §4.1.2 · p. 29, §4.1.5 · p. 34

The participants trust the robot enough to rely on it appropriately

  • Trust dropped after the wrong-tool error and recovered in Run 6, while reliance rose from 4.77 in Run 1 to 5.77 in Run 6.Reliance dipped to 5.20 in Run 5; because the robot delivered every tool, reliance was built into the task rather than freshly chosen each run.Source: Figure 8 · p. 35, The participants trust the robot… · p. 35

The participants feel in control rather than threatened

  • Among 23 participants who chose robot-initiated handover, perceived control was 5.58 for human-initiated versus 4.37 for robot-initiated handover.For robot-led handovers, perceived control was 4.99 when participants chose the configuration versus 4.33 when it was assigned.Source: Figure 9 · p. 36, The participants feel in control… · p. 36

Workers experience manageable physical and cognitive demand

  • Mental demand decreased across runs, with a temporary Run 5 rise, while physical demand fell to around 1.9 in self-chosen runs excluding Run 5.Human-led initiation came with higher mental demand than robot-led initiation (2.90 vs 2.22); demand responded more to experience than to any single configuration or event.Source: Workers experience manageable physical… · pp. 36–37, Figure 10 · p. 37

The valve is assembled efficiently

  • Average completion time fell from 174 seconds in Run 1 to 123 seconds in Run 4, rose to 157 seconds in Run 5, then reached 118 seconds in Run 6.Self-chosen runs averaged 132 seconds versus 150 seconds for assigned runs (120 seconds excluding Run 5), but the comparison mixes configuration choice with practice because chosen runs came later.Source: The valve is assembled efficiently · p. 37, Figure 11 · p. 38

The collaboration adapts to the worker's changing preferences

4.2 What the Use Case Demonstrates

  • Robot-led initiation improved efficiency but reduced perceived control; confirmation added interaction burden, while detailed explanation slowed performance.The case study concludes that design must balance outcomes and adapt collaboration to the worker, task, and context.Source: §4.2 · p. 39, Implications for Practice · p. 41

Post-Task Survey

The post-task survey asked participants to rate control and AI usefulness on a seven-point scale.

Post-Task Survey

  • Participants completed the survey after each of the six assembly runs, rating each item on a seven-point scale.Source: Post-Task Survey · p. 44

Q4 How much control did you feel you had over the system?

Q5 How helpful or useful was the AI system?

Loading 2609.31793v1…