PaperlayerView exact revision on arXiv

Source-linked AI summary

Human-AI Collaboration: The Effect of AI Delegation on Human Task Performance and Task Satisfaction

Patrick Hemmer, Monika Westphal, Max Schemmer, Sebastian Vetter, Michael Vössing, Gerhard Satzger

arXiv:2303.09224v1cs.HCcs.AI

TL;DR

Prior delegation research left unclear how AI-assigned tasks affect human performance and satisfaction, particularly when people know delegation is occurring. In an experiment, the paper finds improved performance and satisfaction with AI delegation regardless of awareness, with increased self-efficacy explaining these effects.

  • Problem

    Prior evaluations used synthetic or context-independent human predictions, leaving unclear how AI delegation affects human performance, task satisfaction, and their underlying mechanisms in human-AI teams.

    Source: §3 · p. 3

  • Method

    The authors propose a behavioral model and test AI delegation in an image-classification experiment, examining self-efficacy as a possible mechanism.

    Source: §1 · p. 2, §4.1 · p. 4, §6 · p. 9

  • Results

    AI delegation improved human task performance and satisfaction, regardless of awareness, while increased self-efficacy explained the effects; combined team performance exceeded human-alone and AI-alone performance.

    Source: §6 · p. 9, §7 · p. 10

  • Takeaways & Limitations

    The findings provide initial evidence that AI models taking on task-management responsibilities can be an effective form of workplace human-AI collaboration.

    Source: §6 · p. 9, §7 · p. 10

    The experiment did not examine human delegation, and the finding that communicating AI delegation had no effect needs verification across other communication forms and task settings.

    Source: §6 · p. 9

Abstract

from arXiv · show

Recent work has proposed artificial intelligence (AI) models that can learn to decide whether to make a prediction for an instance of a task or to delegate it to a human by considering both parties' capabilities. In simulations with synthetically generated or context-independent human predictions, delegation can help improve the performance of human-AI teams -- compared to humans or the AI model completing the task alone. However, so far, it remains unclear how humans perform and how they perceive the task when they are aware that an AI model delegated task instances to them. In an experimental study with 196 participants, we show that task performance and task satisfaction improve through AI delegation, regardless of whether humans are aware of the delegation. Additionally, we identify humans' increased levels of self-efficacy as the underlying mechanism for these improvements in performance and satisfaction. Our findings provide initial evidence that allowing AI models to take over more management responsibilities can be an effective form of human-AI collaboration in workplaces.

1 INTRODUCTION

AI delegation assigns task instances according to the relative capabilities of the AI model and the human, but prior evidence left open how human performance and task perceptions respond when delegation occurs. In a randomized experiment, delegation improved human performance and satisfaction regardless of awareness, with increased self-efficacy identified as an underlying mechanism.

1 INTRODUCTION

  • AI models can match or exceed human experts in some applications, while humans may remain more accurate on cases where their capabilities complement the model.The paper notes possible reasons including limited model capacity or training data and human access to side information.Source: §1 · p. 1
  • Prior evaluations of capability-aware delegation used synthetic or context-independent human predictions, leaving open how people perform and perceive tasks when an AI assigns their instances.The study asks whether task performance and satisfaction change and whether self-efficacy helps explain those effects.Source: §1 · pp. 1–2
  • Task satisfaction matters alongside performance because it relates to organizational commitment and productivity.The study therefore examines whether AI-assigned instances affect satisfaction as well as performance.Source: §1 · p. 2
  • The study tested delegation in a randomized experiment with 196 participants classifying images using an AI model that estimated instance-specific human confidence.The model delegated instances when estimated human confidence exceeded classifier confidence.Source: §1 · p. 2
  • Delegated instances significantly improved human performance and satisfaction, and combined team performance exceeded both human-alone and AI-alone performance.The improvements did not differ according to whether participants knew that delegation was taking place; increased self-efficacy explained the effects.Source: §1 · p. 2

2 RELATED WORK

Human-AI collaboration includes both AI-assisted decisions and AI-initiated delegation, in which the model assigns instances to humans or handles them itself. Prior delegation methods considered human capabilities, but behavioral evidence and understanding of human perceptions and mechanisms remained limited.

2 RELATED WORK

  • AI-assisted decision-making gives recommendations while humans retain the final choice, creating challenges around appropriate reliance and over-reliance.Explanations may affect reliance, and expertise influences confidence in overruling recommendations.Source: §2 · p. 3
  • AI-initiated delegation differs from recommendation-based assistance because the model decides whether to predict or pass an instance to a human.Delegation approaches may learn human strengths from human predictions as well as ground-truth labels.Source: §2 · p. 3
  • AI delegation assigns task instances to the human or AI based on their capabilities, but earlier evaluations relied on synthetic or context-independent human predictions.Human behavior may change when people know they are part of a human-AI team, and effects on task satisfaction and underlying mechanisms had not been established.Source: §2 · p. 3

3 THEORY DEVELOPMENT AND HYPOTHESES

The paper develops hypotheses that capability-aware AI delegation improves human performance and satisfaction, with self-efficacy as a possible mediator. It tests these predictions against human-alone and AI-alone comparisons and explores whether communicating delegation changes the outcomes.

3 THEORY DEVELOPMENT AND HYPOTHESES

  • The proposed model expects AI delegation to improve human task performance and satisfaction compared with working alone, with self-efficacy mediating both effects.The hypotheses draw on organizational-behavior research on supervisor-to-employee delegation and self-efficacy.Source: §3 · pp. 3–4
  • The proposed delegation effect assumes that the AI learns human strengths and weaknesses and assigns instances aligned with human competencies.The rationale draws on organizational research linking competency-aligned supervisor delegation to employee empowerment, motivation, and performance.Source: §3 · p. 3
  • The model compares outcomes for humans handling AI-delegated instances with human-alone and AI-alone conditions.Figure 2 summarizes the proposed effects on human performance and satisfaction.Source: §3 · p. 4, Figure 2 · p. 4
  • The study also asks whether task performance and satisfaction differ when the AI's delegation is hidden rather than communicated to participants.The authors expect a positive delegation effect in either condition and explore whether informing participants yields an additional benefit.Source: §3 · p. 4

4 METHODOLOGY

The study evaluates capability-aware AI delegation through image classification, using a model that estimates human error and a web experiment with informed-delegation, hidden-delegation, and human-alone conditions. It measures classification accuracy, satisfaction, self-efficacy, and participant characteristics.

4 METHODOLOGY

  • The methodology tests whether AI delegation affects human performance, satisfaction, and self-efficacy through a web-based experiment.The section describes the data, AI model, experiment, and evaluation measures.Source: §4 · p. 4, §4.3 · p. 5

4.1 Data

  • The test bed contains 1,200 ImageNet-subset images balanced across 16 classes, with phase-noise distortion and existing human predictions for model training.The generic classification task was selected for non-specialist participants and to let the model learn human strengths and weaknesses.Source: §4.1 · p. 4

4.2 Development of the AI Model

  • The AI system combines an image classifier with a human-error model and delegates an instance when the latter has higher confidence.Both components use DenseNet-161 models fine-tuned on distorted images.Source: §4.2 · pp. 4–5

4.3 Experimental Design

  • Participants were randomly assigned to informed delegation, hidden delegation, or human-alone conditions to compare task outcomes.The experiment included 196 participants in the final sample after attention and manipulation-check exclusions.Source: §4.3 · p. 5, §4.3.1 · p. 5, §4.3.2 · p. 6

4.3.1 Participants.

  • The final sample comprised 196 online recruits after exclusions for failed attention or manipulation checks and missing data.Participants were recruited via Prolific; the planned sample size was 176.Source: §4.3.1 · p. 5

4.3.2 Study Procedure.

  • In the informed condition, participants labeled AI-selected images and saw additional images the AI had already classified; hidden-delegation participants labeled AI-selected images without being told.Human-alone participants labeled a random test-set sample, and all participants completed 20 main-task images.Source: §4.3.2 · pp. 5–6, Figure 3 · p. 5

4.3.3 Evaluation Measures.

  • Performance was classification accuracy, while satisfaction and self-efficacy were measured with multi-item Likert scales.The evaluation also recorded visual cognitive ability and other participant characteristics as controls.Source: §4.3.3 · p. 6

5 RESULTS

Both communicated and hidden AI delegation improved human task performance and satisfaction compared with working alone. Self-efficacy mediated the effects on both outcomes.

5.1 Statistical Specification

  • Direct effects were tested with univariate regressions, mediation with PROCESS model 4, and all analyses controlled for seven participant characteristics.The controls were task experience, algorithm attitude and use, cognitive ability, education, age, and gender.Source: §5.1 · pp. 6–7

5.2 Effect of AI Delegation on Task Performance

  • Combined human-AI team performance reached 80.01%, exceeding AI-alone (75.83%) and human-alone (67.13%); both delegation groups also outperformed human-alone.Individual task performance was 84.51% for delegation and 83.73% for hidden delegation, versus 67.13% for human-alone.Source: §5.2 · p. 7, Figure 4 · p. 7

5.3 Effect of AI Delegation on Task Satisfaction

  • Task satisfaction averaged 3.65 (delegation) and 3.62 (hidden delegation), versus 3.35 (human-alone), with no significant difference between delegation groups.Satisfaction improved whether or not participants were told about delegation.Source: §5.3 · p. 7, Figure 4 · p. 7

5.4 Mediation of Self-efficacy in Effect of AI Delegation on Task Performance

  • Self-efficacy mediated AI delegation’s effect on task performance in both the communicated and hidden delegation groups.The mediation indices were β = 0.47 for delegation and β = 0.50 for hidden delegation.Source: §5.4 · p. 8, Figure 5 · p. 8, Table 1 · p. 8

5.5 Mediation of Self-efficacy in Effect of AI Delegation on Task Satisfaction

  • Self-efficacy mediated AI delegation’s effect on task satisfaction in both the delegation and hidden delegation groups.The mediation indices were β = 0.29 for delegation and β = 0.31 for hidden delegation.Source: §5.5 · p. 8, Figure 5 · p. 8, Table 1 · p. 8

6 DISCUSSION

AI delegation improves human task performance and satisfaction, regardless of whether people know about it; increased self-efficacy explains these effects. In this image-classification study, delegation also yields complementary team performance, while the findings’ workplace implications depend on task and worker context.

6 DISCUSSION

  • The study tested delegation on image classification, a task selected for independent human and AI decisions and complementary strengths.The authors note that image classification ranges from low-stakes animal classification to high-stakes cancer detection.Source: §6 · p. 9
  • Combined human-AI performance surpasses both human and AI performance when each works alone.The authors attribute this complementary performance to humans receiving images that match their skills.Source: §6 · p. 9
  • AI delegation improves human task performance and satisfaction, regardless of whether humans know delegation is taking place.Communicating the delegation neither boosts nor reduces performance or satisfaction in this study.Source: §6 · p. 9
  • Increased self-efficacy explains the performance and satisfaction effects, partially mediating performance and fully mediating satisfaction.People reported greater confidence in completing tasks composed by the AI model.Source: §6 · p. 9
  • The study extends prior delegation research by considering both team members’ capabilities and human perceptions of AI as a task manager.The authors suggest managers could consider AI delegation for higher performance and employee satisfaction, while noting that preferences and task context may shape reactions.Source: §6 · p. 9
  • AI delegation could create scaling potential in gig-economy services by assigning work to humans or algorithms according to urgency, requirements, or human availability.The proposed arrangement differs from typical algorithmic management because the AI serves as both manager and worker.Source: §6 · p. 9
  • Algorithmic management can bring efficiency and improved perceptions, but may also cause uncertainty or discomfort; when people view delegation positively remains an open question.Reactions may vary with task meaning, personality, and whether work is perceived as tedious or desirable.Source: §6 · pp. 9–10
  • The study’s findings are bounded by its nonexpert participants and nonspecialized task, and may differ in settings requiring expertise.The authors call for testing other communication approaches and task settings, including explanations of why instances are delegated.Source: §6 · pp. 9–10

7 CONCLUSION

This study examines AI delegation as human-centered collaboration and proposes a behavioral model to investigate its effects and underlying mechanism. Its findings show improved human performance and task satisfaction, with increased self-efficacy explaining these positive effects.

7 CONCLUSION

  • AI delegation improves human task performance and task satisfaction, with increased self-efficacy explaining these positive effects.The conclusion raises whether AI-managed humans are suitable collaborators in particular workplace settings.Source: §7 · p. 10

Related papers

Loading 2303.09224v1…