Source-linked AI summary
The future of human-AI collaboration: a taxonomy of design knowledge for hybrid intelligence systems
Dominik Dellermann, Adrian Calma, Nikolaus Lipusch, Thorsten Weber, Sascha Weigel, Philipp Ebel
TL;DR
Real-world business tasks still exceed what machines can solve alone, motivating socio-technological systems that combine human and artificial intelligence. Using an iterative taxonomy-development method, the paper structures design knowledge for hybrid intelligence systems and concludes with guidance for their development, while noting that the taxonomy remains an initial and descriptive step.
Problem
Machines cannot yet solve many complex real-world business tasks alone, creating a need for structured design knowledge about systems combining human and artificial intelligence.
Method
The paper develops a taxonomy by reviewing interdisciplinary human-in-the-loop research and examining seven empirical applications of hybrid intelligence systems.
Results
The taxonomy organizes hybrid intelligence design knowledge into four meta-dimensions, 16 sub-dimensions, and 50 categories.
Takeaways & Limitations
The taxonomy offers initial conceptual design knowledge and useful guidance for developing hybrid intelligence systems in real-world applications.
Takeaways & Limitations
The taxonomy is an initial, descriptive step based on a case selection biased toward decision-problem contexts and requiring broader practical applications and more prescriptive guidance.
Abstract
from arXiv · showhide
Recent technological advances, especially in the field of machine learning, provide astonishing progress on the road towards artificial general intelligence. However, tasks in current real-world business applications cannot yet be solved by machines alone. We, therefore, identify the need for developing socio-technological ensembles of humans and machines. Such systems possess the ability to accomplish complex goals by combining human and artificial intelligence to collectively achieve superior results and continuously improve by learning from each other. Thus, the need for structured design knowledge for those systems arises. Following a taxonomy development method, this article provides three main contributions: First, we present a structured overview of interdisciplinary research on the role of humans in the machine learning pipeline. Second, we envision hybrid intelligence systems and conceptualize the relevant dimensions for system design for the first time. Finally, we offer useful guidance for system developers during the implementation of such applications.
1. Introduction
AI has advanced rapidly, but machines still cannot independently handle many complex real-world business tasks. The paper therefore proposes hybrid intelligence systems and a taxonomy to guide their design and implementation.
- Recent deep-learning advances have produced strong performance on tasks including autonomous driving, cancer detection, and complex games.
- Machines still struggle with expertise-based decision making, planning, creativity, adaptation to dynamic environments, self-adjustment, and common sense.
- The study reviews interdisciplinary research and examines practical business applications using a taxonomy development method.
- The paper provides a structured overview of human roles in machine learning, conceptualizes hybrid intelligence systems and design dimensions, and offers implementation guidance.
2. Related Work
The related work frames hybrid intelligence as a continuously collaborating human-machine ensemble rather than isolated human involvement in the machine learning pipeline. Its purpose is to combine complementary strengths, achieve superior outcomes, and learn from one another over time.
- Artificial intelligence refers to machines performing activities associated with human thinking, including decision-making, problem solving, and learning.
- Machine learning is a subset of AI in which performance on tasks improves with experience according to a performance measure.
- Human-in-the-loop learning incorporates people into parts of the machine learning pipeline, contrasting with systems built around static knowledge repositories.
- Even with deep learning and AutoML, humans remain involved in feature engineering, parameter tuning, sense-making, and training.
- Hybrid intelligence combines complementary human and artificial strengths in a socio-technological ensemble that can achieve superior results and continuously learn from each other.
- The central design question is which decisions and how they should be made when implementing systems whose human and AI components co-evolve over time.
3. Methodology
The study develops its taxonomy through an iterative procedure grounded in interdisciplinary literature and empirical applications. It defines four generic design dimensions and evaluates the taxonomy against objective and subjective conditions.
- The taxonomy follows Nickerson et al.’s method, which iteratively defines meta-characteristics, stopping conditions, and empirical-to-conceptual or conceptual-to-empirical approaches.
- The four meta-characteristics are task characteristics, learning paradigm, human-AI interaction, and AI-human interaction.
- Objective stopping conditions required examining all sampled literature and empirical cases, classifying at least one object under each characteristic, and adding no new dimensions or characteristics in the final iteration.
- Subjective conditions assessed conciseness, robustness, comprehensiveness, extensibility, explanatory value, and information availability.
- The study conducted three iterations, beginning with interdisciplinary theoretical knowledge and extending it through seven real-world applications of human-AI combinations.
- The literature search used interdisciplinary databases and returned 2505 hits for screening, while empirical analysis examined seven applications.
4. Taxonomy of Design Knowledge on hybrid intelligence Systems
The taxonomy organizes hybrid intelligence design around task characteristics, learning paradigms, human-AI interaction, and AI-human interaction. It further specifies how tasks, shared representations, pipeline timing, teaching, and augmentation shape system design.
- The taxonomy is organized around four meta-dimensions: task characteristics, learning paradigm, human-AI interaction, and AI-human interaction.The dimensions follow the sequence of design decisions needed to develop hybrid intelligence systems.
- Task characteristics: Task characteristics describe how humans and machines collaboratively carry out tasks, including recognition, prediction, reasoning, and action.Recognition identifies objects or language, prediction forecasts future events, reasoning builds models for complex problems, and action requires an agent to act.
- Task characteristics: Shared data representation determines what data humans and machines see before executing their tasks, ranging from features and instances to concepts and schemas.These levels vary in granularity and abstraction to create shared understanding between humans and machines.
- Learning paradigm: Human input enters the machine-learning pipeline through feature engineering, parameter tuning, and training, with annotations, usage behavior, and demonstrations supporting learning.Human annotations support datasets such as ImageNet and LUNA16, while recommender systems use human behavior and robotic applications use human examples.
- Human-AI interaction: Hybrid intelligence systems support human, machine, and hybrid augmentation by combining algorithmic predictions, human input, and reciprocal improvement.The taxonomy also distinguishes machine-learning paradigms, human-learning paradigms, teaching forms, interaction timing, expertise, aggregation, and incentives.
- AI-human interaction: Human teaching can be explicit or implicit and may involve demonstrations, labeling, troubleshooting, or verification, with individual or collective input aggregated through different mechanisms.Query strategies may be offline, online, or active, determining when human actions enter the learning process.
5. Discussion
The taxonomy frames hybrid intelligence as socio-technical collaboration that combines human and artificial intelligence to improve task performance and adaptation. It identifies benefits for both machines and humans, including domain integration, interpretability, control, and feedback.
- The taxonomy offers initial descriptive design knowledge for developing hybrid intelligence systems and identifies applications, mechanisms, and benefits of human-machine collaboration.Its purpose is to guide developers in implementing hybrid intelligence systems for real-world applications.
- Combining human and machine intelligence can integrate domain insights into models, adapt learners to dynamic problems, and enhance trust through interpretability and human control.These benefits are presented as examples of how hybrid systems can leverage complementary strengths.
- Hybrid systems can also improve human problem solving through feedback about task conduct or performance and machine feedback that augments human intelligence.
6. Conclusion
The paper develops descriptive design knowledge for hybrid intelligence systems through a taxonomy built from interdisciplinary research and seven empirical applications. It presents four meta-dimensions, 16 sub-dimensions, and 50 categories while identifying clear boundaries for future refinement.
- The taxonomy structures design knowledge across task characteristics, learning paradigm, human-AI interaction, and AI-human interaction.
- The taxonomy contains 16 sub-dimensions and 50 categories developed through a taxonomy methodology, interdisciplinary research, and examination of seven empirical applications.
- The paper contributes a structured overview of humans' role in machine learning, an initial conceptualization of hybrid intelligence systems, and guidance for real-world implementation.
- The taxonomy is a first step because its empirical case selection is biased toward decision-problem contexts and its results remain overly descriptive.Future work is intended to expand practical applications, consolidate overlapping dimensions, and develop prescriptive guidance.