PaperlayerView exact revision on arXiv

Source-linked AI summary

Transitioning to human interaction with AI systems: New challenges and opportunities for HCI professionals to enable human-centered AI

Wei Xu, Marvin J. Dainoff, Liezhong Ge, Zaifeng Gao

arXiv:2105.05424v3cs.HCcs.AI

TL;DR

AI systems can benefit or harm people, while their evolving and autonomous behavior creates challenges that conventional HCI approaches do not fully address. Through a high-level literature review and holistic analysis, the paper identifies seven main issues, maps HCI opportunities to HCAI design goals, and recommends action by HCI professionals. It concludes that these contributions can guide development toward human-centered AI.

  • Problem

    AI professionals may focus primarily on algorithms rather than useful systems meeting user needs, while AI introduces challenges not encountered in conventional HCI work.

  • Method

    The paper conducts a high-level literature review and holistic analysis to examine AI-related HCI challenges, opportunities, and HCAI design goals.

  • Results

    The analysis identifies unique AI-interaction challenges and ties new HCI opportunities to specific HCAI-driven design goals.

  • Takeaways & Limitations

    The findings guide HCI professionals in addressing AI-specific issues and advancing human-centered AI systems.

Abstract

from arXiv · show

While AI has benefited humans, it may also harm humans if not appropriately developed. The focus of HCI work is transiting from conventional human interaction with non-AI computing systems to interaction with AI systems. We conducted a high-level literature review and a holistic analysis of current work in developing AI systems from an HCI perspective. Our review and analysis highlight the new changes introduced by AI technology and the new challenges that HCI professionals face when applying the human-centered AI (HCAI) approach in the development of AI systems. We also identified seven main issues in human interaction with AI systems, which HCI professionals did not encounter when developing non-AI computing systems. To further enable the implementation of the HCAI approach, we identified new HCI opportunities tied to specific HCAI-driven design goals to guide HCI professionals in addressing these new issues. Finally, our assessment of current HCI methods shows the limitations of these methods in support of developing AI systems. We propose alternative methods that can help overcome these limitations and effectively help HCI professionals apply the HCAI approach to the development of AI systems. We also offer strategic recommendations for HCI professionals to effectively influence the development of AI systems with the HCAI approach, eventually developing HCAI systems.

1. Introduction

AI technology introduces autonomous, potentially non-deterministic behavior that distinguishes AI systems from conventional automated systems and raises risks for humans and society. The paper positions HCI professionals to address these risks through human-centered AI.

  • Why human-centered design matters: AI-related risks include accidents, biased decisions, amplified prejudice, inequality, and potential harm to individuals.The AI Incident Database had collected more than 1000 AI-related accidents, including autonomous-vehicle, trading-algorithm, and facial-recognition incidents.
  • Why human-centered design matters: The paper argues that HCAI should place humans at the center of AI development because ignoring human-centered design may have severe consequences.HCAI promotion remains in its infancy and requires further advancement directed especially toward the HCI community.
  • Unique autonomous characteristics: AI systems can exhibit self-executing, self-adaptive, and potentially non-deterministic behavior under situations not fully anticipated.This contrasts with conventional automation, which performs well-defined tasks using fixed rules and deterministic outputs.
  • Unique autonomous characteristics: The essential distinction between non-AI automation and AI-based autonomy is the presence of human-like cognitive or intelligent capabilities.Both types of systems may still require human intervention for operational safety.
  • Paper aims: The paper aims to identify new HCI challenges and opportunities for applying HCAI as interaction transitions from conventional systems to AI systems.It calls on HCI professionals to take action in addressing these challenges.

2. New challenges for HCI professionals to develop human-centered AI systems

The paper’s review identifies seven main HCI issues that distinguish interaction with AI systems from conventional non-AI systems. It maps these challenges to HCAI design goals centered on human control, usable and explainable systems, human augmentation, and ethical responsibility.

  • Identifying new challenges: A qualitative review of about 890 papers categorized interaction issues into seven main HCI challenges for AI systems.The categories were derived by grouping issues into 10 initial groups and then analyzing primary references.
  • Identifying new challenges: The seven issues capture differences between familiar HCI concerns for non-AI systems and new challenges introduced by AI interaction.Examples include intelligent or invisible user interfaces, explainability, human roles, and more significant ethical concerns.
  • Advancing HCAI: The paper maps specific HCAI-driven design goals to new HCI opportunities and recommends collaboration between HCI and AI professionals.The intended overall outcome is reliable, safe, and trustworthy AI.
  • Advancing HCAI: The HCAI framework places humans at the center across technology, human factors, and ethics, with these aspects working interdependently.Human factors emphasize usable, explainable, human-controllable AI; technology emphasizes human augmentation; ethics emphasizes fairness, justice, privacy, and accountability.
  • Advancing HCAI: HCAI design goals include human-controlled AI, augmented human abilities, usable and explainable AI, and ethical and responsible AI.Meaningful human control supports human decision authority, responsibility, and accountability.

3. New Opportunities for HCI professionals to enable HCAI

The paper presents a holistic HCAI-oriented assessment of human interaction with AI systems and identifies opportunities for HCI professionals across the seven main issues. These opportunities are tied to specific design goals to guide future HCI contributions.

  • HCAI-driven opportunities: The paper identifies new HCI opportunities for professionals to address AI-specific challenges through the HCAI approach.The opportunities are linked to specific HCAI design goals rather than presented as general recommendations alone.
  • Scope of opportunities: The section reviews current research and application status across the seven main issues in human interaction with AI systems.It combines challenge identification with an HCAI-oriented assessment of existing work.
  • HCAI-driven opportunities: The identified opportunities are intended to guide HCI contributions to the development of AI systems.The section frames these contributions as driven by specific HCAI design goals.

3.1 From expected machine behavior to potentially unexpected behavior

AI systems introduce machine behavior that can be unexpected, evolve over time, and reflect choices in algorithms, architectures, training, and data. The paper therefore calls for HCI professionals to help shape, test, and continually improve this behavior with users and AI specialists.

  • Unexpected machine behavior: Unlike fixed-rule systems, AI machine behavior can be non-deterministic, unexpected, and acquired through learning processes.Its behavior is shaped by algorithms, architecture, training, and data.
  • Shaping machine behavior: Interactive machine learning lets users iteratively select, mark, or generate training examples while interacting with system functions.The approach emphasizes human roles, goals, and capabilities in the learning process.
  • Continued improvement: HCI methods can be used to continuously improve AI design through iterative evaluation with end users until undesirable results are minimized.This requires translating user needs into data needs and understanding how AI-generated behaviors are generated, trained, optimized, and tested.
  • Shaping machine behavior: Training-data selection and labeling can substantially influence AI behavior and contribute to biased responses.HCI professionals are expected to incorporate user expectations when tuning algorithms to prevent bias.
  • Continued improvement: AI behavior can evolve after release through software and hardware upgrades, learning, training data, and continuous user input.Recommendation algorithms, for example, update recommendations based on users’ ongoing input.
  • Testing AI systems: Traditional software testing assumes predictable outputs, so AI systems require testing that measures evolving performance and considers human-AI systems over time.Long-term assessment of AI’s influence on humans and society may require social scientists and psychologists.
  • HCI design opportunities: HCI professionals can partner with AI professionals to design interfaces that communicate models and results while accounting for users’ visualization needs and mental models.The paper also stresses that target end users, rather than only AI engineers, must be fully considered.

3.2 From interaction to potential human-AI collaboration

HCI is shifting from conventional human–computer interaction toward potential human-AI collaboration, requiring designers to clarify roles, model teaming, and develop collaborative interfaces. The shared design goal is human-controlled AI in which humans retain critical decision authority.

  • Human-controlled AI requires humans to retain critical decision-making roles while AI serves as a supportive team player or super tool.
  • HCI must address interaction and collaboration between humans and AI systems rather than only conventional human–computer interaction.
  • Future work should determine whether AI agents function as collaborative teammates, peers, leaders, or super tools, and identify who makes final decisions.
  • Conventional interaction models such as MHP and GOMS cannot meet the needs of complex AI-era interactions and collaboration.
  • HCI opportunities include modeling teaming relationships and processes, comparing CSCW with human-AI collaboration, and designing adaptive control and human-directed authority.

3.3 From siloed machine intelligence to human-controlled hybrid intelligence

The paper distinguishes human-controlled hybrid intelligence from siloed machine intelligence and cognitive-model-based systems that do not preserve human centrality. It calls for integrating human-in-the-loop control, cognitive computing, and HCI frameworks so humans remain ultimate decision makers.

  • The HCAI goal is human-controlled AI built through human-machine hybrid intelligence that combines complementary human and AI capabilities.
  • Cognitive-model-based hybrid systems are not true human-machine hybrid intelligence under HCAI because they do not ensure human operators’ central decision-making role.
  • Human-controlled hybrid intelligence embeds AI as a supporting super tool within the human loop, keeping human-driven decision-making primary.
  • Human-controlled AI requires integrated attention to system design, human-machine interaction, and ethical AI design, including effective emergency handover.
  • Future HCI work should integrate cognitive computing with human-in-the-loop methods and develop theory and methods for joint human-machine cognitive systems.

3.4 From user interface usability to AI explainability

AI systems introduce opacity and non-intuitive outputs that make conventional interface usability insufficient. The paper therefore emphasizes user-centered explainability, interactive interfaces, behavioral validation, psychological theory transfer, and sociotechnical analysis.

  • AI’s opaque learning processes and non-intuitive decisions create black-box problems that can reduce decision-making efficiency and public trust.
  • Explainable AI should help target users understand what AI outputs and why, making user understanding the ultimate purpose of explanation.
  • HCI should replace static one-way interpretations with co-adaptive, dialog-based, visual, task-driven, and exploratory interaction designs.
  • Existing explainable AI work often lacks user-participated validation and rigorous behavioral-science methods, while many psychological theories remain without usable computational models.
  • Explainability should account for social and organizational factors including trust, culture, user knowledge, skills, personality, and cognitive styles.

3.5 From human-centered automation to human-controlled autonomy

Autonomous systems amplify familiar automation problems while introducing evolving, uncertain behaviors and new safety concerns. HCAI therefore advocates human-controlled autonomy, reciprocal collaboration, meaningful control, and empirical assessment of autonomy’s effects.

  • Autonomous systems can produce unexpected behaviors, safety risks, inappropriate expectations, and misuse as their learning abilities evolve across environments.
  • Automation research links increasing automation with reduced attention and situation awareness, out-of-the-loop effects, difficult emergency control, and accidents.
  • Research indicates autonomous systems may produce stronger operator shock than automation surprise and may trigger emotional and social effects on operators.
  • HCAI advocates human-controlled autonomy in which humans remain ultimate decision makers and can monitor and quickly take over during emergencies.
  • HCI should empirically assess automation surprise and lumberjack effects across autonomy levels and use the results to inform design recommendations.
  • Future designs should support mutual trust, shared situation awareness, shared control authority, human-machine co-driving, and effective take-over or handoff.

3.6 From conventional interactions to intelligent interactions

AI systems introduce intelligent, multimodal, pervasive interaction that differs from conventional interfaces and creates new HCI design demands. The section identifies opportunities to develop usable interfaces and adapt AI technology to human capabilities.

  • AI-era HCI is transitioning from interaction with non-AI computing systems to interaction with AI systems exhibiting new qualities.
  • HCI design standards for AI systems: Existing interaction paradigms and design standards require empirical verification and development specific to AI systems.
  • New interaction paradigms: New interaction paradigms must integrate visual, audio, touch, gestures, and parallel modalities for intelligent interaction with AI systems.
  • Usable user interface: HCI professionals have an opportunity to design usable interfaces that facilitate human-AI interaction and potential collaboration.
  • Usable user interface: AI technologies can broaden human input through gesture, motion, speech, facial, emotion, intent, and context recognition.
  • Adapting AI technology to human capability: Pervasive, implicit, and multimodal interaction can compete for limited cognitive resources and create high cognitive workload.
  • Adapting AI technology to human capability: HCAI-oriented design should reduce cognitive workload by adapting AI technology to human capabilities rather than adapting humans to AI.

3.7 From general user needs to specific ethical AI needs

AI creates ethical interaction needs beyond conventional usability, functionality, and security concerns. The section connects HCI opportunities to meaningful human control, ethical design practice, explainability, and multidisciplinary assessment.

  • Ethical AI requires HCI attention because conventional HCI focused on general user needs in non-AI system interaction.
  • Meaningful human control: Meaningful human control requires informed decisions, sufficient information for lawful action, and systems that support effective operator control.
  • Meaningful human control: Ethically sensitive decisions should remain under human control, aligning meaningful human control with an HCAI design goal.
  • Meaningful human control: Transparent interaction and situation awareness can keep human operators informed and involved in autonomous-system use.
  • Integration of HCI approaches into AI development: Current ethical AI guidelines often lack shared aims, professional norms, methods for practice, technical detail, and detailed design examples.
  • Integration of HCI approaches into AI development: Human-centered HCI methods such as iterative design and testing can translate user needs into data needs and support behavior testing and bias reduction.
  • An HCI multidisciplinary approach: HCI’s interdisciplinary skills and social-behavioral methods can help assess ethical issues involving privacy, acceptance, and decision-making needs.

3.8 Summary of the opportunities for HCI professionals to enable HCAI

The paper maps seven new interaction issues to HCI opportunities and primary HCAI design goals. Its assessment identifies unique challenges, links them to design goals, and recommends proactive HCI participation.

  • The opportunity summary organizes future HCI work around seven main issues and corresponding primary HCAI design goals.
  • The identified shifts include human-controlled hybrid intelligence, AI explainability, human-controlled autonomy, intelligent interactions, and ethical and responsible AI.
  • The review answers the second research question by identifying unique challenges arising during the transition to interaction with AI systems.
  • The paper ties new HCI opportunities to HCAI-driven design goals to guide professionals in addressing new AI-development issues.
  • Proactive HCI participation is presented as a way to promote and further advance the HCAI approach in AI-system development.

4. The need to improve HCI approaches for applying HCAI

Conventional HCI methods were developed for non-AI systems and leave gaps when applied to AI’s dynamic, pervasive, and intelligent characteristics. The paper assesses existing methods and presents alternatives for HCAI development across stages.

  • Existing HCI approaches may not effectively address AI-specific issues because they were primarily defined for non-AI computing systems.
  • The paper assesses more than 20 methods from HCI, human factors, and related disciplines to identify gaps and alternatives for HCAI.
  • Alternative HCI methods: Table 4 compares conventional HCI methods with seven alternative methods enhanced or borrowed from other disciplines.
  • Limitations of conventional HCI: Conventional HCI methods can assume static human-machine task allocation, tool-like machines, non-intelligent functions, and single user-artifact interaction.
  • Alternative HCI methods: Alternative methods support dynamic function allocation, human-machine teaming, prototyping intelligent functions, and ecological assessment of pervasive environments.
  • Alternative HCI methods: The alternatives can address conventional-method limitations across various AI-system development stages and support HCAI design goals.
  • Strategic recommendations: The paper recommends integrating enhanced HCI methods, leading HCAI promotion, cultivating interdisciplinary talent, collaborating across sectors, and fostering supportive governance.

5. Conclusions

The paper calls on HCI professionals to lead human-centered AI development by addressing AI-specific interaction challenges, expanding HCI opportunities, and improving existing methods. It recommends coordinated methodological, educational, research, and organizational action to maximize AI’s benefits and reduce risks to humans.

  • The paper identifies three research questions concerning AI-related HCI challenges, opportunities for HCAI leadership, and improvements to current HCI approaches.
  • AI systems introduce unique characteristics and challenges that differ from interaction with non-AI computing systems, while inappropriate development may harm humans.
  • It argues that HCI professionals should take a leading role in applying HCAI to address these challenges and maximize AI’s benefits while avoiding risks to humans.
  • The paper identifies new opportunities tied to HCAI-driven design goals across seven main issues in human interaction with AI systems.
  • Recommended actions include enhancing HCI methods, promoting HCAI, updating AI-related skills, training future developers, conducting cross-disciplinary research, and fostering supportive organizations.
  • The paper further calls for a mature organizational environment to implement HCAI and draws on HCI’s earlier human-centered-design leadership during the PC era.
Loading 2105.05424v3…