PaperlayerView exact revision on arXiv

Source-linked AI summary

From Explainable to Interactive AI: A Literature Review on Current Trends in Human-AI Interaction

Muhammad Raees, Inge Meijerink, Ioanna Lykourentzou, Vassilis-Javed Khan, Konstantinos Papangelis

arXiv:2405.15051v1cs.HC

TL;DR

Human-AI interaction research remains dominated by explanations, with limited attention to users’ agency and active control. This paper systematically reviews interaction research beyond explainability, finding persistent gaps in user participation and practical evidence while advocating feedback, co-creation, and adaptation of AI mechanics.

  • Problem

    Most human-AI interaction literature focuses on explainability, leaving active user interaction, agency, and control beyond explanations insufficiently examined.

  • Method

    The paper conducts a systematic literature review using structured protocols and analyzes interaction across users, implementations, and goals.

  • Results

    The analysis finds limited user involvement in co-design and practical interaction, with user agency over AI systems beyond explainability still largely constrained.

  • Takeaways & Limitations

    Future Interactive AI should prioritize active user interaction through feedback, co-creation, and opportunities to adapt AI mechanics.

  • Takeaways & Limitations

    The review includes theoretical interactivity studies without actual user testing, and its search may have missed relevant work in a rapidly changing field.

Abstract

from arXiv · show

AI systems are increasingly being adopted across various domains and application areas. With this surge, there is a growing research focus and societal concern for actively involving humans in developing, operating, and adopting these systems. Despite this concern, most existing literature on AI and Human-Computer Interaction (HCI) primarily focuses on explaining how AI systems operate and, at times, allowing users to contest AI decisions. Existing studies often overlook more impactful forms of user interaction with AI systems, such as giving users agency beyond contestability and enabling them to adapt and even co-design the AI's internal mechanics. In this survey, we aim to bridge this gap by reviewing the state-of-the-art in Human-Centered AI literature, the domain where AI and HCI studies converge, extending past Explainable and Contestable AI, delving into the Interactive AI and beyond. Our analysis contributes to shaping the trajectory of future Interactive AI design and advocates for a more user-centric approach that provides users with greater agency, fostering not only their understanding of AI's workings but also their active engagement in its development and evolution.

1. Introduction

This review examines human-AI interaction beyond explanations and contestation, emphasizing active user agency, participation, and control. It synthesizes research gaps and proposes participatory design and stakeholder evaluation guidelines.

  • Research gap: Most human-AI literature centers on explainability, while users typically receive explanations or occasionally contest decisions rather than shape AI systems.The review identifies limited agency beyond these forms of interaction.
  • Review scope: The review consolidates state-of-the-art work on explicit, intentional, and informed interaction across Human-Centered AI, Explainable AI, and Interactive AI.It extends the review scope beyond explanations to active interaction with AI systems.
  • Findings: The analysis finds that many studies exclude end-users from co-design and even simpler forms of interaction with AI systems.This finding concerns user participation across the reviewed literature.
  • Findings: Active interaction research predominantly targets low-risk areas and trivial tasks, while high-risk domains such as healthcare and security receive less attention.The review contrasts education, leisure, and sports with healthcare and security.
  • Findings: Only a handful of studies permit active modification of AI mechanics, particularly in Interactive Machine Learning, leaving user agency and control limited.The review identifies Interactive Machine Learning as an opportunity for greater agency beyond system understandability.
  • Future directions: The proposed guidelines advocate participatory human-AI interaction design and reflective evaluation by relevant stakeholders.The guidelines aim to support more balanced interaction between humans and AI systems.

2. Background

The background traces AI’s shift toward predictive accuracy and opaque complexity, alongside the rise of explainability as the dominant response. It frames the review’s broader concern: enabling meaningful human participation, agency, and control beyond explanations.

  • AI complexity: Data-intensive AI development has emphasized predictive accuracy over human agency, while complex systems increasingly obscure how decisions are made.Deep neural networks and other complex systems are described as difficult to trace and interpret.
  • Explainable AI: Explainable AI makes AI decisions more understandable through transparency and traceability, but research on users’ effective understanding and interaction remains limited.The review describes XAI as a major advance that does not address every purpose of increased user integration.
  • Explainable AI: Many explainability studies interpret AI decisions without sufficiently examining how users interact with explanations across stakeholder contexts and goals.The review calls for explanation mechanisms adapted to usage contexts and stakeholder needs.
  • Beyond explainability: Human-AI interaction spans human involvement in development through humans-in-the-loop to empowering end-users as co-creators of Interactive AI systems.The background presents collaboration as a central way to enhance the human role.
  • Beyond explainability: Human expertise, trust, and control influence collaboration, while improper integration can undermine its benefits for decision-making, trust, and acceptance.The cited discussion links human involvement with promising outcomes but emphasizes the need for balance.
  • Review motivation: The review therefore evaluates whether current HCI practices support interactive systems that foster user agency and adaptive control beyond explainability.Its motivation is to establish the state of practice through systematic literature evaluation.

3. Methodology

The review used a systematic, protocol-guided process to identify studies of explicit, intentional, and informed human-AI interaction, retaining 54 studies for analysis. Its eligibility framework excluded purely technical, passive, and data-provider roles while emphasizing active user interaction.

  • Review protocol: The review adapted PRISMA and used documented protocols to guide study assessment and ensure evaluation consistency.The methodology covered inclusion and exclusion criteria, searching, data collection, and study evaluation.
  • Eligibility criteria: Eligible interaction had to be explicit, intentional, and informed, meaning users could affect system functions, initiate interaction, retain control, and understand its impact.Users had to be active participants rather than merely passive data providers.
  • Search strategy: Searches covered human-centered, interactive, collaborative, hybrid, explainable, and contestable AI literature across Google Scholar and Scopus.Search terms were selected through an explanatory search of established research libraries before querying the two search engines.
  • Study selection: Snowballing added 15 studies to the 39 primary studies, producing 54 studies selected for the survey.One forward and backward snowballing iteration was applied to studies categorized as level 4.
  • Study corpus: Approximately 75% of papers in almost all categories were published in the last five years, indicating a recent surge of interest in interactivity and user experience.The survey included only two studies published before 2014.

4. Analysis of Dimensions

The analysis organized extracted research attributes into dimensions covering users, data, implementations, applications, and study goals. Selective coding and design thinking were used to connect recurring patterns and insights.

  • Analytical framework: Selective coding grouped study attributes by users, data modality, implementation, application area, and study goals.The user categories included AI, domain, and novice users; goals included user experience, transparency, and interaction.

4.1. AI Users

The review treats users and their participation as central to interactive, user-centered AI, but finds that end-user involvement and agency remain limited. Studies often rely on experts or generic inclusion approaches, while active interaction is concentrated in lower-risk applications.

  • User participation: Interaction is presented as a key factor in transforming traditional AI into user-centered AI and assessing applicability for target audiences.Restricting user participation affects interactivity and users’ perception of being part of the process.
  • User groups: Target users range from AI experts to novices, and their selection depends on application type, recruitment resources, and system complexity.Personas help distinguish user groups and their needs, including abilities and preferences.
  • User groups: Most interactive studies target expert AI or domain users, while many collaborative and human-centered AI studies do not identify a specific target group.This pattern indicates uneven profiling of intended users across the surveyed literature.
  • User testing: Actual end-user testing is not widely practiced, and relying on AI experts may not represent how affected users behave with the system.Domain experts may themselves be novices in AI when using systems for professional work.
  • Agency and control: User exclusion reduces human control, while balancing system autonomy with user control depends partly on the knowledge and input of users.The review links exclusion with concerns about monopolies, fairness, and agency in AI decision-making.
  • Agency and control: Overall, many studies fail to include or test end-users, limiting their agency and control despite recent emphasis on inclusion in the loop.Human-centered and collaborative approaches are often too generic for diverse, targeted user groups.

4.2. AI Implementations

Interactive AI studies span theoretical proposals to practical systems, but implementation, application coverage, and data reporting remain uneven. Practical work is concentrated in lower-risk domains and often uses textual or conversational interaction.

  • Implementation of the Solution: Implementation level is used to distinguish working systems, theoretical solutions, and prototypes, with greater implementation generally corresponding to greater interactivity.The review evaluates how concrete implementation affects the ability to assess interaction in specific contexts.
  • Implementation of the Solution: Many collaborative, human-centered, and hybrid AI studies remain theoretical, whereas interactive machine learning and interactive AI have more developed practical implementations.The review contrasts intended systems with implemented interactive solutions.
  • Implementation of the Solution: Practical collaborative systems report promising interaction outcomes, including better user satisfaction, while interaction mode can shape behavior such as question frequency.One cited comparison found that users asked more questions through text than voice input.
  • Application Domain: Application coverage is uneven: about 33% of studies do not specify an application, and implemented work is concentrated in education and leisure rather than high-stakes domains.Healthcare and other high-stakes applications remain comparatively under-explored, despite examples involving diagnosis, patient investigation, and fairness.
  • Modalities of Data: Text is the most common data modality, appearing in around 33% of studies and supporting conversational interfaces across educational, robotic, and other interactive applications.Image and audio/video data are also used, especially for recognition and classification, while multimodal combinations appear less consistently.
  • Summary: The review concludes that many studies lack concrete implementations, explicit data descriptions, or multimodal perspectives, leaving room for practical experimentation in high-risk settings.Human-centered AI and interactive machine learning provide more opportunities for concrete interaction, including conversational, immersive, and parameter-based approaches.

4.3. AI Goals

The review identifies user experience, transparency, interactivity, and active algorithm modification as central AI goals, but finds that most systems remain opaque and provide limited user control. Active modification is especially rare, while interaction commonly supports task completion, feedback, or restricted adaptation rather than access to AI mechanics.

  • 4.3.1. User Experience: User experience is a central goal, with 39 (72%) studies advocating improved user experience with AI systems.The studies commonly connect user experience with system acceptability and collaborative interaction.
  • 4.3.2. Transparency and XAI: Around 11% of studies appear to use transparent AI, while most remaining studies do not specify system transparency.The review reports widespread opaque implementations despite substantial explainable-AI research.
  • 4.3.3. Interactivity: Interactivity is prevalent, but most studies use it for task completion, automated interaction, or feedback rather than interaction with underlying AI systems.Interactive approaches can include rapid feedback and parameter-based adaptation, but their domain coverage remains limited.
  • 4.3.4. Algorithm Modification: Only 5% of studies allow active user modification at the model or algorithm level.Reported barriers include limited practical implementations, insufficient user expertise, task stakes, implementation hurdles, and potential human bias.
  • 4.3. AI Goals: The review classifies AI-interaction goals as improving user experience, increasing transparency, enabling interactivity, and allowing active algorithm modification.Its synthesis calls for more user-inclusive, transparent, and user-controlled AI assistance while emphasizing the balance between autonomy and user control.

5. Discussion and Directions

The discussion frames interactive AI around balancing autonomy, user agency, and user needs through collaboration, augmentation, and feedback. It argues that human-AI systems should complement human expertise while giving users meaningful control.

  • Purpose of Interactive AI: Interactivity seeks to build AI around user needs while allowing systems to automate substantial portions of the process.
  • Collaboration: Human-AI collaboration divides tasks according to respective strengths, with each partner complementing the other rather than AI simply replacing human work.
  • Collaboration: AI systems can use human interaction to gain knowledge, gather performance feedback, and potentially correct wrong decisions.
  • Augmentation: Current augmentation research emphasizes alleviating human task difficulties, but balancing human input and system assistance remains essential.

5.2. Interactive AI in Practice

Interactive AI practice is moving toward user-centered feedback and user-needs-based design, but implementation remains uneven. The literature reports a trade-off in which high-stakes tasks generally have less interaction, while many systems remain theoretical or concentrated in assistive applications.

  • User feedback often remains stored for future expert-led improvement instead of directly changing the AI system.
  • Interactive design should incorporate feedback so users can contribute to system improvement.
  • Recent approaches emphasize identifying user needs and user expertise, while interactive AI shows potential to reduce acceptance barriers and augment users or systems.
  • Some implementation proposals remain theoretical and do not address the technicalities required for practical interaction.
  • High-stakes tasks generally have lower interactivity, while many applications focus on assistive tasks and interactive AI assistants.

5.3. AI and User Agency

The discussion presents interactivity as a route to greater user agency in systems traditionally optimized for autonomous control. It also notes that active user alterations can increase the likelihood of using an imperfect algorithm, while the balance between autonomy and control remains unresolved.

  • Interactivity enables users to exercise agency through input, feedback, prompt correction, and active participation in AI systems.
  • The balance between AI autonomy and user control remains an open consideration for interactive system design.
  • Allowing users to correct an algorithm’s output, even slightly, may increase their likelihood of using an imperfect algorithm.

5.4. Interfacing AI

AI interfaces span conversational, graphical, sensory, and AR/VR forms, with interaction organized around instructing, conversing, exploring, and manipulating. Richer exploratory and manipulative interfaces remain less explored because they require greater implementation effort.

  • AI interaction can use textual, graphical, sensory, and AR/VR interfaces, extending beyond conventional screen-based interaction.
  • Interfaces are categorized into instructing, conversing, exploring, and manipulating dimensions.
  • Conversing: Conversational interfaces include text assistants, robot interaction, and other systems built around communicating through conversation.
  • Exploring and manipulating: Exploring and manipulating interfaces are less explored because they provide richer interaction and require more implementation effort.
  • Practical implementation is necessary to realize interface capabilities, although theoretical contributions can provide foundations for experimentation.

5.5. Addressing HCAI challenges: current trends and gaps

Current HCAI research is moving beyond explanation toward feedback, adaptation, personalization, and broader user interaction. However, interaction remains constrained by transparency challenges, limited non-expert accessibility, and low-stakes applications.

  • Current trends: Human-centered AI research increasingly emphasizes transparency, explainability, feedback, adaptation, personalization, and customization to support user acceptance and trust.These approaches seek to account for human factors and users’ diverse needs, including those of non-experts.
  • Current trends: Interactivity spans helping users understand, control, contest, provide feedback to, or modify AI systems while completing tasks and improving the system.The most common form allows users to complete tasks while simultaneously improving the AI system.
  • Design tensions: User studies are central to designing interaction that augments users or AI systems, but excessive interactivity can threaten trust.The analysis also identifies algorithmic transparency and explanations as continuing concerns in interactive systems.
  • Current gaps: Modification of underlying AI models remains largely restricted to educational or experimental applications involving low-stakes tasks.The analysis suggests that designers should empower users while addressing transparency and explanation requirements.

5.6. Limitations and Future Directions

The review’s conclusions are constrained by how interactivity is defined and evaluated, by the scale and changing nature of the literature, and by transparency limitations in practical AI applications. It calls for more evidence on balanced user agency and control over AI adaptation.

  • Limitations: Including theoretical solutions without actual user involvement means the findings require more caution than user-centered studies with participants.The review notes that user involvement is central to interactivity, yet some included studies do not test it directly.
  • Limitations: The field’s diversity and rapid change limit the scale and completeness of extracting relevant human-AI interaction studies.The review also limits its temporal scope to literature available up to 2023.
  • Future directions: The review focuses on intentional, explicit, and informed user interactions, while acknowledging this as an idealized human-centered scenario.It identifies balanced control over AI adaptation as an important direction and calls for evidence on balancing autonomy with user agency.
  • Future directions: Practical AI implementations remain opaque, and explainability needs vary by users’ roles, creating challenges for evaluation without defined XAI benchmarks.Different stakeholders may require substantially different explanations from the same system.

6. Conclusion

The review finds that human-AI interaction research increasingly recognizes human influence but still rarely provides substantial user control or agency beyond explanation. It therefore highlights feedback, co-creation, and adaptation of AI mechanics as priorities for future work.

  • Conclusion: The review analyzes patterns, gaps, and opportunities across AI users, implementations, and goals, with multiple sub-dimensions.Its primary contribution is identifying the need to prioritize the user’s role in the AI loop beyond explanations.
  • Conclusion: Recent research partially incorporates human influence, but practical user-centered interaction and control remain under-explored beyond largely theoretical calls for greater participation.A nascent group of studies allows AI to adapt to user adjustments and feedback, although concrete solutions remain limited.
  • Conclusion: Interactive AI research commonly focuses on assistants that teach or simplify tasks for humans rather than enabling deeper user control.This pattern indicates that interaction often improves task experience without granting users substantial influence over AI mechanisms.
  • Conclusion: Only a few studies let users modify algorithm design parameters, leaving agency over AI decision-making very limited beyond explainability.Some studies support bidirectional feedback and awareness of internal mechanisms, but fewer provide complete agency.
Loading 2405.15051v1…