Source-linked AI summary
How Much Can AI Understand? Toward AI-Assisted Sensemaking of Collaborative Discussion in Groups with Shared History
Soobin Cho, Mark Zachry, David W. McDonald
TL;DR
AI discussion-support tools often overlook the social context of groups with shared history, although that context shapes how complex discussions should be understood. Drawing on two studies of experienced Wikipedia editors, the paper proposes a model that represents arguments, norms, participants, and their contexts while supporting early sensemaking stages with varying interpretive work. Higher system interpretation reduces users’ work but increases reliance on system judgment, creating intelligibility and misrepresentation risks that require safeguards.
Problem
Discussion-support AI typically treats discussions as standalone content, overlooking the social context of groups with shared history.
Method
The paper combines two studies of experienced Wikipedia editors with a model extending human sensemaking through AI support for early stages.
Results
The model captures six information types and defines AI support functions whose interpretive work ranges from low to high.
Takeaways & Limitations
More interpretive outputs reduce users’ sensemaking burden but require greater reliance on system-generated judgments.
Takeaways & Limitations
Whether an AI system can make its interpretive work intelligible in complex social settings remains an open question.
Abstract
from arXiv · showhide
AI tools that support collaborative discussion typically treat the discussion as a standalone task, focusing only on its content and setting aside the social context of the group having it. But it is groups with a shared history, with their own norms, hierarchies, and relationships, where the most tangled and complex discussions tend to arise. These discussions cannot be understood apart from that context, and AI that overlooks it risks failing to convey what a discussion means, or even misrepresenting it. Drawing on two studies of how experienced Wikipedia editors read and make sense of discussions, we propose an AI-Assisted Sensemaking Model for Collaborative Discussions, which captures not only a discussion's arguments but also the norms and participants behind it, along with the context that gives each meaning. In this model, the system supports the early stages of the sensemaking process, and the degree to which it performs interpretive work can range from low to high. We argue that higher interpretive work reduces the burden on users but increases their reliance on the system's judgment. We then discuss the risks of an insufficiently intelligible system, what it would take to make one more intelligible, and the safeguards it still requires.
1 Motivation
Existing discussion-support AI generally analyzes discussion content without the social context of groups with shared history. This matters because complex discussions reflect norms, hierarchies, relationships, and changing participation.
- Collaborative discussions help groups understand positions, work through disagreement, and reach decisions, but can become complex as practices and participants change.
- Existing AI discussion-support systems commonly focus on content through argument visualization, decision support, argument mining, facilitation bots, and consensus generation.
- Groups with shared history bring norms, hierarchies, experience differences, and relationships that shape how discussion contributions should be understood.
- Ignoring social context risks failing to convey contributions’ meaning or misrepresenting them.
- The paper asks how collaborative groups with shared history make sense of discussions and how AI can support that process.
2 AI-Assisted Sensemaking Model for Collaborative Discussions
The paper develops an AI-assisted sensemaking model from studies of experienced Wikipedia editors. The model represents arguments, norms, participants, and contextual information while adding graduated AI support to early human sensemaking stages.
- Evidence base: Two studies of experienced Wikipedians—unassisted reading and use of an LLM-assisted prototype—informed the proposed AI-assisted sensemaking model.
- Information types: The model identifies six information types: argument, community norm, norm context, people, people context, and discussion topic.
- Information types: Norm context and people context come from the broader community, while discussion topic provides domain-level information outside the community.
- Sensemaking process: Human sensemaking follows Search & Filter, Read & Extract, Schematize, Build Case, and Tell Story, with feedback able to prompt earlier-stage re-evaluation.
- System support: The AI overlays early human stages through Extraction, Prioritization, Connecting, and Presentation, with Prioritization and Connecting able to iterate.
- Interpretive work: System interpretive work ranges from low to high: lightly condensed summaries preserve more source material, whereas selective summaries reshape data around judged importance.
3 Difficulty of Interpreting Discussion Correctly, and What It Means for System Design
Correctly interpreting collaborative discussion is difficult because meaning depends on subjectivity, irrationality, emotion, and social context. System design must therefore balance interpretive assistance with intelligibility, transparency, and human control.
- 3 Difficulty of Interpreting Discussion Correctly, and What It Means for System Design: Interpretive work can be incorrect, including when an AI misinterprets participants’ arguments during discussion sensemaking.The paper distinguishes the system’s interpretive work from the interpretation humans ultimately reach.
- 3.1 Interpreting Human: Human behavior is difficult to interpret because subjective meaning, irrationality, and emotion shape social action.Emotion influences how actors construct intended meaning and can contribute to irrational action.
- 3.2 Implications for the System’s Interpretive Work: Higher system interpretive work reduces users’ interpretive burden but increases their reliance on system-generated judgments.More selective outputs can reduce the time users spend finding what matters while requiring greater trust in the system.
- 3.2.1 The Risks of a System That Falls Short of Intelligibility.: If the system misjudges what matters, it can dehumanize participants by recognizing contributions according to system legibility rather than substantive importance.The paper warns that this systematic loss of context can steer discussion incorrectly and contribute to poor decision making.
- 3.2.1 The Risks of a System That Falls Short of Intelligibility.: Selective surfacing can create false consensus by erasing minority opinions and nuance from the discussion.The resulting apparent agreement may lack genuine buy-in, allowing disagreement to resurface later as conflict.
- 3.2.2 Designing for Intelligibility.: An intelligible system must represent arguments together with norms, norm context, participants, and people context.People context includes members’ experience, background, role, relationships, and character; safeguards still include human final interpretation and access to original data.
4 Toward a Design Agenda
The paper frames AI-supported group discussion as a design agenda. It focuses on choosing interpretive-work levels, improving system intelligibility, and retaining safeguards.
- 4 Toward a Design Agenda: Designing AI tools for group discussion requires deciding interpretive-work levels, making systems more intelligible, and building in safeguards.The agenda is motivated by AI’s growing role in tools such as meeting-minute agents.