Source-linked AI summary
InsightToast: Proactive Information Retrieval & Glanceable Visualization in the Side Channel of Data-Rich Meetings
Mohammad Abolnejadian, Matthew Brehmer
TL;DR
Meetings can be disrupted when participants search for missing institutional context, especially during cognitively demanding decisions. InsightToast monitors conversation, detects information needs, and proactively retrieves source-grounded text and charts through a multi-agent RAG pipeline. In a study of N=16, participants made informed policy decisions while maintaining natural conversation flow.
Problem
Retrieving scattered institutional context during meetings can disrupt individual focus and collective conversational flow, particularly during cognitively demanding decision-making.
Method
InsightToast monitors verbal discourse, detects topics and information needs, and uses multi-agent LLM-based RAG to deliver ephemeral, source-grounded text and interactive charts through a peripheral side channel.
Results
N=16 participants arrived at informed policy decisions while maintaining natural conversation flow with proactively surfaced insights.
Takeaways & Limitations
Proactive, glanceable, source-traceable information delivery can support synchronous collaborative deliberation without requiring participants to initiate searches.
Takeaways & Limitations
Mean synthesis latency was 29 seconds per insight-triggering utterance, and inference cost was approximately half a dollar per fifteen-minute session.
Abstract
from arXiv · showhide
Missing institutional context during meetings can impede effective participation. Retrieving relevant information, often scattered across heterogeneous internal and external sources, requires costly task-switching that disrupts both individual focus and collective conversational flow, particularly detrimental during cognitively demanding tasks such as decision-making. We introduce InsightToast, a mixed-initiative application that monitors verbal discourse in real time, identifies topics and informational needs as they emerge, and proactively retrieves relevant information through a multi-agent large language model (LLM)-based pipeline integrating retrieval-augmented generation (RAG) to produce source-grounded insights as succinct text and glanceable interactive charts, delivered through a peripheral interface as ephemeral toasts in the conversation's side channel. To demonstrate the potential for yielding serendipitous insights, we showcase a usage scenario involving a knowledge base of legislative documents as the meeting's context. We then report on a comparative study (N=16), in which participants arrived at informed policy decisions while maintaining natural conversation flow.
1 Introduction
InsightToast addresses the disruption and keyword burden of retrieving missing meeting context by making information retrieval ambient and proactive. It monitors discourse, detects emerging needs, and delivers source-grounded insights through a peripheral side channel.
- Meeting participants often lack institutional knowledge and relevant data during decisions, planning, or ideation, while retrieving it can disrupt conversational flow.
- Reactive meeting assistants still require conscious initiation, so the paper advocates background retrieval that does not demand deliberate user action.
- InsightToast monitors spoken discourse, detects knowledge gaps and topics, and proactively delivers ephemeral, source-grounded text or interactive charts.
- The system uses a multi-agent LLM pipeline with RAG to formulate complementary searches and synthesize insights from indexed knowledge bases and open-web sources.
- A legislative-policy scenario demonstrates the approach, followed by a mixed-methods evaluation of participants using glanceable insights during collaborative deliberation.
2 Related Work
Prior work frames synchronous data-rich collaboration as an attention, provenance, and decision-support problem. InsightToast combines proactive retrieval with glanceable peripheral presentation to address an identified gap in evaluated support for synchronous data-driven meetings.
- Synchronous collaboration depends on shared contextual information, but searching for missing evidence can impose individual cognitive costs and collective coordination costs.
- Maintaining common ground, analytic provenance, and handoff-ready records competes with the attention required for collaborative analysis and decision-making.
- Proactive retrieval can preserve conversational flow, while peripheral interfaces reduce attentional demands through ambient, glanceable information delivery.
- InsightToast combines context-driven retrieval with glanceable text and visualizations, a pairing the authors describe as not previously prototyped or evaluated for synchronous data-driven meetings.
- Decision-making requires lightweight support because participants must track dialogue, process information, and reason about trade-offs while under-informed members may withhold unshared knowledge.
- The system extends conversational AI support by monitoring live discussion, tracking topics, identifying knowledge gaps, and delivering RAG-grounded insights in the periphery.
3 Design Goals
The design goals organize InsightToast around autonomous retrieval, non-disruptive glanceable delivery, and participant control and trust. Together, they target information support that follows the conversation without forcing participants to leave it.
- The design framework groups twelve design concepts around retrieval, presentation, and participant control and trust.
- DG1 Retrieval: DG1 calls for autonomous monitoring that triggers retrieval upon genuine knowledge gaps and decomposes each need into multiple complementary queries.
- DG2 Presentation: DG2 proposes secondary-region delivery through non-modal, time-limited notifications and concise text or compact visualizations that participants inspect at their own pace.
- DG3 Control and Trust: DG3 requires dual attribution to retrieved sources and triggering conversational moments, while allowing participants to curate information as conversational salience changes.
4 InsightToast
InsightToast implements proactive meeting support through ephemeral side-channel toasts, topic-organized insights, and a sequential multi-agent pipeline. It transforms diarized speech into source-traceable text or interactive visual evidence using meeting and external sources.
- System: InsightToast defines insights as succinct synthesized representations of detected information gaps, delivered proactively in the conversation’s side channel.
- Interface: In the interface, ephemeral toasts show retrieval progress and titles, while an expandable sidebar organizes insights by dynamically detected meeting topics.
- Traceability: The interface links each insight to both its retrieved document sources and the conversational trigger through a secondary transcript panel.
- Insight Presentation: Text insights summarize unstructured evidence with contextual grounding and inline citations, while chart insights compact structured evidence into interactive visualizations.
- Scenario: In the sustainability scenario, a question about global fiscal-gap precedents prompts insights comparing corporate tax-rate trends across countries.
- Participant Control: Participants can browse topic-organized insights, perform reactive searches, and collaboratively archive irrelevant items or pin valuable ones.
- Pipeline: The pipeline processes diarized utterances incrementally by maintaining context, assessing information needs, retrieving from indexed and web sources, and synthesizing insights.
5 Evaluation
InsightToast reduced the need for disruptive manual searching while supporting more informed, broader policy deliberation. Participants valued its proactive, source-grounded side-channel insights, although the volume and novelty of content could also create cognitive disruption.
- T1: Proactive and serendipitous information retrieval results in less task switching.: InsightToast participants reported lower physical demand than baseline (M=1.75 vs. M=2.31; p=.030) and conducted fewer reactive searches.Participants described offloading the hard part of searching and not needing to formulate a question before retrieval occurred.
- T1: Proactive and serendipitous information retrieval results in less task switching.: InsightToast shifted exploration from pause–search–resume cycles toward layered deliberation with proactively synthesized insights.Baseline participants repeatedly halted conversation to formulate queries and scan documents, whereas proactive insights supported ideas building across the session.
- T2: Deliberation broadens and decisions feel more informed.: Participants felt significantly more informed about their decisions with InsightToast (M=4.06 vs. M=2.62; p=.004).They also included significantly more source citations in their justifications (M=2.31 vs. M=1.25; p=.006), while decision confidence did not differ significantly.
- T2: Deliberation broadens and decisions feel more informed.: InsightToast broadened perspectives: participants reported considering more important aspects (M=3.44 vs. M=2.19; p=.003) and encountering information they would not have sought themselves.Participants described unexpected evidence changing or recontextualizing their positions, while emphasizing that the system supported rather than replaced their agency.
- T3: Adapting to proactive meeting intelligence elicits new usage scenarios.: Participants rated InsightToast favorably and valued titled summaries and clickable citations, but reactions to chart-based insights were mixed.Visual learners found charts useful for quickly understanding statistics, while some participants preferred smaller, more digestible text insights.
- T3: Adapting to proactive meeting intelligence elicits new usage scenarios.: Participants were enthusiastic about future meeting use and connected InsightToast to academic, cross-domain, policy, vacation-planning, and organizational knowledge-base scenarios.They particularly framed it as a bridge to institutional knowledge that might otherwise be lost when experienced team members depart.
- T3: Adapting to proactive meeting intelligence elicits new usage scenarios.: InsightToast generated an average of 28.53 insights per session (SD=8.58), while some participants reported hijacked attention, noise, and cognitive load.Participants also requested reactive insight generation alongside proactive breadth, allowing targeted depth on demand.
6 Discussion
The discussion identifies both benefits and boundaries of proactive meeting support. It highlights reduced task switching and promising adaptability while noting notification noise, latency, cost, model risks, and limited evaluation scope.
- Proactive retrieval reduced task switching by delegating query formulation and evidence synthesis and narrowing the separation between communication and information discovery.
- Some participants experienced frequent toast notifications as noise, attributed to occasional pipeline over-triggering and high information density.
- Voice commands, gestures, and inline controls could add on-demand generation and failure reporting, giving participants more control over insight synthesis.
- 29 seconds of mean synthesis latency per insight-triggering utterance may briefly disrupt conversation, while approximately half a dollar per fifteen-minute session may limit adoption.
- LLM-related bias and hallucination persist despite retrieval grounding, although traceable citations and participant disclosure provide safeguards.
- The within-subjects evaluation used one study session and does not capture larger groups, longer deliberations, or multiple subject-matter experts.
7 Conclusion
InsightToast delivers proactive, ephemeral information in meeting participants’ periphery to fill knowledge gaps and ground conversation in data. In an N=16 legislative decision-making study, participants were receptive to surfacing unknown unknowns and serendipitous content.
- InsightToast provides proactive information delivery through ephemeral peripheral toasts containing attributed text statements or glanceable charts.
- The N=16 study contrasted InsightToast with a baseline reactive search interface in a legislative decision-making scenario.
- Participants were receptive to proactive assistance, especially for identifying unknown unknowns and surfacing content that might otherwise have gone amiss.
A Appendix: Pipeline Implementation Details
The appendix documents the proactive retrieval trigger and chart-design criteria used in InsightToast’s pipeline.
- The appendix details the proactive retrieval-trigger and chart-design criteria referenced in Section 4.2.
A.1 Knowledge Gap Detection Criteria
InsightToast triggers retrieval by evaluating each current utterance within recent conversational context and the meeting’s running context. The criteria favor recall over precision to address ambiguity and stalled discussion.
- Retrieval evaluates the current utterance against the recent conversational window and the meeting’s running context.
- At least one defined trigger category must be satisfied, while topical relevance or an isolated hedge word alone is insufficient.
- Trigger categories are disjunctive, but conditions within each category are conjunctive.
- The resulting judgment weights recall over precision because dismissable toasts are treated as lower-cost than missed gaps in high-stakes deliberation.
A.2 Chart Design Constraints and Verification
InsightToast constrains visualization planning through a restricted chart inventory organized by data shape and analytic intent, with validation rules governing chart simplification and verification.
- Fixed generation rules suppress non-essential elements, adding legends or gridlines only when needed and exposing exact values through tooltips.
- The restricted chart inventory organizes available chart types by data shape and analytic intent.
- Secondary chart types require explicit justification when no primary type serves the analytic intent.
B Appendix: Evaluation Task Materials
The appendix documents the materials used to administer the deliberation task described in Section 5.
- The appendix documents materials used to administer the deliberation task.
- These materials correspond to the study’s Section 5 deliberation procedure.
- The appendix serves as documentation for administering the task rather than reporting new evaluation findings.
B.1 Task Briefs
Participants deliberated between competing policy positions using task cards, justification criteria, scripted conversational cues, and materials covering rental housing and healthcare wait times.
- Task structure: Each task framed a citizen advisory brief in which participants recommended one of two competing policy positions.
- Task structure: Participants considered value for public money, evidence of effectiveness, and political feasibility during deliberation.
- Task 1: Rental Housing Affordability: Task 1 addressed rental housing affordability through a national rent increase cap or purpose-built rental construction.
- Task 1: Rental Housing Affordability: The construction position proposed federal tax incentives and direct funding for developers building new rental-only housing, with long-term supply benefits weighed against immediate affordability concerns.
- Task 2: Healthcare Wait Times: Task 2 addressed healthcare wait times through conditional health funding or direct federal hospital staffing and infrastructure investment.
- Task 2: Healthcare Wait Times: The staffing position proposed direct federal funding to hire clinical staff, balancing responses to shortages against concerns about maintaining improvements after funding ends.
- Conversation procedure: A research-team conversation partner used scripted cues across three deliberation phases to sustain rapport and topic focus without steering participants toward either position.
- Conversation procedure: Cues prompted discussion of expert evidence, parliamentary action, voting records, and petitions relevant to the policy arguments.