DocuTeam: Mixed-Initiative Multi-Agent Discussions around Evolving Documents

1 Introduction

• The design addresses the challenge of helping users coordinate multiple autonomous discussions and soliciting their input when progress depends on unstated preferences or constraints.The paper positions this as a mixed-initiative approach to team collaboration around evolving workflows.Source: §1 · pp. 2–3
• Workshop participants wanted discussion to unfold as an ongoing exchange, with ways to follow its flow, steer it, and selectively bring useful ideas into their work.These needs informed four design implications: task-oriented modes, discussions shown alongside relevant work with summaries, direct and indirect steering, and selective incorporation.Source: §1 · p. 3
• DocuTeam lets users and agents initiate and steer discussions grounded in document regions, while agents attend to edits and can start conversations proactively.Users can redirect threads, answer clarification questions when intentions or constraints are needed, and adapt useful ideas into the document.Source: §1 · p. 3
• The study compared DocuTeam with a side-panel baseline requiring users to initiate, ground, and steer discussions across two constrained event-planning tasks.The within-subjects study involved 20 participants.Source: §1 · p. 3
• Plans created with DocuTeam were rated significantly more novel, relevant, and specific, while overall cognitive load did not significantly differ from baseline.Participants described effort shifting: they processed additional information but spent less effort orchestrating discussion and kept more attention on the document.Source: §1 · p. 3
• Participants sent 46% fewer steering messages and made 39% more document modifications than in the baseline condition.The collaboration shifted from chat-centered idea sourcing toward document-centered iterative refinement.Source: §1 · p. 3

2.1 Collaborative Cognition through Discussions

• The benefits of diverse perspectives depend on teammates coordinating them through discussion; around shared artifacts, teams monitor contributions, surface concerns, coordinate actions, and ground subsequent communication.Source: §2.1 · p. 4
• The work investigates interfaces that make discussions among multiple AI participants a collaborative resource for users’ ongoing document work, while supporting awareness of their activity and grounding exchanges in the evolving artifact.Source: §2.1 · p. 4

2.2 Human–Multi-Agent Collaboration through Discussion

• Existing multi-agent discussion systems often require users to initiate and manage exchanges [25] [62] [79], and many remain detached from working artifacts.The paper investigates agents that autonomously initiate and sustain discussions integrated into users’ documents and workflows.Source: §2.2 · p. 5

2.3 Interfaces for Mixed-Initiative Human-AI Collaboration

• Prior interfaces for mixed-initiative assistance largely address single agents, leaving open how autonomous multi-agent discussions can fit into task environments without added coordination burdens.DocuTeam enables agents to proactively initiate discussions grounded in evolving documents; the comparison is framed around initiative, discussion development, and grounding.Source: §2.3 · p. 6

3.1 Methods

• The online workshop involved 15 frequent AI users with interface-design experience, who discussed familiar problem-solving scenarios in 90-minute sessions.Source: §3.1 · p. 6

3.2 Findings

• Participants envisioned discussion as an ongoing flow that supports work as documents and user needs evolve, while remaining easy to follow, steer, and act on.Source: §3.2 · p. 7

3.2.1 What support do users expect from multi-agent discussion across the overall task flow?

• Participants expected agents to broaden options, examine trade-offs, and critique or validate outputs, with all three support types relevant throughout the task flow.Emphasis varied by stage: refining goals and constraints, exploring and fact-checking information, comparing alternatives during solution building, and using distinct personas or criteria in final review.Source: §3.2.1 · p. 7

3.2.2 What interaction challenges arise from mixed-initiative multi-agent discussion?

• Users wanted discussions to continue in the background without obstructing document work, while offering peripheral awareness and summaries that could be inspected on demand.Suggested cues included showing the discussion’s focus or current speaker and notifying users when important changes occurred.Source: §3.2.2 · pp. 7–8
• Participants wanted to steer discussion at different effort levels, from joining directly to indirectly shaping its focus, participants, or emphasis.Ideas included like/dislike controls, adding or excluding agents, and pointing to document sections to emphasize.Source: §3.2.2 · p. 8
• Users wanted to review multiple ideas, preview their effects in the working document, and selectively apply preferred suggestions rather than receive one final answer.Source: §3.2.2 · p. 8

3.3 Design Implications

• The design implications call for task-oriented discussion modes that support expanding possibilities, deliberating trade-offs, and evaluating current documents.Source: §3.3 · p. 9
• Interfaces should surface the focus, flow, and disagreements of ongoing discussions without requiring users to monitor them continuously.Source: §3.3 · p. 9
• Concise summaries should help users grasp how discussion is developing and where agents agree or disagree.Source: §3.3 · p. 9
• Interfaces should bring emerging issues to the user’s attention when needed.Source: §3.3 · p. 9
• Participants wanted to intervene when discussions drifted from their intentions.Source: §3.3 · p. 9
• Because participants varied in engagement, interfaces should support direct participation and lighter-weight ways to steer discussion.Source: §3.3 · p. 9
• Since discussion produced competing ideas and evolving versions, interfaces should make alternatives easy to inspect.Source: §3.3 · p. 9
• Interfaces should make ideas emerging from discussion easy to selectively incorporate into users’ ongoing work.Source: §3.3 · p. 9

4 DocuTeam: Mixed-initiative Multi-Agent Discussion on Documents

• DocuTeam supports user- and agent-initiated discussions and mixed-initiative steering, so both users and agents can shape ongoing conversations as needs emerge.Users can also configure agent profiles and add or remove agents from ongoing threads.Source: §4 · pp. 9–10
• Three task-oriented modes—Idea, Discussion, and Evaluation—align agent support with different work stages, while discussions are anchored to document regions and summarized at varying levels of detail.Users can still open the full discussion.Source: §4 · p. 10

4.1.1 Agent Configuration.

• Users configure agents with distinct roles or viewpoints, then initiate a discussion by assigning agents to a document block and selecting a mode.They can optionally add a query to focus the discussion.Source: §4.1.1 · p. 10, §4.1.2 · p. 10

4.1.3 Agent-Initiated Discussions.

• As users edit, agents attend to recently changed blocks, assess whether ideation or discussion is needed, and can autonomously start a discussion with a selected topic and mode.The resulting discussion is anchored beside the corresponding block.Source: §4.1.3 · p. 10

4.1.4 Reading and Using Discussions.

• Users can inspect long discussions through mode-specific summaries and move useful snippets or summary items into the document, where they are adapted to its surrounding style.Users can also redirect conversations, add or remove agents, and respond to agent questions.Source: §4.1.4 · p. 11, §4.1.5 · p. 11

4.2.1 Discussion Engine.

• The discussion engine iterates through intent extraction, action selection, utterance generation, and updates to agent memories and the discussion summary.User messages can reactivate closed discussions or override the current loop.Source: §4.2.1 · pp. 11–12
• A moderator uses conversation history, goals, and live document context to choose the next speaker, ask users to clarify preferences, or conclude when the goal is met.Agents retain long-term memories of user preferences, prior decisions, and earlier discussion takeaways.Source: §4.2.1 · p. 11

4.3 Implementation Details

• Implemented as a Chrome extension for Notion, DocuTeam uses gpt-5-mini for utterance generation and gpt-4o-mini for its other LLM components.The system monitors typing with a two-second debounce and filters trivial edits before analyzing updated content.Source: §4.3 · p. 12, §4.2.2 · p. 12

5.1 Study Design

• DocuTeam was compared with a user-initiated multi-agent baseline using the same discussion engine and summaries, across two event-planning tasks with partially written plans and reference materials.Twenty participants completed both conditions in counterbalanced task blocks; two external evaluators blind-rated the 40 final plans on four dimensions.Source: §5.1.1 · p. 12, §5.1.2 · p. 13, §5.1.3 · p. 13

5.2.1 (RQ1) More novel, relevant, and specific outcomes.

• Novelty (DocuTeam = 3.20 ± 0.98, Baseline = 2.68 ± 0.91), Relevance (3.30 ± 1.04, 2.78 ± 0.91), and Specificity (3.25 ± 0.88, 2.28 ± 0.85) were significantly higher; Workability was not.Workability scores were DocuTeam = 3.40 ± 0.81 and Baseline = 3.28 ± 0.72.Source: Figure 3 · p. 14, §5.2.1 · p. 14
• Workability did not significantly differ, as participants in both conditions struggled to complete real-event-ready plans within the study’s time constraints.Source: §5.2.1 · pp. 14–15
• Agent-initiated exchanges exposed users to unanticipated perspectives and surfaced counterarguments, risks, and constraints, helping generate ideas and align drafts with requirements.Discussions anchored to document blocks also supported specialized, section-by-section refinement.Source: §5.2.1 · pp. 14–15
• Participants sometimes felt less certain about outcome quality as agents introduced contradictions and broadened the issues to organize, despite unchanged confidence and satisfaction ratings.Confidence was DocuTeam = 4.35±1.63, Baseline = 4.40±1.47; satisfaction was DocuTeam = 5.15 ± 1.50, Baseline = 5.15 ± 1.23.Source: §5.2.1 · p. 15

5.2.2 (RQ2) From chat-centered ideation to document-mediated discussion and iterative refinement.

• In DocuTeam, the document became a medium for user-agent collaboration, and participants shifted their user-initiated discussions toward Discussion mode.Source: §5.2.2 · p. 15
• 1.53 ± 1.18 vs. 2.8 ± 0.95 user-initiated Idea-mode conversations (t=2.59, p=0.018) were less frequent in DocuTeam than baseline, which relied more on broad ideation.The discussion pattern shifted toward developing and refining content already in the document.Source: §5.2.2 · p. 15
• Proactive agent discussions prompted users to revisit document content, and some participants kept writing to elicit further reactions, making the document both an editing object and interaction medium.Source: §5.2.2 · p. 15
• 3.25 ± 1.55 vs. 4.1 ± 1.74 on whether AI felt like a mere tool rather than collaborator (W=19.0, p=0.060) was not significant, though interviews described DocuTeam agents as more alive and companion-like.One participant found their visible independent activity reassuring, though sometimes distracting.Source: §5.2.2 · pp. 15–16
• 5.7±1.22 vs. 4.6±1.43 (W=49, p=0.025) indicated easier application of AI-generated ideas; DocuTeam participants also modified 448.7 ± 100.95 vs. 322.95 ± 135.33 draft words (t=3.68, p<0.01).Participants’ confidence in and satisfaction with their own outcomes did not differ significantly between conditions (Fig. 4).Source: §5.2.2 · p. 15, Figure 4 · p. 16

5.2.3 (RQ3) Comparable overall workload despite different interaction demands.

• NASA-TLX scores were similar for DocuTeam and Baseline (4.65±0.59 vs. 4.26±0.89; W=47.5, p=0.169), with no significant perceived-workload change; attention to agent discussions was offset by reduced directing effort and tracking scaffolds.Source: §5.2.3 · p. 16
• 2.7 ± 0.77 vs. 5 ± 1.50 steering messages (W=31.0, p=0.013) were sent in DocuTeam versus baseline, as proactive agents grounded in the evolving document reduced the need to direct discussion through chat.Source: §5.2.3 · p. 17

6.1 Toward Mixed-Initiative Multi-Agent Discussion Interfaces

• Participants treated agents more like teammates when they initiated or intervened, while initially experiencing learning costs and information overload before adapting.Source: §6.1 · p. 18

6.2 Future Design Implications for Mixed-Initiative Multi-Agent Discussions on Documents

• Participants wanted summaries that conveyed discussion flow and involvement, such as agent tensions, rather than only text-based outcomes.Source: §6.2 · p. 18
• Agent configuration can prompt reflection on missing perspectives, while automatic agent generation could make forming a discussion team easier.Source: §6.2 · pp. 18–19
• Discussion snippets adapted to document context and style can be treated as reusable, manipulable materials across writing and discussion.Source: §6.2 · p. 19

6.3 Shared Artifacts as Coordination Interfaces for Human–Multi-Agent Collaboration

• A shared document lets user edits signal changing priorities and agent activity remain visible, creating an iterative work–discussion loop that reduces repeated steering.Source: §6.3 · p. 19

6.4 Multi-Agent Discussions and the Value of Productive Friction

• Agent pushback surfaced draft problems and overlooked constraints; although it required more reading and judgment, it helped users produce more specific plans reflecting constraints.Source: §6.4 · p. 19

6.5 Limitations

• The study’s short-term event-planning setting leaves uncertain whether effects extend to other workflows or how collaboration strategies change over longer use.Source: §6.5 · p. 20
• Because both study conditions used the same engine, the study did not independently evaluate conversation quality, which can substantially shape user experience.Source: §6.5 · p. 20

7 Conclusion

• DocuTeam anchors discussions to document regions and supports proactive, mixed-initiative interaction around the evolving document.Source: §7 · p. 20
• The within-subjects study compared DocuTeam with a typical multi-agent discussion baseline and found reduced steering burden alongside document-centered iterative refinement and more novel, relevant, and specific outcomes.Source: §7 · p. 20
• Effective human–multi-agent teamwork depends on team composition, how autonomous activity is surfaced, and how shared artifacts help users understand and steer discussion.Source: §7 · p. 20

• Rows represent stages of work, while columns represent resolving conflicting objectives, validating emerging outcomes, and synthesizing new concepts.Source: Figure 6 · p. 25
• Each cell gives a representative expectation and participant identifiers.Source: Figure 6 · p. 25
• The figure maps expected support across the task rather than focusing on a single stage or support type.Source: Figure 6 · p. 25

A.1 Procedure

• Participants worked in groups of three online for a 90-minute workshop and first received an introduction to multi-agent discussion and prior examples.Source: §A.1 · p. 25
• Participants chose a familiar UI design, report-writing, or house-hunting scenario because these tasks combine creative and analytical thinking with diverse information and requirements.Source: §A.1 · p. 25

A.2 Full stage-by-support examples

• At goal-setting, participants expected help negotiating competing priorities, interpreting ambiguous standards, checking their criteria, and surfacing overlooked goals or standards.Source: §A.2 · p. 26
• During exploration and formulation, participants expected agents to prioritize information, fact-check it as it emerged, introduce diverse perspectives, and probe underexamined areas.Source: §A.2 · p. 26
• At solution collection, participants expected agents to debate options against users’ circumstances, rate solutions against relevant conditions, and propose alternatives to broaden the solution space before convergence.Source: §A.2 · p. 26
• During presentation and final review, participants envisioned criterion-specific feedback, stakeholder simulations to identify issues, and explanations tailored to readers’ expertise.This late-stage support included reframing and reinterpretation as well as evaluation.Source: §A.2 · p. 26

B.1 Study Setting Materials

• The academic workshop scenario asked participants to improve an incomplete draft under a reduced budget of 1,400,000 KRW and varied participant requirements.Source: §B.1 · p. 27
• The sports-day scenario asked participants to supplement an incomplete proposal while balancing interdepartmental harmony with competition and revising its reward structure.Source: §B.1 · p. 28
• Both tasks used evaluation criteria of novelty, workability, relevance, and specificity, and the materials included a summary interface for each discussion mode.Source: Figure 7 · p. 27

B.2 Survey Questions

• Post-survey items asked participants to rate agreement with statements about useful perspectives, future use, plan confidence and satisfaction, engagement, and awareness of AI actions.Items also asked whether ideas were easy to apply and whether the AI felt more like a tool than a collaboration partner.Source: §B.2 · p. 28
• Nine survey items were adapted to assess cognitive process [79], task satisfaction [71], and perceived collaboration.The idea-application item measured perceived translation cost between the chat interface and document workspace.Source: §B.2 · p. 29

C Prompt Details

• The dialogue prompts infer a discussion focus, orchestrate turns, generate agent utterances, update agent and user-specific memories, and produce sidebar and bubble summaries.Turn-level decisions include selecting a speaker, asking the user for clarification, or deciding convergence.Source: C Prompt Details · p. 29
• Goal extraction, workspace-change interpretation, agent selection, and discussion-necessity scoring support proactive discussion triggering, with optional mutters and suppression of redundant neighboring discussions.Together, these prompts instantiate document-grounded, mixed-initiative discussion behavior.Source: C Prompt Details · p. 30
• The prompt appendix also documents prompts for inferring user intent, updating long-term memory from user contributions, and generating mode-specific and single-sentence summaries.The summary prompts produce a structured sidebar summary and an in-situ bubble summary.Source: C Prompt Details · p. 29, Figure 9 · p. 31