PaperlayerView exact revision on arXiv

Source-linked AI summary

VISAR: A Human-AI Argumentative Writing Assistant with Visual Programming and Rapid Draft Prototyping

Zheng Zhang, Jie Gao, Ranjodh Singh Dhaliwal, Toby Jia-Jun Li

arXiv:2304.07810v2cs.HCcs.AIcs.CLcs.LG

TL;DR

Argumentative writing requires writers to coordinate hierarchical goals, persuasive reasoning, and iterative plan revision, while chat-based LLM interfaces can provide limited context, control, and planning support. VISAR addresses these gaps with visual programming, synchronized text and visual editing, human-in-the-loop recommendations, and rapid draft prototyping. A 12-participant study found VISAR useful, usable, and effective for creating and evaluating argumentative writing plans, although its current prompts, argument types, and generated drafts remain technically constrained.

  • Problem

    Argumentative writers must manage hierarchical goals and iterative planning, while chat interfaces can provide limited user control, context, and support for revising writing plans.

  • Method

    VISAR combines visual programming, synchronized text and visual editors, human-in-the-loop prompt chains, and rapid draft prototyping for argumentative planning.

  • Results

    A controlled study with 12 participants found VISAR useful, usable, and effective for creating argumentative writing plans and evaluating outlines.

  • Takeaways & Limitations

    VISAR supports user-controlled exploration, organization, and validation of argumentative writing plans through integrated visual and textual workflows.

  • Takeaways & Limitations

    VISAR uses fixed prompt templates, supports limited argument and relation types, and may produce inaccurate, inconsistent, contradictory, or weakly cohesive prototype drafts.

Abstract

from arXiv · show

In argumentative writing, writers must brainstorm hierarchical writing goals, ensure the persuasiveness of their arguments, and revise and organize their plans through drafting. Recent advances in large language models (LLMs) have made interactive text generation through a chat interface (e.g., ChatGPT) possible. However, this approach often neglects implicit writing context and user intent, lacks support for user control and autonomy, and provides limited assistance for sensemaking and revising writing plans. To address these challenges, we introduce VISAR, an AI-enabled writing assistant system designed to help writers brainstorm and revise hierarchical goals within their writing context, organize argument structures through synchronized text editing and visual programming, and enhance persuasiveness with argumentation spark recommendations. VISAR allows users to explore, experiment with, and validate their writing plans using automatic draft prototyping. A controlled lab study confirmed the usability and effectiveness of VISAR in facilitating the argumentative writing planning process.

1 INTRODUCTION

VISAR addresses challenges in argumentative writing by supporting hierarchical, iterative planning and combining visual programming with rapid draft prototyping. It is designed to preserve user control while helping writers explore and revise argument structures, and a controlled lab evaluation found the system usable and effective.

  • Argumentative writers must coordinate hierarchical goals, evidence, reasoning, and counterarguments while developing persuasive claims.
  • VISAR supports hierarchical goal exploration through an LLM, visualized and editable logical relationships, and synchronized text and visual planning interfaces.
  • Rapid draft prototyping lets writers examine concrete implementations of argument structures while treating generated text as mid-fidelity rather than final writing.
  • Existing LLM chat interfaces can generate and revise text but provide limited transparency, user control, and support for revising writing plans.
  • A controlled lab evaluation found that users successfully used VISAR for argumentative writing planning and considered its intelligent features and interaction strategies useful.
  • The paper contributes a human-AI collaboration approach, the VISAR tool, and a within-subjects study with 12 participants validating usability and effectiveness.

2 RELATED WORK

Prior writing-support research spans language correction, style assistance, text generation, creative writing, and targeted argumentative support. VISAR complements this work by focusing on user-controlled collaboration during argumentative prewriting and planning through visual programming and rapid prototyping.

  • Commercial writing tools primarily address grammar, spelling, style, rewriting, outlines, or generated text rather than the full argumentative planning process.
  • Academic writing-support systems have explored creative ideation, next-step suggestions, personalized rewriting, plot visualization, and other forms of human-AI collaboration.
  • Argumentative writing tools have supported paragraph revision, argument-quality feedback, and related targeted parts of the writing process.
  • VISAR focuses on interaction strategies and collaborative workflows for the prewriting and planning stages of argumentative writing.
  • AI-supported creative work remains constrained by challenges involving originality, authenticity, factual reliability, and authorship.
  • The system uses a human-in-the-loop prompt chain that combines language-model reasoning with human expertise for iterative argumentative writing.

3 BACKGROUND OF ARGUMENTATIVE WRITING

Argumentative writing presents and supports positions on controversial issues through evidence and reasoning, requiring structured arguments and attention to counterarguments. VISAR focuses on deductive, top-down writing and supports prewriting, planning, and iterative outline review.

  • Argumentative writing seeks to persuade readers by presenting positions supported with evidence and reasoning.
  • Toulmin’s model describes arguments through claims, data, warrants, backing, qualifiers, and rebuttals.
  • VISAR uses Toulmin-inspired support to surface counterarguments, provide supporting evidence, and examine logical fallacies.
  • Argumentative writing includes deductive and inductive reasoning, with deductive argumentation proceeding from general principles toward specific conclusions.
  • VISAR focuses on deductive, top-down argumentative writing, which the paper describes as common in written communication.
  • The writing process includes prewriting, planning, drafting, and revising, while VISAR primarily assists prewriting and planning and adds rapid draft prototypes for iterative review.

4.1 Design Goals

VISAR’s design goals address the hierarchical, non-linear, and iterative nature of argumentative planning. The system supports goal construction, plan ideation, persuasive argumentation, and reflection through rapid draft prototyping.

  • VISAR is designed to support hierarchical and iterative planning based on a literature review of argumentative writing processes and writing-support tools.
  • DG1: Accommodating the non-linear and hierarchical nature of writing planning process: Writers need help maintaining coherence when revisions to hierarchical writing goals cascade through subordinate subgoals.
  • DG1: Accommodating the non-linear and hierarchical nature of writing planning process: VISAR should help writers construct hierarchical goal sequences while maintaining overall coherence during revisions.
  • DG2: Facilitating the ideation of writing plans: Writing plans may begin without clear endpoints or complete components, creating a need for support in retrieving and organizing relevant ideas.
  • DG3: Scaffolding to enhance persuasiveness of users’ argumentation: Argumentative planning requires support for persuasive strategies, logical-fallacy awareness, and context-specific counterarguments.
  • DG4: Facilitating users’ reflection and assessment of plans: Rapid draft prototyping helps writers identify incomplete, weak, or unclear parts of outlines and revise plans through concrete draft implementations.

4.2 Example scenario

In the example scenario, Alice uses VISAR to develop an argumentative essay plan, prototype drafts, inspect persuasive weaknesses, and manipulate a synchronized visual outline.

  • Goal planning: Alice begins with an argumentative statement and receives step-by-step recommendations for key aspects and discussion points.She selects breadth of knowledge, well-rounded education, career preparation, and critical thinking, then explores lifelong learning and adaptability.
  • Draft prototyping: VISAR generates prototype drafts for each outline goal, supports alternative drafts, and enables conversational refinement.Alice directly edits generated drafts, requests alternatives for lifelong learning and personal growth, and provides further instructions through conversation.
  • Argumentative Sparks: Argumentative Sparks provides likely counterarguments, potential logical fallacies, and supporting-evidence suggestions for selected draft blocks.VISAR also shows draft implementations of counterarguments and evidence, plus revised discussion-point drafts addressing logical weaknesses.
  • Visual planning: VISAR automatically synchronizes a visual argument outline with the text editor, allowing Alice to add, edit, and rearrange goals through drag and drop.She can create a new key-aspect node, connect it to the main argument, and view the updated outline as she modifies it.

4.3 Key Features

VISAR supports hierarchical goal planning, synchronized text and visual organization, argumentative feedback, and rapid draft prototyping while retaining user direction over planning and revision.

  • Hierarchical writing goal recommendation: VISAR recommends writing goals step by step, lets writers direct generation at each level, and integrates user-defined goals with AI-generated goals.The process follows brainstorming from broad topics toward specific discussion points while promoting user autonomy.
  • Synchronized text and visual planning: VISAR links text-based and visual-based planning, combining direct text editing with visual programming for organizing argument structures.Writers can transfer selected text into the visual graph and manipulate the corresponding representation.
  • Synchronized text and visual planning: Writers can organize arguments in an editable hierarchical tree whose nodes and edges represent components and logical relationships.Supported node types include main argument, key aspect, discussion point, counterargument, and supporting evidence; edge types encode relations such as elaboration, attack, and support.
  • Argumentative sparks: Argumentative sparks recommend counterarguments, identify logical fallacies, and suggest supporting evidence for selected content.Writers can select generated counterarguments, inspect explanations of possible fallacies, and explore evidence types based on ethos, pathos, logos, and example.
  • Automatic draft prototyping: VISAR supports flexible draft-generation timing, recursively updates dependent components, and offers direct editing, alternatives, and instruction-based refinement.These features let writers prototype during planning, maintain consistency among interdependent nodes, and iteratively improve drafts.

4.4 Implementation

VISAR uses a React and Flask web application with GPT-3.5-based prompts to recommend goals, generate argumentative sparks, and prototype coherent drafts.

  • System implementation: The web application uses React with Lexical for hierarchical text content, Flask for server communication, and MongoDB for database access.Custom editor nodes represent key aspects, discussion points, counterarguments, and evidence.
  • Goal recommendation: VISAR uses OpenAI’s GPT-3.5-turbo with role prompts, examples, and structured requests to recommend high-level aspects and discussion points.Separate prompts elicit aspects supporting an argument and discussion points supporting a selected aspect.
  • Argumentative sparks: Argumentative sparks are generated with GPT-3.5 using few-shot examples grounded in argumentation-theory strategies and taxonomies.The system prompts the model to generate counterarguments, identify logical fallacies, and propose supporting evidence.
  • Draft generation: Prototype drafts include parent-node content as context to improve coherence between each writing component and its parent.Prompt templates cover component generation, revision, and refinement.

5 USER STUDY

A 12-participant lab study evaluated whether VISAR supported argumentative-writing planning, its usefulness across planning activities, and users’ encountered challenges.

  • Study design: The lab study included 12 participants and evaluated VISAR’s usability, effectiveness, and usefulness.The study addressed successful planning, support for ideation, organization, and revision, and user challenges.
  • Participants: Participants were recruited from a private R1 university and included undergraduates and graduate students with intermediate, advanced, or expert writing experience.The group comprised 1 sophomore, 3 juniors, 4 seniors, and 4 graduate students; 4 were intermediate, 7 advanced, and 1 expert writers.
  • Prior experience: All 12 participants had previously used generative AI tools, while their prior use of writing-support tools varied across grammar, ideation, refinement, polishing, and plagiarism detection.Eight had used AI for grammar and spelling, seven for prompts or ideas, six for polishing content or language, and one for plagiarism detection.
  • Writing challenges: Participants most commonly reported difficulties organizing ideas and information logically, followed by generating ideas, making arguments persuasive, and developing counterarguments.The reported counts were 9 for organization, 7 for idea generation, 4 for persuasiveness, and 3 for counterarguments.

5.2 Study Design

The study compared VISAR with a plain editor and GPT Playground in within-subjects argumentative-writing planning sessions using GRE Issue Essay topics.

  • Writing task: The study used GRE Issue Essay tasks designed to require critical thinking and argumentation across topics approachable from multiple perspectives.Three topics were selected from the GRE issue-writing sample pool without requiring expert knowledge.
  • Experimental conditions: The three conditions were a plain text editor, GPT Playground with GPT-3.5-turbo, and full VISAR.VISAR included visual programming, step-by-step ideation support, argumentative sparks, and rapid draft prototyping.
  • Experimental design: A Latin square design balanced the sequence of topics and conditions across study sessions.

5.3 Result

Participants generally found VISAR helpful for generating, organizing, and validating argumentative outlines, while its visual-text coordination and draft prototypes supported iterative planning. However, generated drafts and some interface features remained limited, and expert ratings found VISAR drafts below excellence on logic and organization.

  • Post-study questionnaire: Participants rated VISAR more effective than GPT Playground and baseline for generating argumentative elements, organizing outlines, and validating plans.They also reported confidence in VISAR’s outline quality and found it easy to learn and enjoyable to use.
  • Hierarchical ideation: VISAR’s hierarchical recommendations helped participants discover missing ideas and develop more structured outlines.Participants described exploring many possible directions through a few clicks.
  • Synchronized views: Synchronized visual and text editors supported complementary planning and reviewing activities.Participants used the visual editor to arrange hierarchical structure and the text editor to review and modify concrete discussion-point drafts.
  • Argumentative sparks: Argumentative sparks helped participants identify counterarguments, evidence, and potential logical fallacies to strengthen persuasiveness and articulate ideas.Participants also reported that these recommendations supported objectivity and clearer expression of ideas.
  • Draft prototyping: Rapid draft prototypes helped participants inspect plans in concrete contexts, expand outlines, and save effort, but generated drafts had robotic, repetitive, poorly transitioned, and monotonous language.Participants viewed prototypes as useful for planning rather than suitable final products.
  • Challenges and suggestions: Participants reported usability and cognitive-load challenges, including small connection handles, problematic node layout, a steeper learning curve, and the need to read long drafts for alignment.They also suggested visual mind maps, outline-level sparks, evidence citations, and more intuitive reordering controls.
  • Qualitative draft analysis: Baseline drafts were generally clear and coherent but lacked details, whereas GPT Playground drafts contained many details but could be poorly structured or disjointed.These qualitative contrasts show different trade-offs between structure and detail across conditions.

6 DISCUSSION

VISAR combines direct manipulation, conversational interaction, visual programming, and rapid draft prototyping to support context-aware, iterative argumentative-writing planning while preserving user control. The discussion also identifies ethical and cognitive risks of AI-generated prototypes that require continued human oversight.

  • 6.1 Interaction Modalities and Strategies for Collaborating with LLMs: VISAR combines direct manipulation and conversational interaction across modalities, matching interaction flexibility to users’ task needs in a unified workspace.Direct manipulation supports intuitive, context-aware interaction, while conversational interfaces support expressive draft-refinement instructions.
  • 6.2 Generative AI for Rapid Prototyping in Creative Tasks: Rapidly generated draft prototypes make argument plans tangible, helping writers compare alternatives, clarify communication, and iteratively improve their outlines.Writers can create parallel visual plans, generate concrete drafts, and examine their strengths and weaknesses.
  • 6.2 Generative AI for Rapid Prototyping in Creative Tasks: Prototype generation can increase review effort and constrain creative exploration when users must inspect machine-generated content or encounter biased suggestions.Participants reported reading lengthy drafts to check alignment with their intent, while the discussion identifies risks to broad, unconstrained ideation.
  • 6.1 Interaction Modalities and Strategies for Collaborating with LLMs: Visual programming lets writers define hierarchical goals and relationships while the model follows the proposed logical sequence during planning.The interface represents arguments, discussion points, evidence, and their relationships visually, supporting top-down organization.
  • 6.3 Ethical Considerations: VISAR limits generative AI primarily to prewriting and planning prototypes and emphasizes human control to mitigate bias, stereotypes, misinformation, and reproduction of existing sources.The paper frames generated drafts as tools for sensemaking and comprehension rather than finished writing, while calling for transparency and critical assessment.

7 LIMITATION & FUTURE WORK

VISAR’s current limitations include constrained argument structures, imperfect draft cohesion, potentially unreliable model outputs, and a study evaluated mainly through GRE-style tasks. Future work targets broader writing contexts, user groups, and collaborative planning.

  • Technical Constraints: VISAR currently supports only limited argument and relation types and represents argumentation as top-down trees rather than more complex directed graphs.The system lacks support for components such as warrants and qualifiers and for bottom-up deductive argumentation.
  • Technical Constraints: Fixed prompt templates may produce similar responses across distinct contexts, while missing transitions between generated discussion points can reduce prototype-draft cohesiveness.These constraints affect drafting, recommendation, and argumentative-sparks generation.
  • Technical Constraints: GPT 3.5 may generate factual inaccuracies, inconsistent responses, or contradictory prototype statements that can expose users to confusing or misleading information.The system therefore treats generated drafts as prototypes rather than authoritative final content.
  • Study Design: The evaluation used GRE argumentative-writing topics, so deployment studies should examine VISAR in more diverse real-world writing tasks and participant populations.The authors identify task choice and participant demographics as potential threats to validity.
  • Future Work: VISAR is not designed for a particular user group, and future work plans educational applications and collaborative writing-planning spaces.Planned studies include K-12 argumentative writing and shared visual programming spaces for multiple users.

8 CONCLUSION

VISAR supports argumentative-writing prewriting and planning through hierarchical recommendations, synchronized text and visual editors, argumentative sparks, and rapid draft prototyping. A 12-participant user study found that writers could use it effectively and perceived it as useful, usable, and likable.

  • 8 CONCLUSION: VISAR supports prewriting and planning with hierarchical goal recommendations, synchronized text and visual editors, argumentative sparks, and rapid draft prototyping.These features address ideation, outline organization, supporting evidence, counterarguments, logical fallacies, and reflection on draft implementations.
  • 8 CONCLUSION: A user study with 12 participants demonstrated that writers could effectively create argumentative-writing plans using VISAR.The system was perceived as useful, usable, and likable.

Task Prompt template Few-shot examples

The examples illustrate prompt templates that guide VISAR in developing argumentative content from multiple perspectives. They cover ideation, counterarguments, logical weaknesses, supporting evidence, and paragraph elaboration.

  • Ideation: VISAR prompts users to identify key aspects and discussion points for developing an argument.The templates ask for aspects worth discussing and points from a specified aspect.
  • Critical examination: VISAR generates counterarguments and identifies potential logical weaknesses in argumentative statements.Examples include counterargument generation and prompts targeting logical weaknesses and fallacies.
  • Logical weaknesses: The examples identify fallacies such as slippery slope, false analogy, false dilemma, and unsupported causal reasoning.The supplied examples explain how these weaknesses arise in statements about punishment, building safety, and student circumstances.
  • Supporting evidence: VISAR prompts writers to consider ethos, pathos, logos, and concrete examples as supporting evidence types.The supporting-evidence template explicitly organizes evidence by professional experience, audience emotion, facts and reasoning, and practical examples.
  • Draft elaboration: VISAR also prompts generation of topic sentences and paragraphs that elaborate arguments from selected perspectives and discussion points.These templates connect a selected argument with a key aspect or discussion point for paragraph development.
  • Example drafts: The examples include extended argumentative drafts on government research funding and lucrative career preparation.These drafts present claims followed by supporting reasoning and examples.
Loading 2304.07810v2…