Beyond Text Generation: Supporting Writers with Continuous Automatic Text Summaries

• Revision is challenging because preparation and repeated editing compete with finishing, growing drafts are harder to oversee, and writers may focus on local edits instead of scope and structure.Writing also requires conveying the writer’s intention while addressing readers’ needs, beyond spelling and grammar.Source: §1 · p. 2
• Higher-level revision support using co-creative AI remained under-explored, motivating research into NLP capabilities beyond text generation and correction.Prior work focused on local revision, such as word replacement and error correction.Source: §1 · p. 2
• The prototype continuously generated paragraph-wise summaries beside the text, automating reverse outlining as a strategy for reflection and structural revision.Source: §1 · p. 2
• Across two studies with 12 participants writing analytic essays, summaries provided an external perspective that supported reflection on paragraph content and scope, revisions, and a quick overview of the text.Participants also developed strategies for integrating insights from the automated summaries into their writing.Source: §1 · p. 2
• The work contributes a text-editor prototype and two-study evaluation, exploring interactive NLP and co-creative AI for writers beyond direct text generation and correction.Source: §1 · p. 2

2.1 What is a Summary and What is its Goal?

• The paper adopts informative summaries that convey a text’s main ideas in less space, at paragraph rather than document level.Its system creates objective summaries to prompt the writer’s own interpretation, and explores both extracts, which reuse source text, and abstracts, which recreate it.Source: §2.1 · p. 2

2.2 Automatic Text Summarization

• Automatic summarization can be extractive, selecting important text, or abstractive, writing a new summary; the prototype uses TextRank and T5 [36], respectively.The paper also notes mixed approaches, such as pointer concepts that copy some text while rewriting other parts [41].Source: §2.2 · p. 2

2.3 Reading and Writing Support Systems

• Prior interactive systems use summaries to adapt website text, describe mobile screens, report document information on request [47], or navigate audio [25].These examples position summarization primarily as reading or content-navigation support.Source: §2.3 · pp. 2–3
• Writing-support tools include text generation, controllable creative writing, word replacement, and error correction; Arnold et al. highlight support without text generation.The cited work motivates the paper’s design direction beyond generating or correcting text.Source: §2.3 · p. 3
• The system explores using AI summarization to support writing as it happens, alongside literature that has mainly applied summaries to reading and text generation to writing.The authors identify this as the remaining combination in the literature.Source: §2.3 · p. 3

2.4 Text Summarization in Writing Research, Practice and Instruction

• Reverse outlining summarizes each drafted paragraph to help writers reflect on structure and decide whether to reorder, split, merge, or delete paragraphs.The paper treats its descriptive AI summaries as a form of self-annotation that can support self-monitoring during writing.Source: §2.4 · p. 3

3.1 Core Concept: AI Text Summarization as (Self-)Annotation

• The concept turns reverse outlining into AI-generated self-annotations, reducing the effort of manual summarization to invite writers’ self-assessment and reflection.Writers still need to read the summaries, but this may give them a “fresh” view of their text, as reverse outlining does.Source: §3.1 · p. 3

3.2 UI: Summaries as Margin Annotations

• Each editor paragraph generates a linked sidebar summary card, and clicking a card highlights its paragraph.The interface pairs a conventional word-processing editor with a margin-style sidebar for annotations.Source: §3.2 · p. 3

3.3.1 Reordering: Move paper notes →Drag & drop cards.

• Paper-inspired cards support note actions such as dragging paragraph summaries vertically to reorder text for structural revision.Each card represents one paragraph, mapping paper notes or snippets to digital interactions.Source: §3.3 · p. 4, §3.3.1 · p. 4

3.3.2 Removing: Throw paper away →Delete card.

• Reverse outlines may reveal redundant content to remove; the interface provides a delete button for each card.Source: §3.3.2 · p. 4

3.3.3 Splitting: Cut/rewrite paper notes →Add line break.

• When a paragraph mixes topics, writers can split it by adding a line break, which automatically updates the sidebar cards.Source: §3.3.3 · p. 4

3.3.4 Merging: Glue/rewrite paper notes →Delete line break or drag & drop cards onto each other.

• Writers can merge paragraphs by deleting the intervening line break, or drag one card onto another to review an automatically generated merge suggestion.The merge suggestion can be accepted or canceled, and an accepted result remains editable in the main text.Source: §3.3.4 · p. 4

3.3.5 Revision: Rewrite/reprint main text →Copy card content.

• For revision, version 1 let writers replace a paragraph with its summary; version 2 changed this to copying card content to the clipboard.Source: §3.3.5 · p. 4

3.4 Annotation Content and Controls

• Writers can choose among summary levels, explored as abstract zoom numbers from one to five or as descriptive options.Multiple options reflect writing guides’ varying recommendations for summary length and type.Source: §3.4 · p. 4

4.1 Frontend / UI

• The React.js interface paired a main text editor with one summary card per paragraph, which users could reorder, delete, or merge.The card interactions implemented the concepts described in Section 3.Source: §4.1 · p. 4

4.1.2 Merge

• Dragging one card onto another triggered a merge suggestion made from the five highest-ranked sentences across both paragraphs, preserving their original order.Version 2 highlighted retained and cut text; users could cancel, replace the text directly, or copy the suggestion, with other interactions disabled until then.Source: §4.1.2 · p. 5, Figure 3 · p. 5

4.2 Language Models and NLP Methods

• The team chose language models and NLP methods pragmatically, balancing output quality against computation speed after trying models of different sizes.The paper frames these implementation choices as supporting its focus on writing insights.Source: §4.2 · pp. 5–6

4.2.1 Extractive Summarization

• Extractive summarization used GloVe embeddings1 and TextRank; it supplied Levels 1–4 in study 1 with k decreasing from 4 to 1, and the Central Sentence level in study 2 with k=1.The extractive method selected the top k sentences.Source: §4.2.1 · p. 6

4.2.2 Abstractive Summarization

• Abstractive summaries used the T5 transformer model [36], with a maximum length of 70% of source tokens; it powered Level 5 in study 1 and Summary in study 2.The team mostly used defaults, setting num_beams=4, no_repeat_ngram_size=2, and early_stopping enabled.Source: §4.2.2 · p. 6

4.2.3 Keyword Extraction

• Keyword extraction used the open-source wordwise3 library, which uses a RoBERTa model4, for the Keyword level in study 2; it was absent from study 1.Source: §4.2.3 · p. 6

4.3 Backend / Server

• A cache reused prior paragraph summaries and recomputed only paragraphs changed by author revisions; updates were almost instantaneous for extractive methods and keywords, and took up to 2 seconds for abstractive summaries.The prototype ran on a university server with 32GB RAM and a 12GB GPU, using a lightweight Python Flask API that processed paragraph arrays and returned results as JSON.Source: §4.3 · p. 6

5.1 Think-Aloud During Interaction

• During the writing task, participants articulated their thoughts while researchers recorded video, audio, and screens and took and merged observational notes.Researchers occasionally asked questions to understand participants’ thinking or remind them to think aloud.Source: §5.1 · p. 6

5.2 Semi-Structured Interviews

• After the writing task, semi-structured interviews explored observed moments and participants’ likes, dislikes, and desired prototype changes..Source: §5.2 · p. 6

5.3 Coding of Think-Aloud & Interviews

• Researchers used open, axial, and selective coding to develop themes from session notes and interview transcripts, then checked relevant automatically transcribed parts against the videos.Two researchers independently assigned inductive codes, compared and clustered them, and agreed on cluster labels; three researchers then reviewed all interview transcripts for supporting evidence and counterexamples.Source: §5.3 · p. 6

5.4 Questionnaire

• After the video call, participants rated each feature on a five-point Likert scale with a “did not use” option and answered two open questions about use cases and prototype changes..Source: §5.4 · p. 6

6.1 Participants

• Twelve people participated across the two studies: four in the first and eight in the second, with varied writing backgrounds and frequencies.Participants were 22–36 years old, were not native English speakers, and regularly used English; the sample was recruited through university networks and personal contacts.Source: §6.1 · pp. 6–7

6.2 Procedure

• Study sessions averaged 60 minutes and began with study information and a researcher demonstration of the prototype’s summarization levels, merge function, and paragraph-card links.The demonstration used text from an arbitrary Wikipedia article.Source: §6.2 · p. 7, §6.2.2 · p. 7

6.2.2 Essay Writing (30 minutes).

• Participants chose one of two opinionated articles and wrote an analytical essay about how its author built an argument, while thinking aloud.The SAT practice-test task was chosen to balance working from an existing text with producing an essay rather than simply copying or editing; participants could switch between the prototype and article.Source: §6.2.2 · p. 7

6.2.3 Concluding Interview and Questionnaire (10+10 minutes).

• After writing, participants completed a semi-structured interview and then filled in a questionnaire after the video call.Source: §6.2.3 · p. 7

7 RESULTS

• All summary settings and both extractive and abstractive methods were used considerably; interaction and interviews averaged 40 minutes, and final texts averaged 416 words.Figure 5 shows time spent in each summary setting across the two studies.Source: Figure 5 · p. 7, §7 · p. 7

7.1 Design Insights and Improvements

• Study 1 findings motivated prototype changes, which the authors report alongside their observed consequences in study 2.Source: §7.1 · p. 7

7.1.1 Unclear Semantic Zoom and Summarization Levels (Study 1) were Redesigned (Study 2).

• Reversed plus/minus semantics and hard-to-distinguish levels, especially for short paragraphs, led to four text-labeled summary buttons; study 2 showed no indication of misunderstanding.The four levels were the original paragraph, a central sentence, a short abstractive summary, and extracted keywords.Source: §7.1.1 · pp. 7–8

7.1.2 Unclear Correspondence of Text and Cards (Study 1) and Im-

• Because participants lost track of which card matched each paragraph, study 2 added card-to-text highlighting, which all participants appreciated; their behavior also indicated a need for reverse highlighting.One participant clicked through cards to find the paragraph of interest, suggesting paragraph-to-card highlighting would help.Source: §7.1.2 · p. 8

7.1.3 Unflexible Direct Changes (Study 1) were Replaced with a Copy to Clipboard Feature (Study 2).

• Direct paragraph replacement was inflexible, confusing without visual feedback, and framed summaries as suggestions; copying to the clipboard gave users control over placement and discouraged an “accept” mindset.The clipboard change required users to paste the summary themselves.Source: §7.1.3 · p. 8, §7.1.2 · p. 8

7.1.4 Difficulty of Assessing Merge Changes (Study 1) was Addressed with Better Merge Preview (Study 2).

• Study 2 participants responded positively to a merge preview that used color and font styling to show which text was retained or removed.Source: §7.1.4 · p. 8

7.2.1 Using Automated Summaries to Understand Longer Text.

• Summaries offered a quick overview of longer source texts, helping participants avoid rereading everything and assess which material mattered to their argument.Some copied source passages into the editor and consulted paragraph summaries or central sentences; this strategy occurred before writing.Source: §7.2.1 · p. 8

7.2 Interaction and Writing Strategies

• Before writing, participants tried summary levels and then either integrated summaries into a draft or removed the source text to begin with a blank page.This workflow followed their use of summaries to read and assess source material.Source: §7.2.1 · p. 8

7.2.2 Developing Text Top-down vs. Bottom-up.

• In bottom-up writing, participants built drafts from an empty editor while checking the source; in top-down writing, they used summaries as building blocks and drafted connecting sentences.Across both approaches, participants edited summaries or source text to avoid plagiarism and match their writing style.Source: §7.2.2 · p. 8

7.2.3 Adapting Text as Part of Integrative Leaps and “Reverse Leaps”.

• Writers reformulated summaries to match their style and changed their own paragraphing to test how summaries reflected their intended message, an integration process Singh et al. call “integrative leaps.”One participant combined paragraphs or removed spaces to explore changes in the summaries.Source: §7.2.3 · p. 8, §7.2.2 · p. 8

7.2.4 Summaries as Textual Building Blocks for Overview Texts.

• Participants used summaries as building blocks for overview texts, including conclusions and abstracts.One participant skimmed existing summaries to recall main messages for a conclusion; another described generating an abstract from the text.Source: §7.2.4 · p. 8

7.3 Reflection Strategies

• Participants treated summaries as an external view on their writing, comparing them with their drafts and sometimes integrating summary content into the text.They also valued seeing their paragraphs expressed in different, slightly condensed wording.Source: §7.3.1 · p. 9

7.3.1 Considering Summaries as Another, External Perspective.

• Differences between a paragraph’s summary and a writer’s mental summary prompted closer analysis of the text and often led to edits.Participants examined why the summaries differed, including whether the tool’s version was longer than their own.Source: §7.3.1 · p. 9

7.3.2 Using Summaries to Check and Revise Paragraph Scope and Structure.

• Participants used summaries to check whether paragraphs conveyed their intended message, then elaborated the main point or split the paragraph when they did not.They also valued reordering and merging paragraphs, as well as simplifying large paragraphs.Source: §7.3.2 · p. 9

7.4 Feedback in the Final Questionnaire

• Participants described summaries as useful for organizing chapters, quickly surveying text, and writing texts that require structure and conciseness.Examples included blog entries, news articles, and summaries or conclusions.Source: §7.4 · p. 9
• Study 1 questionnaire feedback informed a design iteration, while study 2 suggestions mainly sought a clearer visual link between paragraphs and summaries.One suggestion was to highlight a paragraph on hover rather than only on click.Source: §7.4 · p. 9
• Although participants differed in which features they found helpful or used, every feature helped a considerable proportion, and “Delete” was least helpful.Figure 6 reports ratings for individual features; counts vary because respondents could select “did not use.”Source: §7.4 · p. 9, Figure 6 · p. 9

7.5 Comparisons against Writing without Automatic Summary

• Participants used manual highlighting and delayed structural revision in their usual workflow, a process one said the central-sentence summaries corresponded to.The participant described writing while summarizing source paragraphs, then returning to consider structure after finishing the text.Source: §7.5 · p. 9
• Without margin summaries, one participant expected to finish drafting before realizing the submission was too long and beginning to shorten it.Source: §7.5 · p. 9
• Participants said automatic summaries could replace laborious manual reading and summarization when writing from source material.One participant described the summaries as concretely identifying what was needed and avoiding hours of reading.Source: §7.5 · p. 9

8.1 How Continuous Automatic Text Summaries Support Writers

• Writers used continuously updated summaries to reflect on paragraph content, scope, and structure, gain an overview of source material and their essays, and see their writing from an external perspective.Participants valued the summaries as support for reflection and overview, including when summarizing their own writing; the authors suggest this external view may be especially helpful when writing alone.Source: §8.1 · pp. 9–10

8.2 Self-Annotation and “AI Annotation”

• Comparing mental summaries with AI annotations suggests automatic summaries influenced self-annotation rather than replacing it, while their “fresh eyes” quality may benefit writers.The authors contrast potentially useful low perceived authorship for annotations with concerns that AI generation or editing could reduce agency.Source: §8.2 · p. 10
• AI annotations need clear correspondence with user content; although participants expected textual annotations, the authors suggest exploring mixed manual-and-automated and non-text annotations [17].Manual annotation offers more spatial and symbolic flexibility, while the study’s click-to-highlight design clarified what each AI annotation referred to.Source: §8.2 · p. 10

8.3 Limitations & Reflections on Methodology

• The authors note that summaries were not state-of-the-art and that the controlled, single-article task and lack of a baseline limit generalization, motivating in-the-wild and longer-text studies.Participants still worked productively with summaries despite quality reservations; the authors also propose studying writing-style match and comparing argumentative, informative, and creative texts.Source: §8.3 · p. 10

8.4 AI Role Perception Through Design

• Changing “Replace in Text” to “Copy to Clipboard” shifted perception away from text suggestions, illustrating how small interface choices can shape the AI’s perceived role.The authors position the system as an interaction model in which AI generates parallel content to empower users’ self-evaluation, extending the surveyed models [37].Source: §8.4 · pp. 10–11

8.5 Designing Human-AI Co-creative Systems from Existing Cognitive Strategies

• The authors recommend using established human writing strategies to guide NLP tool design: reverse outlining shaped initial choices and constrained the AI to summarization rather than interpretation or idea generation.They propose examining other writing strategies as a template for developing new NLP tools for writers.Source: §8.5 · p. 11

9 CONCLUSION

• Continuously updated AI-generated summaries gave writers an overview and an external perspective during writing.Participants used these insights to check and revise the scope and structure of their drafts.Source: §9 · p. 11
• The work demonstrates writing support beyond text generation and correction.It envisions future AI writing tools combining direct text edits with indirect reflection support.Source: §9 · p. 11
• Future work could explore adaptive margin annotations and integrate nontextual annotations into drafting tools.The authors released the prototype and further project material to facilitate research in co-creative writing tools.Source: §9 · p. 11

A ADDITIONAL MODEL EVALUATION

• Evaluating summary quality after every interaction was impractical because text and summaries changed continuously, so the evaluation used the given articles instead.Source: A ADDITIONAL MODEL EVALUATION · p. 13
• Three people wrote abstractive summaries for each article paragraph and selected its most central sentence as references for evaluating summaries on the given articles.Source: A ADDITIONAL MODEL EVALUATION · p. 13
• Mean ROUGE-L F-score was ca. 0.2 against human summaries and ca. 0.5 against human-selected sentences, generally comparable to T5’s benchmark [36].The authors present these values as a point of comparison for future work using AI summaries in similar systems and use cases.Source: A ADDITIONAL MODEL EVALUATION · p. 13
• The central sentence method selected the same sentence as human annotators in 48% of cases.Source: A ADDITIONAL MODEL EVALUATION · p. 13