Source-linked AI summary
Interacting with next-phrase suggestions: How suggestion systems aid and influence the cognitive processes of writing
Advait Bhat, Saaket Agashe, Niharika Mohile, Parth Oberoi, Ravi Jangir, Anirudha Joshi
TL;DR
Research on writers’ interactions and decision-making with next-phrase suggestions is still emerging. The authors qualitatively studied 14 amateur writers completing reviews with and without suggestions and with varied sentiment-bias alignment. Writers used suggestions in nuanced ways, while the system also affected writing plans and distraction.
Problem
Research on suggestion systems has focused largely on writing products or observable behavior, leaving writers’ cognitive decisions during interaction less understood.
Source: §1 · pp. 1–2Method
Results
Writers used suggestions in nuanced ways, including incorporating selected parts despite disagreement, while suggestions affected evaluation, writing plans, and distraction.
Source: §6 · p. 25Takeaways & Limitations
The authors propose a writer-suggestion interaction model for studying and designing suggestion systems in the movie-review setting.
Source: §1 · p. 2, §6 · p. 25Movie reviews are an uncommon, relatively benign task, so the findings may not extend directly to more nuanced or sensitive writing such as personal emails or political opinion pieces.
Source: §5.4 · p. 25
Abstract
from arXiv · showhide
Writing with next-phrase suggestions powered by large language models is becoming more pervasive by the day. However, research to understand writers' interaction and decision-making processes while engaging with such systems is still emerging. We conducted a qualitative study to shed light on writers' cognitive processes while writing with next-phrase suggestion systems. To do so, we recruited 14 amateur writers to write two reviews each, one without suggestions and one with suggestions. Additionally, we also positively and negatively biased the suggestion system to get a diverse range of instances where writers' opinions and the bias in the language model align or misalign to varying degrees. We found that writers interact with next-phrase suggestions in various complex ways: Writers abstracted and extracted multiple parts of the suggestions and incorporated them within their writing, even when they disagreed with the suggestion as a whole; along with evaluating the suggestions on various criteria. The suggestion system also had various effects on the writing process, such as altering the writer's usual writing plans, leading to higher levels of distraction etc. Based on our qualitative analysis using the cognitive process model of writing by Hayes as a lens, we propose a theoretical model of 'writer-suggestion interaction' for writing with GPT-2 (and causal language models in general) for a movie review writing task, followed by directions for future research and design.
1 INTRODUCTION
As next-phrase suggestions become viable in writing interfaces, research on writers’ decisions and cognitive processes remains emerging. This study examines how writers interact with suggestions, how they affect writing, and how sentiment-bias alignment matters.
1 INTRODUCTION
- Transformer-based language models have enabled inline next-phrase suggestions in writing interfaces, while research on writers’ interactions with these systems remains emerging.The passage names Google Smart Compose and LightKey as examples.Source: §1 · p. 1
- Research on suggestion systems has examined effects on vocabulary, perceived sentiment, and interpersonal perception, mainly through the resulting writing product.The cited studies focus on the product rather than writers’ decision-making processes.Source: §1 · pp. 1–2
- The study uses Hayes’s cognitive model to examine writers’ decisions while comparing writing with and without suggestions and varying sentiment-bias alignment.Fourteen amateur writers wrote two reviews each using a GPT-2-based system fine-tuned on IMDb reviews.Source: §1 · p. 2
- Writers incorporated selected parts of suggestions, evaluated them by multiple criteria, and experienced changes to writing plans and distraction.The study also proposes a writer-suggestion interaction model for its movie-review setting.Source: §1 · p. 2
2 RELATED WORK
Prior work frames writing as a complex cognitive activity and documents effects of suggestions on writing products, but behavioral accounts reveal less about writers’ decision-making. This study draws on cognitive models, protocol methods, and research on Indian non-native English writers to situate that gap.
2.1.1 The Cognitive Process Model of Writing.
- Hayes’s model organizes writing into control, process, and resource levels, including planning, idea generation, translation, transcription, evaluation, the task environment, and working memory.The paper adds Working Memory State to describe the writer’s composition state when encountering a suggestion.Source: §2.1.1 · pp. 3–4
2.2 Writer-Suggestion Interaction
- Prior studies report more predictable language and shorter sentences, positive sentiment shifts, and changes in efficiency and interpersonal perception [38].Suggestion systems have been studied both as writing tools and as AI-mediated communication technologies.Source: §2.2.1 · p. 4
2.2.1 Interactions and effects of suggestion systems on writing.
- Prior interaction studies describe behavior patterns and integrative leaps, while this paper asks how cognitive processes govern interaction and how suggestions affect writing compared with no suggestions.The paper formulates separate research questions about writer-suggestion interaction and effects on the writing process.Source: §2.2.1 · p. 4
2.2.2 Studying writer-suggestion mis/alignment.
- Controlling suggestion content through model bias enables examination of alignment, which is normative because writers judge suggestions through their own values.The paper asks how misalignment between writer opinion and model bias affects interaction.Source: §2.2.2 · pp. 4–5
- Interaction logs and replays capture behavior but do not reveal writers’ inner cognitive decision-making processes.Concurrent think-aloud protocols provide insight into steps before decisions, while retrospective protocols provide more insight into final decisions.Source: §2.2.3 · p. 5
2.2.3 Methodological choices in studying writer-suggestion interaction.
- Think-aloud and retrospective protocols offer complementary views of writing decisions, but frequent prompting can slow writing and requires balancing elicitation with usual practice.Concurrent protocols yield more segments and insight into decision steps; retrospective protocols provide more insight into final decisions.Source: §2.2.3 · p. 5
2.3 Working with Indian non-native English speakers
- Prior work suggests L2 writers may use more suggestions and pause after shorter writing episodes; the study focuses on Indian non-native English writers.Participants had non-English first languages but completed K–12 schooling in English, a context the authors distinguish from other non-English-speaking countries.Source: §2.3 · p. 5
- The study aims to build inductive observations rather than test hypotheses, offering a theoretical springboard for further empirical research.Its intended areas include interaction with next-phrase suggestions, differences between writing with and without them, and effects of writer-system misalignment.Source: §2.3 · p. 6
3 METHOD
The study used a movie-review task and deliberately varied alignment between writers’ opinions and suggestion-system bias to examine how writers interacted with suggestions. It compared writing with and without suggestions and analyzed think-aloud data using the Hayes model.
3 METHOD
- Fourteen amateur writers each wrote two movie reviews, one without suggestions and one with a system biased toward positive or negative reviews, to sample varied writer–system alignment.The study used concurrent and retrospective think-aloud protocols and screen recordings to examine writers’ actions and decisions.Source: §3 · p. 6
3.1 Movie selection
- Nine movies were selected from aggregated internet ratings and grouped as Bad (1–3), Neutral (4–7), or Good (7–10).The researchers averaged ratings from IMDb, Rotten Tomatoes, and Letterboxd to rank 30 candidate movies before selecting three per category.Source: §3.1 · pp. 7–8, Table 1 · p. 8
3.2 Apparatus
- The editor offered word and phrase completions at text ends and within text, while displaying one-word-at-a-time acceptance.Suggestions appeared after a 300 ms pause; GPT-2 models fine-tuned on positive or negative IMDb reviews generated completions using sampled beam search.Source: §3.2 · pp. 8–9
- Sentiment validation classified 76% of positive-model generations as positive and 78% of negative-model generations as negative.For phrase usefulness, the positive and negative GPT-2 models scored 0.3 and 0.29, respectively, compared with 0.23 for the AWD-LSTM baseline.Source: §3.2 · pp. 9–10
3.3 Participants
- Participants were university students or recent graduates, fluent desktop typists, and non-native English speakers educated in English-medium K–12 schools.The participant data table records first language, assigned movie, writer rating, system alignment, and degree of misalignment.Source: §3.3 · p. 10, Table 3 · p. 11
3.4 Data Analysis
- Four authors analyzed 28 sessions through iterative coding that combined inductive codes with Hayes-based deductive codes, then assessed the codebook against remaining sessions.The researchers stopped at 14 participants after judging that the data had reached saturation and included diverse alignment levels; Table 4 presents examples of different writer–system alignments.Source: §3.4 · p. 11, Table 4 · p. 12
4 FINDINGS
The findings begin with examples of suggestions and writer–system alignments, then examine how suggestions relate to writing processes and evaluation. They also compare writers’ processes with and without suggestions.
4 FINDINGS
- Eight examples show distinct writer–system alignments and the different generations and interactions they produced.The examples include the text writers had produced immediately before each suggestion.Source: §4 · p. 12
- The findings analyze suggestions through Hayes’s writing processes, writers’ evaluation criteria, and observed differences between writing with and without suggestions.The authors report effects on the writing process after describing how writers evaluated suggestions for incorporation.Source: §4 · p. 12
4.1 Contribution of suggestions to the processes of composition
Suggestions contributed to composing at the proposer, translator, and transcriber stages, sometimes supplying ideas or language and sometimes only saving typing. Writers could reuse parts of a suggestion without adopting its whole opinion or proposal.
4.1 Contribution of suggestions to the processes of composition
- Suggestions contributed to one or several Hayes composition processes, from generating proposals and translating them into language to transcribing text.Writers accepted contributions immediately or stored them for later in memory or the editor.Source: §4.1 · p. 12
4.1.1 Contributions to the proposer.
- Writers abstracted suggestion themes to prompt new proposals or augment existing ones, often expressing them in their own words and adding detail.They also drew new, sometimes unrelated ideas from suggestions, but did not abandon existing proposals for suggested themes.Source: §4.1.1 · pp. 12–13
- Writers also borrowed sentence structures from suggestions, even when they adopted neither the suggested opinion nor its topic.One writer reused a comma-separated adjective–noun pattern to describe different movie features.Source: §4.1.2 · p. 13
4.1.2 Contributions to the translator.
- With suggestions, writers used borrowed phrases and familiar movie-review wording, while their unaided reviews were longer, more nuanced, and structurally complex.The suggestion outputs often used shorter single-noun, single-adjective sentences, a pattern adopted by most writers in the with-suggestion trials.Source: §4.1.2 · pp. 13–14, Table 5 · p. 14
- Writers sometimes replaced their planned translation with a suggestion when it expressed their meaning better or saved typing effort.They could select only a fitting part rather than accept the complete suggestion.Source: §4.1.2 · p. 14, §4.1.3 · pp. 14–15
4.1.3 Contributions to the transcriber.
- Writers used suggestions to transcribe parts of text already intended, including by tabbing through matching phrases or completing a word after typing its first letters.A writer typed “per” and accepted the completion “perception” because it matched their intended translation.Source: §4.1.3 · p. 15
4.2 Evaluating suggestions
Writers evaluated suggestions independently, against their text, or against what they held in working memory, with their beliefs about the system also shaping their responses. Acceptance depended on factors such as fit with their opinions, plans, facts, and intended meaning.
4.2 Evaluating suggestions
- Writers evaluated suggestions independently, against text already written, or against their working-memory composition, while beliefs about the system also shaped their judgments.Independent evaluation occurred when writers had no ready proposal or translation in working memory.Source: §4.2 · p. 15, §4.2.1 · p. 15
4.2.1 Independent evaluation of suggestions.
- In independent evaluation, writers rejected suggestions that conflicted with their sentiment, specific opinions, movie-review schemas, or factual knowledge.Schemas included avoiding spoilers and following a planned review structure; rejected suggestions could still be stored for later use.Source: §4.2.1 · pp. 15–16
4.2.2 Evaluation with respect to text written so far.
- Writers also judged whether suggestions flowed consistently with the text they had already written.A suggestion could be declined as an abrupt change or accepted because it flowed well with preceding text.Source: §4.2.2 · p. 16
4.2.3 Evaluation with respect to the Working Memory State (WMS).
- Against their working-memory state, writers checked whether suggestions fit their intended meaning or improved their wording, but suggestions could distract them while they were forming ideas or language.When writers lacked a definite proposal, they could be more open to a suggestion; they also accepted alternatives that seemed better than their own wording.Source: §4.2.3 · pp. 16–17
4.2.4 Evaluation influenced by the beliefs about the suggestion system.
- Beliefs that the system reflected public opinion or adapted to a writer’s context influenced trust and openness to suggestions.Some writers ignored early suggestions while establishing context, while unmet expectations for sentiment alignment reduced openness over time.Source: §4.2.4 · p. 17
4.3 Effects on the writing process
Suggestions changed writers’ criticality depending on alignment with the model’s training-corpus sentiment, while also disrupting plans and increasing distraction. Effects ranged from unnoticed or persuasive suggestions to frustration and resistance.
4.3 Effects on the writing process
- Without suggestions, writers maintained their intended sentiment; with suggestions, changes in criticality depended on alignment with the system’s training-corpus sentiment.Source: §4.3.1 · pp. 17–18
4.3.1 The role of alignment.
- Writers whose sentiment closely matched the system’s bias often failed to notice it, whereas imperfect alignment made the bias noticeable.Source: §4.3.1 · p. 18
- Slight misalignment sometimes shifted writers toward the system’s bias, but U8 resisted strong negative suggestions by softening them.U6 said repeated negative suggestions convinced them to become more negative, while U8 rejected strongly negative wording as unqualified criticism.Source: §4.3.1 · p. 18
- With complete misalignment, writers usually dismissed suggestions in frustration, though some retained their wording while flipping its sentiment.One writer reacted to repeated positive suggestions by making a neutral review decidedly negative.Source: §4.3.1 · pp. 18–19
4.3.2 Changes in plan due to suggestions.
- Suggestions could redirect writers from their usual plans, though some accepted content early to save time; without them, writers drew on memory, planning, rereading, and blurting.Source: §4.3.2 · p. 19, §4.3.3 · p. 19
4.3.3 Distraction due to suggestions.
- Constantly reading suggestions increased cognitive load and distraction, sometimes derailing ideas, hurrying composition, and impeding iterative revision.Source: §4.3.3 · pp. 19–20
5 DISCUSSION
The discussion frames suggestions as participants in writers’ cognitive processes, not merely tools for transcribing ideas already formed. It uses this model to interpret evaluation demands, possible design responses, and the study’s limits.
5.1 A writer-suggestion interaction Model
- The model treats suggestions as interacting with idea generation, translation, transcription, and evaluation rather than merely transcribing already-formed ideas.Writers may compare suggestions with their working-memory compositions, and beliefs about system functioning may shape their openness.Source: §5.1 · pp. 20–22
5.2 Framing interaction with the writer as interaction with writing processes
- The process-level framing distinguishes idea-level from language-level influence and motivates tailoring suggestions to the writing process a writer wants supported.The discussion proposes varying generation strategies and giving writers control over support for ideas, wording, or transcription.Source: §5.2 · pp. 22–23
5.3 The evaluator needs attention.
- The evaluator judges outputs from the writer’s processes and the suggestion system, comparing them to determine which best fits.This extends the Hayes evaluator role to suggestions as well as the proposer, translator, and transcriber outputs.Source: §5.3.1 · p. 23
5.3.1 Evaluator fatigue.
- Suggestions may increase evaluator load and fatigue, potentially leading writers to omit criteria, become less critical, or conform to save effort.The proposed mechanism is comparison against a suggestion while holding a semi-formed composition in working memory.Source: §5.3.1 · p. 23
5.3.2 Muddled Judgment.
- Writers’ inaccurate beliefs about system capabilities may lead them to treat suggestions as authoritative and become more open to misaligned content.Some participants believed suggestions reflected majority opinion or that the system could retrieve accurate contextual information like a search engine.Source: §5.3.2 · pp. 23–24
5.3.3 A Way Forward.
- The authors propose reducing evaluation load and improving writers’ information through explanations, capability cues, and feedback about a suggestion’s effects on composition tone.They also suggest personalisation and interfaces that expose how suggestions are generated.Source: §5.3.3 · p. 24
5.4 Limitations
- Suggestion frequency and alignment measures were constrained by design choices, including a 300 ms trigger and assumed uniformity in positive and negative review corpora.The authors describe Corpus Rating and misalignment scores as representative rather than accurate, and did not include a whole-corpus unaligned condition.Source: §5.4 · pp. 24–25
- The findings remain provisional beyond movie reviews and L2 English speakers in India, and require care when generalized across tasks, models, biases, and interaction modalities.The authors characterize qualitative-study conclusions as provisional and contestable.Source: §5.4 · p. 25
6 CONCLUSION
The study examines how next-phrase suggestions affect writers’ processes through a qualitative comparison of reviews written with and without suggestions. It finds nuanced use and evaluation of suggestions, and proposes a model of writer-suggestion interaction.
6 CONCLUSION
- Fourteen writers each wrote two movie reviews, one without suggestions and one with a randomly assigned system based on a corpus rating of 2.5 or 8.5.Think-aloud and retrospective protocols were analyzed for insights into effects on writing processes.Source: §6 · p. 25
- Writers used suggestions directly or indirectly for idea generation, language, and typing, while evaluating them in varied ways; misalignment primarily affected evaluation.Suggestions also changed writing plans and increased distraction.Source: §6 · p. 25
- The authors propose a writer-suggestion interaction model and identify possible uses, future research, and design opportunities.Source: §6 · p. 25