PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Cognitive Process-Aware Proactive Writing Support

Masahiro Yoshida, Atsuya Kobayashi, Kei Tateno, Xiang 'Anthony' Chen

arXiv:2608.30424v1cs.HCcs.AI

TL;DR

Writing assistants often require users to articulate ambiguous needs, especially in creative writing, creating cognitive and interaction burdens. This paper uses Flower–Hayes cognitive processes as an intermediate layer connecting writing interactions and document context to fourteen support types in AToM CoWriter. Two studies provide initial evidence of improved expressiveness and greater engagement with proactive suggestions when cognitive-process cues are used.

  • Problem

    Existing writing support requires writers to recognize and articulate their needs, which can burden creative writing when intentions are ambiguous.

  • Method

    The paper develops a Flower–Hayes-grounded framework and implements AToM CoWriter to infer cognitive processes from interaction logs and document context for support selection.

  • Results

    Two studies provided initial evidence that system-inferred support improved expressiveness and that interaction-derived cognitive-process cues increased engagement with proactive suggestions.

  • Takeaways & Limitations

    Cognitive processes can serve as a useful intermediate representation for selecting support in proactive writing systems.

  • Takeaways & Limitations

    The exploratory studies used 21 engineers from one technology company, limiting population coverage; intervention timing also remained challenging.

Abstract

from arXiv · show

Large language models can support writing, but existing tools require users to explicitly articulate prompts-particularly burdensome in creative writing, where intentions are often ambiguous. Proactive support that infers users' needs from writing interactions could alleviate this burden, but raises two challenges: determining what support to provide and when to intervene. This work focuses on the former. We hypothesize that Flower and Hayes' cognitive process theory of writing-which characterizes writing through six cognitive processes-offers an interpretable bridge between observable writing behavior and appropriate support types. Through a formative study and literature review, we identify 14 writing support types associated with these cognitive processes, along with characteristic interaction behaviors linked to each process. We then instantiate this framework in AToM CoWriter, which infers support needs from writing interactions and document context. Two within-subjects studies (N = 21) provide initial evidence that this approach improves expressiveness and idea exploration, and that cognitive process inference increases engagement with proactive suggestions. These findings suggest that cognitive processes can provide a promising basis for support selection in proactive writing systems.

1 Introduction

Existing LLM writing tools depend on users articulating prompts, which can disrupt creative writing when intentions and needs are ambiguous. This work addresses what proactive support should provide by using cognitive processes as an interpretable bridge from writing behavior to support selection.

  • Motivation: Prompt-based writing support imposes cognitive and physical burdens because writers must recognize, verbalize, and articulate their needs.These burdens can interfere with natural writing flow, particularly in creative writing where intentions may change dynamically.
  • Motivation: Proactive support shifts part of the prompting burden to the system by inferring needs from interaction traces and document context.Relevant signals include keystrokes, mouse operations, cursor movements, editing patterns, and the evolving document.
  • Research Focus: The paper focuses on selecting what support to offer, distinguishing this problem from deciding when to intervene.The system retains inactivity-based triggering for the when problem while investigating cognitive-process cues for support selection.
  • Framework: A formative study and literature review identify fourteen support types associated with six Flower–Hayes cognitive processes and characteristic writing behaviors.The framework connects observable interactions to cognitive processes and then to appropriate forms of assistance.
  • System: AToM CoWriter instantiates the framework by combining interaction behaviors and document context to infer cognitive processes and select context-appropriate assistance.Its Main Agent coordinates selection among fourteen Functional Agents that generate suggestions after typing inactivity.
  • Evaluation: Two within-subjects studies provide initial evidence of improved expressiveness and idea exploration, while cognitive-process cues increased engagement with proactive suggestions.The studies included 21 participants across complementary evaluations.

2 Related Work

Related work frames proactive writing assistance around when to intervene and what assistance to provide. This paper focuses on the latter, using Flower–Hayes cognitive processes as an interpretable intermediate layer between interaction traces and multiple support types.

  • Proactive Support: Explicit-request systems impose cognitive and interaction costs, while poorly timed proactive interventions can undermine agency in open-ended writing.The central design tension is shifting initiative to the system without losing user control.
  • When to Intervene: Proactive systems use triggers including real-time input, inactivity, editing thresholds, and increasingly adaptive timing mechanisms.These mechanisms address when assistance should be offered rather than which assistance should be selected.
  • What to Offer: The what question concerns choosing an assistance form, with prior systems selecting among fixed or multiple support categories.Examples include a bandit selecting three support types and an LLM predicting eight support categories.
  • Process-Oriented Writing: Flower–Hayes theory decomposes writing into six cognitive processes grouped as Planning, Translating, and Reviewing.Planning includes Generating, Organizing, and Goal Setting; Reviewing includes Evaluating and Revising.
  • Process-Oriented Writing: Prior research indicates that assistance value and appropriate support vary across writers’ cognitive processes.This motivates aligning intelligent writing support with the process in which the writer is engaged.
  • Interaction-Based Modeling: Writing interaction traces such as pauses, revisions, cursor activity, and keystrokes provide signals for modeling ongoing cognitive processes.Related programming assistants similarly infer task-relevant states from editor interactions and context.
  • Paper Positioning: AToM CoWriter estimates cognitive processes before selecting among support types, rather than mapping low-level behavior directly to an intervention.This creates an interpretable connection between interaction-based user modeling and writing theory.
  • Paper Positioning: The paper complements new co-writing interaction techniques by addressing upstream support relevance without requiring writers to articulate their needs.It does not propose one interaction paradigm for all writing activities.

3 Formative Study

The formative study links momentary cognitive processes, writing behaviors, and support needs through self-report data and literature synthesis. It identifies fourteen support types, characterizes behavioral indicators, and evaluates LLM-based process prediction as a basis for proactive support selection.

  • 3 Formative Study: The formative study defines a cognitive process as the momentary process most prominent during a given writing moment, following Flower and Hayes.The study examines relationships among these processes, interaction behaviors, and required support.
  • 3.1 Data Logging Editor: The custom editor recorded detailed writing interactions and self-reported cognitive processes during writing blocks.Writing logs captured keystrokes, mouse operations, cursor positions, and text content; self-reports captured participants’ reflections.
  • 3.1.1 Self-Report Procedure: Participants selected one of six cognitive processes and verbally reported their difficulty and desired support after an inactivity prompt or manual initiation.The Interviewer recorded the selected process, spoken difficulty, and spoken desired support across repeated sessions.
  • 3.1.1 Self-Report Procedure: The procedure used a 10-second inactivity reminder, simple process selection, and voice responses to balance data richness with low interruption.Pilot feedback indicated that shorter intervals distracted participants and longer intervals caused forgotten reports; verbal input reduced typing effort.
  • 3.3 Behavior During Writing Sessions: 152 self-reports were collected, averaging 9.5 per participant, with Generating reported most frequently across early and later story development.The study also examined median pause duration before reported blocks across cognitive processes.
  • 3.5 Relationship Between Writing Interaction Behavior and Cognitive Processes: Writing-process timing and interaction patterns differed: Goal Setting preceded Generating and Organizing, while speed, cursor activity, scrolling, and deletion varied by process.Delete events were most frequent during Evaluating, and writing speed dropped sharply during Organizing.
  • 3.6 Cognitive Process Prediction: LLM prediction using document content and behavioral descriptions outperformed LightGBM and document-only ablations, while behavioral cues improved disambiguation.The combined LLM approach achieved the highest performance across evaluated process-classification metrics and mappings.
  • 3.4 Relationship Between Cognitive Processes and Support Needs: The study integrated self-reports and literature review findings into a taxonomy of fourteen support types mapped to cognitive processes and implemented functions.Goal Setting received a support type defined from observed requirements when the initial literature review found no corresponding type.

4 System Design and Implementation

AToM CoWriter uses cognitive-process inference as an intermediate representation for selecting writing assistance, while keeping intervention timing simple. Its interface combines agent coordination, clarification, generated outputs, and inline suggestions across proactive, prompt-less, and prompt-based scenarios.

  • Design rationale: Cognitive-process inference connects writing interactions and document context to support selection, addressing the proactive system’s “what” problem while using inactivity for timing.This design isolates support-selection effects from novel timing behavior.
  • System architecture: AToM CoWriter combines a rich-text editor, chat interface, and dedicated output area with a Main Agent coordinating up to three Functional Agents in parallel.The Functional Agents correspond to fourteen support types identified in the formative study.
  • Interaction scenarios: In the Proactive scenario, typing inactivity triggers automatic support inference and background generation, with unobtrusive notifications intended to minimize disruption.The Main Agent activates relevant Functional Agents in parallel after the predefined idle period.
  • Interaction scenarios: The Prompt-less scenario lets users press a Help button to control when assistance arrives while the system continues selecting what support to provide.Prompt-based interaction instead lets users specify both timing and assistance type directly through chat.
  • Interface: The interface supports chat input, intent-clarification questions, generated panel content, and inline editing suggestions.Functional Agents can request clarification through interactive components such as buttons, sliders, and text inputs.
  • Implementation: Cognitive-process predictions are generated every 30 seconds and sent to the Main Agent through a background templated message.The system also shares complete message history and programmatically provides the current editor content to agents.

5 User Evaluation

Two exploratory within-subjects studies evaluated system-inferred proactive writing support and cognitive-process-aware support selection. System-inferred support improved expressiveness and supported idea exploration, while cognitive process inference increased engagement with proactive suggestions, alongside timing and control tensions.

  • Experiment Setup: Twenty-one engineers completed two within-subjects studies comparing system-inferred with user-articulated support and isolating cognitive process inference within proactive support.Study 1 included 11 participants, and Study 2 included 10; conditions were counterbalanced via Latin square.
  • Study 1 Results: Expressiveness significantly improved with system-inferred support (Holm-adjusted p=0.034, RBC=0.924), while workload and custom evaluation measures showed no significant differences.Total CSI and Results Worth Effort also had large effect sizes (RBC > 0.58, CLES > 0.64), while total NASA-TLX workload was essentially equivalent.
  • Study 1 Results: System-inferred support helped participants continue writing during blocks and exposed them to more diverse ideas, suggesting expanded exploration beyond writers’ independent choices.Participants emphasized the diversity of generated ideas rather than only the convenience of avoiding prompts.
  • Study 1 Results: Proactive support encouraged exploration without clearly reducing ownership, but sometimes disrupted focus and created tensions around timing and control.Participants often retained authorship by filtering and editing AI suggestions, while some suggestions were irrelevant or increased verification costs.
  • Study 2 Results: In Study 2, cognitive process predictions were unevenly distributed, with Translating at 49%, Generating at 24%, and Evaluating at 13%.The predicted processes transitioned dynamically over time, reflecting the iterative and non-linear nature of creative writing.
  • Study 2 Results: Suggestion conversion more than doubled with cognitive process inference (20.5% w/ Cog vs. 7.4% w/o Cog; 177% relative increase), but Holm-adjusted significance was not retained.The effect was very large (r=0.86), while Tabs and Panel conversions showed no significant differences; the authors describe the pattern as suggestive and requiring larger-sample confirmation.
  • Summary and Limitations: The findings support cognitive process–aware proactive writing support as a feasible exploratory paradigm, not as evidence of population-level effects.The authors prioritized depth of interaction analysis over breadth of population coverage and identify timing adaptation as an important direction.

6 Discussion

The discussion highlights that proactive writing support must balance useful assistance with user autonomy, personalization, unobtrusive delivery, and limits on inference. It also identifies adaptive timing, richer intent elicitation, monitoring effects, and extensions to cognitive-process theory as future priorities.

  • 6.1 Balancing Proactivity and User Autonomy: Users differed substantially in their reactions to proactive support, with some welcoming idea proposals and others finding them intrusive.P14 described unsolicited thematic ideas as invading their territory while still appreciating phrasing suggestions.
  • 6.1 Balancing Proactivity and User Autonomy: Personalizing the scope of AI involvement from naturally occurring interaction data could reduce cognitive load while preserving user agency.The proposed approach avoids relying on explicit feedback to infer preferences.
  • 6.1 Balancing Proactivity and User Autonomy: Lower prompting effort may strengthen ownership by making proactive suggestions easier to reject, avoiding the sunk-cost pressure associated with explicitly crafted prompts.Proactive and on-demand suggestions shared an interface, and users could inspect and reject outputs without investing in prompts.
  • 6.1 Balancing Proactivity and User Autonomy: Even minimal notifications sometimes increased time pressure, motivating a formal account of unobtrusiveness in proactive suggestion design.Participants distinguished noticing that output existed from discovering it only when intentionally checking.
  • 6.2 Future Directions for Intent Inference: Average pauses varied across cognitive processes, suggesting that intervention timing should become adaptive rather than rely only on simple inactivity detection.Future triggers could combine process context, behavioral patterns, and individual workflow preferences.
  • 6.2 Future Directions for Intent Inference: Closed-question intent elicitation was rarely used, indicating a need for low-burden, context-aware ways to gather additional information and improve accuracy and trust.The interface may have lacked sufficient precision or perceived relevance.
  • 6.2 Future Directions for Intent Inference: Continuous behavioral monitoring may create an observer effect that changes writing behavior and affects perceived agency, ownership, or trust.The current study did not explicitly measure this effect, especially among broader populations beyond experienced LLM users.
  • 6.2 Future Directions for Intent Inference: The Flower–Hayes model may need extension because prompting and evaluating AI suggestions introduce meta-cognitive activities absent from its original six-process formulation.Future models could represent new processes and transitions arising in human–AI collaboration.

7 Conclusion

This work addresses proactive support selection by linking writing interactions to cognitive processes and support types, then implementing that framework in AToM CoWriter. Two user studies found initial evidence of improved expressiveness, idea exploration, and engagement with proactively generated suggestions, while timing remains a complementary challenge.

  • 7 Conclusion: The framework links observable writing interactions to Flower–Hayes cognitive processes and fourteen writing support types to address what proactive systems should offer.AToM CoWriter uses document context and writing interaction logs to infer processes and select assistance.
  • 7 Conclusion: Two user studies found initial evidence that system-inferred support improved expressiveness and supported idea exploration.The evidence combines an expressiveness outcome with qualitative indications concerning idea exploration.
  • 7 Conclusion: Cognitive-process cues derived from interactions, combined with document context, were associated with substantially greater engagement with proactive suggestions.The reported result concerns engagement with proactively generated suggestions rather than a general performance measure.
  • 7 Conclusion: Intervention timing remains an important complementary challenge because the current inactivity-based trigger and unobtrusive presentation provide only a pragmatic, coarse solution.Future mechanisms should account for cognitive processes, interaction patterns, and individual preferences.
  • 7 Conclusion: Overall, cognitive processes provide a useful theory-grounded bridge between observable writing behavior and proactive support selection.This conclusion is limited to selecting what support to offer, not fully solving when to intervene.

A Formative Study Data

The formative-study appendix documents the coding scheme, prompt templates, behavioral context, and data features used to analyze writing difficulties, desired support, and cognitive processes. It covers six cognitive processes, multiple user-problem and support categories, editor logs, self-reports, document context, and prediction instructions.

  • A.1 Code Book: The code book analyzed Japanese verbal reports collected when participants felt blocked, separating reported problems from requested AI support.Responses were automatically transcribed before coding.
  • A.1.1 Problem (User Difficulties): User-difficulty codes covered idea generation, grammar and style, proofreading, word choice, writing process, time management, creative blocks, knowledge gaps, and technical issues.They also included story development, character setting, world building, writing technique, story structure, narrative flow, opening writing, and ending writing.
  • A.1.2 Request (User Requests for Support): Support-request codes distinguished output formats and operations, including lists, full text, templates, evaluation, direct editing, idea support, writing support, generation, editing, feedback, planning, search, and consultation.The scheme organized requests by both expected output and requested operation.
  • A.2 Prompts for Cognitive Process Prediction: The cognitive-process prediction prompt used dynamically filled user logs and document information to classify one of six Flower–Hayes processes.The six labels were Goal-Setting, Generating, Organizing, Translating, Evaluating, and Revising.
  • A.2 Prompts for Cognitive Process Prediction: Additional appendix materials included self-report utterance prompts, document excerpts, sentence-completeness criteria, and feature documentation for the prediction models.The sentence criterion treated syntactically or semantically finished sentences as complete even without punctuation or line breaks.
  • A.2 Prompts for Cognitive Process Prediction: The prompt defined the six processes as goal direction, idea generation, organization, linguistic translation, evaluation, and revision.These definitions operationalized the process labels for prediction.
  • A.2 Prompts for Cognitive Process Prediction: Participants commonly began with outlines or ideas in a memo, and sessions typically lasted about 40 minutes; approximately half of blocked moments corresponded to Generating.The appendix also described process-specific behavioral patterns, including speed, cursor movement, scrolling, sentence completeness, and deletions.
  • A.2 Prompts for Cognitive Process Prediction: The prompt incorporated text lengths, cursor position, document ending, writing speed, deletions, scrolling, and cursor movement as current context and recent activity.These variables supplied the editor-state information used for process prediction.

A.3 Cognitive Process Prediction Results

The cognitive-process prediction appendix evaluates six-class and three-class classification using LightGBM and LLM approaches. The LLM using logs plus document context achieved the strongest reported performance across the evaluated metrics, while the system also exposed editor functions for reading, retrieval, questioning, and response collection.

  • A.3 Cognitive Process Prediction Results: The evaluation measured accuracy, macro-F1, and weighted-F1 for both six-process classification and a coarser three-process mapping.The three-process mapping grouped states as Planning, Translating, and Reviewing.
  • A.3 Cognitive Process Prediction Results: The LLM using logs plus document context achieved the highest performance across all reported metrics, followed by document-only LLM and log-only LightGBM models.The LLM approach operated in a near zero-shot manner without gradient-based training.
  • A.3 Cognitive Process Prediction Results: The editor exposed tools for reading current content, retrieving bookmarked outputs, and asking users questions through choice, text, or slider interfaces.These functions supported interaction with generated outputs and user-provided information.

C User Study Data

The study involved 21 manufacturing-company engineers with substantial familiarity with LLMs and varied experience using them for writing-related tasks.

  • 21 engineers from a manufacturing company participated in the study.
  • The questionnaire covered LLM use for writing and tasks including drafting, tone and style consistency, grammar, and persuasiveness or structure review.

C.2 Interview Guide (10–15 min.)

The interview guide examined participants’ impressions, support-seeking, ownership, enjoyment, and preferences regarding proactive writing assistance.

  • Participants were asked to reflect on each stage of the writing process and whether they completed everything within the available time.
  • The guide asked whether proactive support changed participants’ thinking, writing process, or overall experience.
  • Participants evaluated whether proactive support was intuitive and compared on-demand with proactive support.
  • Additional questions addressed helpful features, support-seeking points, Bookmark use, ownership of content, enjoyment, desired support types, and willingness to use the system again.
  • The study included 21 participants whose ages ranged from their 20s to 50s or higher.

C.2.1 Results.

Study measures assessed perceived ability, control, ownership, and whether AI use diminished participants’ creative abilities.

  • The study used a 10-point Likert scale to assess whether AI significantly improved writing efficiency.
  • Control measured whether the creative process remained under the participant’s control.
  • Ownership measured whether the tool enabled content creation based on participants’ own intentions and ideas.
  • Deprivation measured whether AI use made participants feel that their creative abilities were being diminished.

C.4 Study 1 Statistical Results

The study’s statistical results were reported through filtered metrics, while the system implementation coordinated cognitive-process-based support through a Main Agent and Functional Agents.

  • Statistical Results: Table 7 reports filtered metrics meeting either |R_BC| > .6 or CLES > .6 in two-sided Wilcoxon signed-rank tests.
  • System Implementation: Functional agents followed rules governing concise suggestions, clarification questions, editor actions, bookmarking, and delivery of longer outputs.
Loading 2608.30424v1…