DraftTrace: A Multi-View Analytics Environment for AI-Integrated Writing

1 Introduction

• Generative AI lets students use assistance at stages ranging from understanding concepts and brainstorming to revising, co-writing, or generating a complete response.This changes the relationship between the final product and students’ engagement with the material, complicating instructors’ assessment of learning through writing.Source: §1 · p. 1
• Earlier writing-process measures such as pauses are ambiguous, and AI interaction adds another possible explanation for a pause.A pause may reflect planning, reflection, revision, or interaction with an AI system.Source: §1 · p. 1
• Existing tools offer subsets of product, process, and AI-assistance capabilities, but treat these as separate signals rather than complementary views of one writing activity.Source: §1 · p. 1
• DraftTrace jointly captures the writing process, resulting product, and students’ interactions with an integrated AI assistant, organizing them into instructor-facing views at multiple levels.The three views characterize the final artifact, how it develops over time, and how students engage with AI during writing.Source: §1 · p. 1

2.1 Writing Analytics

• Product-level writing analytics assess dimensions such as style, relevance, organization, cohesion, and coherence, and have been used for scoring and feedback.Pande et al. (2026) combine stylometric analytics with an LLM to generate feedback at scale.Source: §2.1 · p. 2
• Keystroke features have been associated with essay scores and can signal essay quality early, but process features vary across tasks and contexts.Prior work links longer writing time and shorter, less variable within-word intervals with higher scores; other work reports variability by grade level, proficiency, and school setting.Source: §2.1 · p. 2

2.2 AI-Mediated Writing

• AI-mediated writing studies examine ownership, AI-use categories, and how students accept, modify, or reject AI suggestions.Together, these studies report substantial variation in how students engage with and use AI during writing.Source: §2.2 · p. 2
• Recent studies combine subsets of writing signals for particular tasks, including process with product information and interaction traces in AI-assisted writing.These approaches suggest the value of considering product, process, and interaction together for a comprehensive view of student writing.Source: §2.2 · p. 2

3 DRAFTTRACE

• DraftTrace’s design goals include aligning process, product, and interaction information, configuring AI support, comparing submissions over time and across a class, and fitting existing workflows.The paper notes that representative platforms offer different subsets of these capabilities.Source: §3 · p. 2

3.1 System Architecture

• The web application uses a shared document schema and analysis modules across browser and server, so instructor playback and server-computed metrics derive from identical logic.The architecture uses a React front end, Express back end, and PostgreSQL store.Source: §3.1 · p. 2
• DraftTrace records timestamped document changes and input channels, then replays events to reconstruct the submitted document and label how its text entered.Events include typing, paste, drag-and-drop, undo/redo, and assistant insertion; periodic snapshots bound replay cost, while queued batches and sequence numbers support retries on unreliable networks.Source: §3.1 · pp. 2–3
• Table 1 compares representative writing platforms across process, product, interaction, assistance, configurability, and editor capabilities.Source: Table 1 · p. 3
• The integrated assistant is proxied through the server, which saves student prompts and transcripts and tags inserted assistant text to connect interaction, process, and product views.Instructors can enable or disable the assistant per assignment, and its system prompt directs it to support understanding rather than write answers.Source: §3.1 · pp. 3–4

3.2 Instructor Interface

• The instructor interface supports quiz creation and analytics review, with controls for questions, instructions, time, and length constraints.For each submission, it summarizes process measures and presents a chronological session timeline; class-level and longitudinal views are also provided.Source: §3.2 · p. 4, Figure 2 · p. 4

3.3 Student Interface

• Students compose responses in the editor and can access the integrated AI assistant when the instructor enables it.Source: §3.3 · p. 4

4.1 Data Collection

• The study compared 81 classroom sessions, 81 automatically entered LLM responses, and 11 manually typed copy-typing sessions.Classroom participants answered two related questions in 15 minutes; the settings varied how responses were formulated and entered.Source: §4.1 · pp. 4–5

4.2 Analysis

• LLM-formulated responses were harder to read and used longer words than classroom responses, whether entered automatically or by copy-typing.Median Flesch Reading Ease was 52.8 for classroom, 38.4 for automated, and 39.9 for copy-typed responses; median word lengths were 4.67, 5.18, and 5.40 characters, respectively.Source: §4.2 · p. 6, Table 2 · p. 6
• Process measures distinguished entry patterns: classroom writers revised most and left the editor, while automated entry had no idle time or departures.Copy-typing involved a median 33 wpm and 5.4 revisions per 100 characters, compared with automated entry at 49 wpm and 2.7 revisions per 100 characters.Source: §4.2 · p. 6, Table 2 · p. 6
• Together, product and process measures distinguish copy-typing: its product resembles automated writing, while its entry process differs.Neither view alone captures this combination.Source: §4.2 · p. 6
• Assistant use varied by writing stage: early first prompts sought clarification or direct help, while late first prompts almost always sought verification.Collaboration appeared only among students who issued three or more prompts.Source: §4.2 · p. 6, Table 3 · p. 6

5 Instructor Survey

• Nine of ten respondents encountered suspected inappropriate AI use at least sometimes, but only one felt very confident in their current investigation process.The findings indicate a gap between encountering questions about how work was produced and confidence in available investigative information.Source: §5 · p. 7
• Seven of ten respondents rated writing-process visibility and final-submission analysis very valuable; six of ten did so for AI-access control, interaction visibility, and playback.Source: §5 · p. 7
• Acceptance of writing/editing activity, detailed typing patterns, and AI processing of student writing was more often conditional than general.Six of ten respondents considered copy/paste activity and AI-interaction history generally acceptable to collect.Source: §5 · p. 7
• Inaccurate or misleading results could discourage adoption for nine of ten respondents, while four of ten cited lack of transparency as a concern.Source: §5 · p. 7

6 Discussion

• Aligned product, process, and interaction information lets researchers observe how writing and AI-assistance use evolve during and across assignments.Source: §6 · p. 7
• Because assistance can be configured per assignment, DraftTrace supports controlled studies of when assistance should be provided and what form it should take.Adapting assistance based on observed writing and interaction patterns is proposed as future work.Source: §6 · p. 7

7 Limitations

• DraftTrace captures only in-environment activity, and the case study covers one short writing activity in one graduate-level course.Patterns may vary across tasks, student populations, and instructional settings.Source: §7 · p. 7

• Students should be informed about what writing and AI-interaction information is collected, how it is processed, and who can access it.Source: §8 · p. 7
• Writing traces and AI conversations should be treated as sensitive educational data.Source: §8 · p. 7
• Sensitive trace data should have appropriate access and retention controls.Source: §8 · p. 7

• Figure 7(a) presents an overall analytics summary for an individual submission.Source: Figure 7 · p. 8, A Interfaces · p. 8
• Figure 7(b) presents a longitudinal view of a student’s writing across submissions.Source: Figure 7 · p. 8, A Interfaces · p. 8
• Figure 7(c) compares class-level writing patterns for the assignment.Source: Figure 7 · p. 8, A Interfaces · p. 8