Source-linked AI summary
Inkitiate: Initiating and Steering Agent Assistance through Ink in a Spatial Workspace
Liangwei Wang, Nan Chen, Luna K. Qiu, Zilong Wang, Fugee Tsung, Yuyu Luo, Yuqing Yang
TL;DR
Situated ink requests combine marks, spatial relations, selected materials, and evolving context, yet how to steer agents across knowledge work remains underexplored. Inkitiate lets users submit ink with relevant materials, inspect agent interpretation and execution, and intervene as tasks unfold; in an 18-participant study, users reported favorable usability and low workload and reused retained ink and materials in further inquiries.
Problem
How situated ink can steer agents across knowledge work remains underexplored, particularly when requests involve heterogeneous materials, prior results, and workspace context.
Source: §1 · p. 2, §2.2 · p. 3, §2.3 · p. 4Method
Inkitiate is a spatial workspace where users submit ink with optional selected canvas objects, inspect agent understanding and progress, and guide or abort execution.
Source: §1 · p. 2Results
In a study with 18 participants, users reported favorable usability and low perceived workload, and reused retained ink and materials to develop further queries.
Source: §1 · p. 2Takeaways & Limitations
The findings highlight ink’s potential to make agent assistance part of how people think and work with materials.
Source: §8 · p. 16The evaluation was a single descriptive session with 18 students experienced with digital handwriting or visual canvases, so relative benefits over alternatives remain open.
Source: §7.3 · p. 16
Abstract
from arXiv · showhide
Handwriting and sketching help people develop ideas around information. As this work unfolds, users may seek assistance with finding, understanding, and organizing materials. We investigate how ink can initiate and steer agent assistance in this process. An eight-participant formative study explored desired assistance and interaction expectations, informing three connected activities: bringing materials in, thinking with materials, and shaping the space. We present Inkitiate, a spatial workspace for ink-based agent assistance in knowledge work. Through ink, users retrieve external materials, build on accumulated content and results, and reorganize the canvas, while inspecting and guiding agent actions. In a study with eighteen participants, users reported favorable usability and low perceived workload, and used retained ink and materials to develop further ink queries. An ink-query vocabulary and cases reveal how users combine linguistic and graphical expressions to direct agents. These findings inform ink-based agent interaction design for expressive requests and continued work with materials.
1 Introduction
Inkitiate explores how ink can initiate and steer agent assistance as people gather, interpret, and organize materials in spatial workspaces. It combines formative findings about cumulative assistance with a workspace designed for ongoing, user-guided work.
1 Introduction
- An eight-participant formative study found that users wanted cumulative assistance, using prior results and existing materials to develop subsequent inquiries rather than treating each request as an isolated exchange.Participants expected selections and spatial marks to delimit inputs and result placement, and wanted to inspect, correct, or stop misinterpreted actions.Source: §1 · p. 2
- Inkitiate makes ink queries explicit, interprets them using bounded spatial context and relevant history, and returns editable results that remain available for reuse.Users can inspect the agent’s interpretation, included materials, and progress, respond to clarification, or abort execution.Source: §1 · p. 2
- 18 participants reported favorable usability and low perceived workload, while their experiences highlighted reuse of retained ink and materials, challenges with local targets, and the importance of output form.An ink-query vocabulary and qualitative cases characterize how people express intentions and use ink to guide iterative creation, revision, and visual organization.Source: §1 · p. 2
- The paper contributes a formative account of three connected activities and interaction expectations, a workspace design, and an empirical account of ink-query vocabulary and interaction possibilities.The activities are bringing materials in, thinking with materials, and shaping the space.Source: §1 · p. 2
2 Related Work
Prior work establishes ink as both content and instruction, spatial workspaces as support for evolving sensemaking, and interaction techniques for forming and steering AI instructions. The paper investigates how situated ink can combine these roles across knowledge work while keeping agent tasks bounded, inspectable, and steerable.
2.1 Ink as Content and Instruction
- Ink research spans annotating and interpreting materials, using sketches to generate visual content, and expressing contextual instructions for actions on existing content.Examples include sketch-guided generation and code or document tasks [11, 17, 56], as well as ink-based queries and edits [31].Source: §2.1 · pp. 2–3
2.2 Evolving Workspaces for Knowledge Work
- Spatial workspaces preserve and reorganize materials as understanding develops, while newer systems make prior results and context reusable in subsequent work.The paper identifies a gap in situated ink assistance across acquiring information, reasoning over selected materials, and creating or reorganizing workspace content.Source: §2.2 · p. 3
2.3 Interaction-Augmented Instructions and User Control
- Interaction-augmented instructions and user-control systems support forming AI instructions from visible materials and inspecting or intervening in agent behavior.The paper frames the underexplored challenge as keeping multi-step tasks involving free-form marks, spatial relations, selected materials, and evolving context bounded, inspectable, and steerable.Source: §2.3 · pp. 3–4
3 Design Foundations
An initial pen-first probe and formative study examined how people would seek and guide agent assistance in ongoing knowledge work. Findings about activities and interaction expectations informed four design goals for Inkitiate.
3.1.1 Initial Prototype.
- The initial prototype used ink to retrieve or generate canvas resources, query or change existing objects, and return results after each agent run.The study recruited eight participants with prior digital handwriting experience; participants explored open-ended tasks, thought aloud, and were interviewed, with recordings and transcripts analyzed through iterative open coding.Source: §3.1.1 · p. 4, §3.1.2 · p. 4, §3 · p. 4
3.1.3 Findings: Activities and Interaction Expectations.
- Participants identified three recurring activities: bringing materials in, thinking with materials, and shaping the space into editable visual structures.The activities can overlap and recur as results become material for further queries; Figure 2(b) relates them to four interaction expectations that informed DG1–DG4.Source: §3.1.3 · p. 4, Figure 2 · p. 5
- Participants wanted brief ink expressions combined with spatial references and explicit object selection to specify both the requested action and its targets.Spatial proximity alone could fail when relevant materials were distant, so participants also wanted explicit selection and bounded regions within objects.Source: §3.1.3 · p. 4
- Participants expected context to follow an evolving line of work, preserving relevant materials, spatial relationships, and history without including unrelated canvas content.Their examples reflected concerns about losing a paper-request relationship and, conversely, using too much context; these interpretations did not establish the technical causes.Source: §3.1.3 · p. 4
- Participants wanted to inspect agent interpretation and progress, then clarify, correct, or stop execution without restarting the larger task.They suggested status feedback, opportunities to revise or interrupt a misdirected action, and selectable options when the agent needed limited-choice information.Source: §3.1.3 · pp. 4–5
- Participants expected write-back to extend local relationships or reorganize clutter into clear, editable structures that they could refine.They used prior results to repair an answer and pursue further inquiries, and wanted placement and form to fit the canvas’s nonlinear organization.Source: §3.1.3 · p. 5
3.2 Design Goals
- DG1–DG4 call for ink combined with canvas materials, bounded continuity of relevant context, inspectable and steerable execution, and persistent results that extend or reorganize visual structure.The goals include provenance that remains attached as results are moved, edited, organized, or reused.Source: §3.2 · pp. 5–6
4 Inkitiate
Inkitiate is a spatial workspace for guiding agent assistance through free-form ink in ongoing knowledge work.
4 Inkitiate
- Inkitiate supports an iterative process by making task interpretation inspectable, allowing intervention during execution, and returning editable, source-bearing results to a persistent canvas.Source: §4 · p. 6
4.1 System Walkthroughs
The walkthroughs show Amy using ink queries to retrieve materials, develop them into structured results, and continue working with those results through later queries and reorganization.
4.1.1 Exploring The Odyssey.
- Amy retrieves an Odyssey introduction, maps key episodes along a sketched route, and creates an editable comparison of Odysseus’s alternatives.An activity card exposes the agent’s interpretation and current execution for inspection and steering.Source: §4.1.1 · p. 6
4.1.2 Reading foundational LLM papers.
- Amy retrieves foundational LLM papers, organizes them by publication year and research direction, and queries selected papers to compare a revised scaling conclusion with its predecessor.Retrieved materials and generated results remain available for subsequent ink queries and reorganization.Source: §4.1.2 · p. 6
4.2 Interaction Support for Inspecting and Steering Ink-Initiated Agent Tasks
Inkitiate bounds each ink-initiated task, exposes its state for inspection and intervention, returns consequential choices to users, and writes reusable results into the canvas.
4.2.1 Bounding Task Input through Ink and Canvas Objects.
- Users explicitly submit free-form ink, optional sigils, and selected canvas objects; resource sigils bind objects, while capability sigils route tasks to named capabilities.Without a capability binding, the system infers one from the query and context; selected objects can serve as task targets, supporting materials, or both.Source: §4.2.1 · pp. 6–7, Figure 3 · p. 7
4.2.2 Exposing and Controlling Task State.
- The activity card exposes understood intent, included materials, task progress, and elapsed time, and lets users stop a task that is unnecessary or going in the wrong direction.In the map example, the user stops an interpreted generation task, binds resource retrieval, and selects among returned candidate maps.Source: §4.2.2 · p. 7, Figure 4 · p. 8
4.2.3 Returning Consequential Decisions to the User.
- When missing information could materially change task direction, Inkitiate pauses and asks users to choose a suggested interpretation or provide a staged custom ink response.The lotus example shows both selecting a map-marking action and writing a custom question interpreted using the drawing and journey map.Source: §4.2.3 · pp. 7–8, Figure 5 · p. 8
4.2.4 Writing Results Back to the Canvas.
- Task-scoped canvas operations write results as persistent, reusable objects that users can edit, move, reorganize, and submit in later tasks.Spatial cues guide placement; structured results can be composed into arrangements, while embedded source indicators remain attached when generated text is moved or reused.Source: §4.2.4 · pp. 8–9, Figure 6 · p. 9
4.3 Context Engineering and Agent Execution
Inkitiate compiles a bounded task context from submitted ink and objects, nearby read-only materials, and relevant task history, then controls how agent results affect the persistent canvas.
4.3.1 Constructing Bounded Task Context.
- The compiler combines the current ink query, explicitly submitted canvas evidence, and task continuity, while making nearby materials available as read-only context rather than direct inputs or operation targets.The context includes relevant history so a new query can continue an established line of work without unrestricted access to the entire canvas.Source: §4.3.1 · p. 9
4.3.2 Coordinating Agent Execution and Write-Back.
- The agent uses built-in and task-scoped tools to interpret and execute tasks, but structured results must pass through a result boundary before canvas operations can modify persistent objects.The coordinator exposes public task state and relays stop requests and clarification responses; separating execution from canvas mutation bounds what results may change.Source: §4.3.2 · p. 9
4.4 Implementation
Inkitiate was implemented as an iPad Safari web application connected to a general-purpose agent runtime. Scenario-based validation found that all 102 runs across 34 cases met the stated stability requirements.
4.4 Implementation
- The standalone web application used a TypeScript and React canvas, a Node.js host, and an existing general-purpose Codex runtime.The study used GPT-5.6-Sol with low reasoning effort; task-scoped canvas tools supported reading and layout preview without directly modifying the persistent workspace.Source: §4.4 · p. 10
Scenario-Based System Validation
- 102 end-to-end runs across 34 ink-query cases completed successfully and met stability requirements.Validation assessed workflow behavior from query submission and context construction through execution and canvas write-back, not model capability or user performance.Source: §4.4 · p. 10
5 User Study: Learning and Appropriating Ink-Based Agent Interaction
The study examined how participants learned to formulate ink queries and use them for increasingly complex knowledge work, and how ink supported self-directed goals. Eighteen participants completed prescribed Reproduction tasks and open-ended Freeform Creation, followed by questionnaires and interviews.
5 User Study: Learning and Appropriating Ink-Based Agent Interaction
- The study asked how participants learn ink queries for increasingly complex tasks and how self-directed use reveals workflows, possibilities, and limitations.Eighteen participants completed prescribed Reproduction tasks and open-ended Freeform Creation.Source: §5 · p. 10
5.1.1 Participants and Apparatus.
- Eighteen university-affiliated participants with prior digital handwriting or visual-canvas experience used the same browser-based system in person or remotely.Ten participated in person and eight remotely; remote participants provided their own stylus-enabled devices.Source: §5.1.1 · p. 10
5.1.2 Procedure.
- A tutorial familiarized participants with canvas operations, while the study collected recordings, logs, artifacts, questionnaires, and interviews for descriptive and qualitative analysis.Questionnaires were administered only after both phases, so they measured overall experience rather than differences between phases.Source: §5.1.2 · pp. 10–11, §5.1.3 · p. 11
- Reproduction progressed from initiating requests to retrieving materials, making sense of them, and structuring them; Freeform Creation let participants pursue their own topics without a prescribed sequence or output.The final visual-structuring task was presented as a bonus; analysis focused on completing shared interaction capabilities rather than reproducing a fixed canvas.Source: §5.1.2 · p. 10
5.2 Study Results
Participants completed simpler prescribed interaction tasks more often than the advanced visual-structuring task, and reported favorable usability and low workload. In self-directed work, they reused retained materials and ink, while fine-grained references and output form revealed boundaries on continued work.
5.2.1 Task Performance and Interaction Experience.
- 18/18 participants initiated and steered tasks and brought materials into the canvas, 16/18 completed sensemaking, and 7/18 completed the advanced visual-structuring task.Three partially completed visual structuring; eight did not, with time constraints or uncertainty about how to begin affecting some L3 and L4 participation.Source: §5.2.1 · p. 11
- Adapted SUS averaged 5.93/7, and all five adapted NASA-TLX workload dimensions were below the scale midpoint.SUS learnability averaged 6.33; NASA-TLX mental demand averaged 2.22.Source: §5.2.1 · pp. 11–12
5.2.2 Developing Work through Retained Ink and Materials.
- 125 of 181 Freeform Creation entries (69.1%) reused prior outputs, and 30 combined multiple selected or linked objects in one request.Figure 8 summarizes case topics, query counts, prior-output reuse, and PDF, web, or image inputs.Source: §5.2.2 · p. 12, Figure 8 · p. 12
- Participants reused retained outputs and editable instructions by changing bindings, wording, or conditions, while some preferred retaining keywords rather than longer instructions.Speech was requested as a complement for longer instructions, not evaluated as a feature.Source: §5.2.2 · p. 12
5.2.3 Referring to Fine-Grained Targets within Canvas Objects.
- Ink referred to targets finer than whole-object bindings, but free-form interpretation was not always reliable and lacked dedicated support for specifying or confirming scope.The reported mismatches could not be attributed to reference interpretation versus action execution.Source: §5.2.3 · p. 12
5.2.4 Seeking Outputs That Support Further Work.
- Participants wanted outputs whose structure, constituent elements, and evolution remained available for inspection and revision, not only information in a single image or dense text.Preferences varied: one participant considered the explanations appropriately detailed.Source: §5.2.4 · pp. 12–13
- Participants commonly combined linguistic requests with depictive, referential, relational, or spatial ink rather than using these forms independently.Figure 9 presents the compositional roles of ink in Freeform Creation.Source: Figure 9 · p. 13
6 Case Gallery: Ink-Initiated Agent Interaction in Practice
The case gallery shows how participants combined written requests with graphical and spatial marks, then used successive queries to develop images and organize materials into visible structures. These cases illustrate varied ways ink could guide agent actions and continued work.
6.1 Ink Query Vocabulary
- Participants combined words with circles, sketches, connections, or paths to identify targets, indicate appearance, express relationships, or place answers.A single mark could serve different functions, such as pointing to a detail, connecting ideas, or directing an answer toward a space.Source: §6.1 · p. 13
- Across 181 entries, linguistic ink appeared in 97.8%, referential marks in 39.2%, depictive ink in 9.9%, spatial/layout ink in 9.4%, and relational ink in 6.1%.In 76 entries ink served at least two functions; writing plus target-marking was the most common assessed combination (46 of 150).Source: §6.1 · p. 13
6.2.1 Exploring image creation through ink.
- Participants iteratively developed image content, composition, and style using spatial sketches, local annotations, successive character revisions, and hand-drawn references.Figure 10 depicts a coastal-village sequence, character edits, and avatar refinement.Source: §6.2.1 · p. 13, Figure 10 · p. 14
6.2.2 Creating visually structured outcomes.
- Participants organized collected or generated information as timelines, connected scenes, card-like papers with branches, or radial arrangements around a shared topic.These arrangements made temporal order, individual materials, and conceptual relationships visible on the canvas.Source: §6.2.2 · pp. 13–14
7 Discussion
The discussion frames ink as a way to express requests through the materials and visual structures users are already developing. It considers how this supports continued work with agent outputs while identifying limits in the study setting and ink interpretation.
7.1 The Distinctive Value of Ink for Steering Agents
- Ink combines brief instructions with depictions, material references, and relational marks, letting users express both existing references and intended forms or arrangements.Selected materials and prior results can supply what brief expressions leave implicit.Source: §7.1 · p. 14
- Retained annotations and sketches can develop ideas and express assistance requests across acquiring, examining, and organizing materials; users may supplement them with speech or typed detail.The study connects this overlap to participants’ reuse of annotations and keywords alongside speech.Source: §7.1 · p. 14
7.2 Supporting Continued Work in Ink-Initiated Agent Interfaces
- Participants reused outputs to ask further questions, add changed circumstances, and revise emerging content, re-entering the process by annotating, connecting, or sketching how results should develop.Examples included annotated timelines, thematic branches, card groups, a topic-centered constellation, and a spatial narrative sequence.Source: §7.2 · pp. 14–15, Figure 11 · p. 15
- Output form matters for further work: responses need readable relationships and interaction suited to whether content is assembled from elements or shown as an image.Participants also marked local details within objects even when responses did not maintain that local scope.Source: §7.2 · pp. 15–16
- Inspecting and correcting the interpreted task can keep users’ judgments about current materials connected to an agent’s next action as goals change.Retaining versions alongside modification intentions could help users compare outputs against what they sought to achieve.Source: §7.2 · p. 16
7.3 Limitations and Future Work
- The evaluation was a single descriptive session with 18 students experienced with digital handwriting or visual canvas tools, leaving comparative benefits and sustained use unexamined.The authors propose broader longitudinal field deployments to explore other populations and everyday work.Source: §7.3 · p. 16
- Inkitiate relies on contextual model interpretation, which may diverge from users’ intended meanings; the authors identify contextual understanding and personalized learning as future directions.Personalization could learn from users’ corrections and repeated use.Source: §7.3 · p. 16
8 Conclusion
This work identified activities and interaction expectations that informed Inkitiate, a workspace for bringing materials in, thinking with them, and shaping the space. In an 18-participant study, users reported favorable usability and low workload; findings describe how retained materials and ink can support further inquiries and ongoing work.
8 Conclusion
- A formative study identified three connected activities that informed Inkitiate: bringing materials in, thinking with materials, and shaping the space.The workspace retains acquired materials and subsequent results for further work.Source: §8 · p. 16
- In a study with 18 participants, Inkitiate received favorable perceived usability and low workload ratings, while findings highlighted retained materials, local-reference challenges, and output-form concerns.An ink-query vocabulary and cases characterized intention expression and illustrated potential for creation, revision, and visual organization.Source: §8 · p. 16