On the Design of AI-powered Code Assistants for Notebooks

1 INTRODUCTION

• Code assistants typically use programming context to suggest code that users select, adapt, and use to continue programming.This interaction can reduce boilerplate, increase perceived productivity, and guide users through unfamiliar tasks.Source: §1 · p. 1
• Notebook work interleaves code with markdown, tables, and plots, and users execute loosely structured snippets in nonlinear, iterative workflows.Without accounting for these differences, a purely code-based assistant interface can make it challenging to specify desired context, understand suggestions, and adapt them.Source: §1 · p. 1
• The design-space analysis surveys notebook systems that generate code and organizes design concerns into interface components and relationships between them.The components include gestures, model artifacts, disambiguation, and refinement; relationships include code context, provenance, specialization, and customizability.Source: §1 · pp. 1–2
• The authors interviewed 15 professional data scientists about designs for context specification, suggestion disambiguation, result adaptation, and code provenance across data-analysis and visualization tasks.The designs were based on the structure of the design space.Source: §1 · p. 2
• Participants were unanimously enthusiastic about notebook code assistants, highlighting polite, sufficiently prominent interfaces, approachable task-specific linters, and varied interface forms.These preferences address user agency and flow while supporting different tasks throughout the notebook.Source: §1 · p. 2

Code Generation

• Code-generation models learn program concepts from large code corpora and support broad languages and tasks, but can handle specific scenarios poorly; prompt engineering or fine-tuning may address this.Generation can be based on high-level specifications such as demonstrations, input-output examples, natural language, or partial implementations.Source: §2 · p. 2

Interacting with Code Assistants

• Assistant-interface work explores ways to control context, refine prompts, and guide synthesis, including code fences, live value displays, and prompt experimentation.These patterns inform the paper’s exploration of possible interface elements for notebooks.Source: §2 · p. 2
• Studies of Copilot describe acceleration and exploration modes, similarities to search and pair programming, and challenges involving learning, code quality, trust, and editing generated code.The paper extends this research by examining assistants in a particular medium.Source: §2 · p. 2

Intervening in Computational Notebooks

• Notebook interventions include history adaptation, program slicing, graphical code specification, spreadsheet-style data manipulation, live programming, and workflow recommendations.These systems seek to improve notebook usability and end-user programming.Source: §2 · pp. 2–3

Code Suggestions

• Code suggestions can use autocomplete or search-like interfaces and varied ways of specifying intent, but this paper excludes purely search-based interfaces, limiting its approach.Prior work reports that search and generation can benefit different tasks [94].Source: §2 · p. 3

3 DESIGN SPACE

• A maximum-variation survey of 14 systems informed the design space, which was extended to include underexplored designs and excludes search-based systems.The authors describe the space as descriptive and generative, not evaluative, and limit their analysis to notebooks.Source: §3 · p. 3, §3.2 · p. 6

3.1 The Design Space

• The design space models a specification-refinement loop across assistant interactions and relationships between assistants and other notebook components.Systems can be characterized by selecting values along dimensions in these two categories.Source: §3.1 · p. 3, §3 · p. 3

3.1.1 Interactions.

• Interaction dimensions cover gesture targets, gesture composition, disambiguation of candidate results, and refinement of the user’s specification.Composition may be one-shot, iterative, or nonlinear; refinement ranges from restarting to bidirectional updates, which are rare because synchronization is complex.Source: §3.1.1 · pp. 4–5

3.1.2 Interface Relationships.

• Interface relationships include assistant location and context, specialization, provenance history, and user customization.Locations range from ambient or cell-level to application-wide; customization can use abstractions or APIs, but may complicate interface design.Source: §3.1.2 · pp. 5–6
• The design space omits features such as secondary notation and input modalities, and does not model the social context surrounding assistants.The authors note that social roles of code can affect which features users find useful.Source: §3.1.2 · p. 6

3.2 Model validation

• Researchers experienced with Copilot responded favorably to the design space, providing a coarse, subjective form of validation.The authors also position the space as descriptive and generative rather than evaluative, and intend it to dovetail with related analyses.Source: §3.2 · p. 6

4 INTERVIEW STUDY

• The study focused on four underexplored areas: code context, disambiguation, adaptation, and provenance exploration.Other topics, including customization, were not probed with specific designs but arose in interviews.Source: §4 · p. 6

4.1 Methodology

• Fifteen participants were recruited from a data-driven software company based on notebook and assistant experience, though three lacked AI-assistant experience.Participants had varied experience with assistants, notebooks, and data science; all were 25–34 and held at least a CS-related master’s degree.Source: §4.1 · p. 6
• Video interviews used slide-based prototypes and collected no quantitative design scores because the semi-structured format covered topics unevenly.Interviews averaged 73 ± 10 minutes; transcripts were open-coded and themes iterated by the research team until saturation.Source: §4.1 · p. 6

4.2 Study design

• Context probes compared targeted-cell, inline, and application-level interfaces spanning implicit to explicit specification of the model’s code context.The study also presented multiple ways to display candidate code and effects, including code-plus-output and summary views.Source: §4.2 · pp. 6–7
• Adaptation probes included linter-style warnings, snippet skeletons, token confidence, and token alternatives; provenance probes tested summaries, cell markers, and interaction logs.The authors distinguish adapting generated code from asking the assistant to produce better code; provenance designs explored whether interactions should be documented.Source: §4.2 · p. 7

5 INTERVIEW STUDY ANALYSIS

• All participants considered adapting code assistants to notebooks valuable.Participants described strong potential for these tools in data science.Source: §5 · p. 7
• Most topics had no universally preferred design; participants nearly unanimously favored Code and Output for exploring suggestions, while other features drew varied preferences.Context interfaces differed by preferences for clutter, politeness, and locality; refinement strategies other than Token-confidence were valued for different reasons, and assistant-use artifacts were often unnecessary.Source: §5 · p. 7
• The findings concern data scientists’ views of code assistants in notebooks and should not be treated as general findings about other users or contexts.The authors note that some results may align with expectations in other interface contexts, but their study sought opinions only from this user group and setting.Source: §5 · p. 7

5.1.1 Context and Interfaces.

• Most participants did not want to manually specify context, while some knowledgeable about code-generating models valued control over included cells or variables.Some also wanted context-free suggestions, similar to search.Source: §5.1.1 · p. 8
• Participants favored different context interfaces for reasons including familiarity, immediacy, locality, separation, and concerns about clutter or attention splitting.Some disliked Wizard popups for interrupting their train of thought, while others valued their locality.Source: §5.1.1 · p. 8
• Participants often used assistants for efficiency rather than exploration, creating tension between notebook analysis and the shorter interaction loops needed for acceleration.Analysis may require alternating between code and outputs such as charts or spreadsheets to check correctness.Source: §5.1.1 · p. 8
• Different user preferences suggest offering multiple places in the interface to interact with assistants, as with ambient and menu-triggered spelling checkers.Source: §5.1.1 · p. 8

5.1.2 Feature Prominence.

• Controls face a prominence trade-off: subtle features can go undiscovered, while prominent or eager suggestions may irritate users and be disabled.Some participants preferred potentially annoying displays to appear on demand.Source: §5.1.2 · p. 9
• Effective assistants must balance respecting user agency with making their features discoverable enough to encourage use.Source: §5.1.2 · p. 9
• Familiarity and novelty shaped expected feature use, but participants cautioned that a feature’s value must outweigh its novelty.Expectations can be subverted in useful, surprising ways, but may also lead to annoyance and disuse.Source: §5.1.2 · p. 9

5.2 Relationship with surroundings

• Participants’ notebooks served exploration, development, presentation, and shared or solitary work, shaping desired contextual and domain-specific affordances.These varied uses make notebook context relevant to interface design.Source: §5.2 · p. 10

5.2.1 Provenance.

• Participants saw provenance documentation as low priority, often trusting colleagues’ notebooks and treating code as a means to insights rather than a deliverable.The authors suggest assistants may become commonplace when they remain ignorable, like spell checkers.Source: §5.2.1 · p. 10

5.2.2 Search.

• Participants wanted code search for reusable or organizational snippets, but also context and reasoning to assess suggestions; the authors caution that metadata may increase automation bias.Some wanted saved, malleable snippet libraries or context-adapted code beyond model training data.Source: §5.2.2 · p. 11

5.2.3 Relationship to domain.

• Because data scientists work across environments with distinct notebook affordances, the authors suggest browser-level assistants that follow familiar patterns.P13 uninstalled Copilot because it was unavailable in all her work environments.Source: §5.2.3 · p. 11
• Participants differed on whether assistants should adapt to personal style or enforce team conventions and best practices, which the authors recommend offering cautiously given automation bias.Some wanted configurable naming or learned preferences; others preferred best practices or worried about staying in a bubble.Source: §5.2.3 · p. 11
• Participants frequently wanted notebook linters for issues such as out-of-order execution, side effects, and inefficient repeated cells, alongside schema- and data-science-specific guidance.Warnings must provide consistent utility; otherwise, participants said linting could be annoying.Source: §5.2.3 · p. 11

5.3.1 Points of control.

• Participants tied trust to control over code and described both efficiency-oriented and exploratory uses, though the study relies on self-reports rather than direct observation.Some used assistants to learn new tasks or coding patterns, while others distrusted them for exploration.Source: §5.3.1 · p. 12
• Participants wanted control over assistant parameters, but choices could overwhelm; hiding them in expert menus might make them undiscoverable.Requested controls included suggestion length, adventurousness, model choice, and a learning-versus-completion toggle.Source: §5.3.1 · p. 12
• Readable generated code and adaptation interfaces mattered because participants felt less in control when suggestions were hard to understand; preferences among annotation designs varied.Some favored code skeletons for agency over boilerplate or familiar linter-style feedback, while others found numeric confidence confusing; the authors favor ambient or actively useful annotations.Source: §5.3.1 · p. 12

5.3.2 The effects of knowledge.

• Prior experience shaped expectations: participants with LLM experience tolerated errors more, while limited understanding could make users abandon assistants after a few bad suggestions.Participants identified popular APIs as more reliable than esoteric or rapidly changing ones; the authors suggest communicating model capabilities.Source: §5.3.2 · pp. 12–13

5.3.3 Verification.

• Verification needs varied with stakes, runtime, and perceived quality, and participants described checking documentation or testing suggestions in a sandbox.Some wanted validity checks or low-friction previews, while others considered ordinary notebook cells sufficient.Source: §5.3.3 · p. 13
• Participants used alternative suggestions to verify outputs and explore approaches, often preferring code-and-output comparisons for visualization and other tasks.Some worried about the computational cost of generating alternatives; selected-option execution was suggested as a hybrid.Source: §5.3.3 · p. 13
• The authors suggest combining lightweight alternatives with richer code-and-output exploration, and integrating previews may shorten accept-validate-repair, tuned to different usage modes.Token alternatives were viewed as useful for lighter-weight adjustments, while code and output supported broader exploration.Source: §5.3.3 · p. 13

6 DISCUSSION

• Task-specific assistants, disambiguation, and polite interaction are promising design directions, while ambient interfaces trade low friction for possible opacity and limited domain specificity.The authors suggest targeting tasks users do not view as core work, such as formatting or visualization.Source: §6 · p. 13

6.1 Limitations and Generalizability

• The findings are local to notebook assistants: Jupyter-like design paradigms do not establish generalizability to other interfaces, even when some observations may align with IDEs.The interview takeaways are grounded in notebook-based data-science tasks.Source: §6.1 · p. 13, Table 1 · p. 14
• The design-space analysis excluded search-based systems, leaving comparison and potential blending of search and generation interfaces for future study.The paper notes that search may yield mixed-media while generation yields fixed-media.Source: §6.1 · pp. 13–14
• Slide-based probes, simple tasks, and a data-science-only interview sample constrain how confidently responses transfer to live use, complex work, or other user groups.Participants’ expectations approximate rather than establish real-world usage; engineering and creative users may have different patterns.Source: §6.1 · p. 14

6.2 Future work

• Future work could examine code generation in spreadsheets, creative coding, low-code analytics, and block-based systems, where graphical and textual specifications are mixed differently.The paper cites spreadsheet prompting as an example of a promising intersection.Source: §6.2 · p. 14
• The interview probes covered only a cross-section of the design space; future studies could explore nonlinear input, synchronized projectional editors, and metadata-only assistants.Examples include spreadsheet-style code generation, visualization annotation, and code explanations.Source: §6.2 · p. 14
• Notebook assistants offer a broad design space, making careful adaptation to the environment—especially context control and disambiguation—important for future systems.The authors suggest considering search, context-free suggestions, recommendation explanations, and lightweight configuration.Source: §6.2 · p. 14

• Figure 10 presents a button for saving an automatically generated summary of assistant interactions that led to a code piece.Source: Figure 10 · p. 18
• Figure 11 shows inline attribution that marks assistant-generated code cells and reveals their interaction history on demand.Source: Figure 11 · p. 18
• Figure 12 shows an explorable log for browsing assistant-interaction history in the current notebook.Source: Figure 12 · p. 18