Spellburst: A Node-based Interface for Exploratory Creative Coding with Natural Language Prompts

1 INTRODUCTION

• Artists repeatedly translate expressive semantic intents into code, track creative variations, and compare, combine, and extend outputs, while existing programming environments poorly support this exploratory process.Artists may move from a stained-glass effect to variations in color and translucency, then take a larger creative jump to a different construct.Source: §1 · pp. 1–2
• Current tools make it difficult to track intermediate outputs with their specifications and compare multiple runs or combine attributes across exploratory paths.Artists may rely on screenshots and manually associate them with code; existing comparison tools remain disconnected from generative coding tasks.Source: §1 · p. 2
• Semantic prompts can support larger creative shifts, while syntax or dedicated controls can better support fine-grained editing and control.The motivating question is how to incorporate generative AI into creative programming workflows to support exploratory generative art-making.Source: §1 · p. 2
• Spellburst uses a node-based layout for outputs and code, branching and merging for variations, and prompting, dynamic controls, and direct editing for semantic and syntactic exploration.Its development drew on interviews with ten expert creative coders and a survey of 52 participants; it was evaluated with ten expert artists.Source: §1 · p. 2

2 RELATED WORK

• Creative coding uses programming to produce artistic rather than functional output, with artists exploring algorithm and output variations through edit-run cycles.Exploration can vary from changing a specific variable to changing a broader computational model.Source: §2 · p. 2

2.1 Tracking Exploration History

• Conventional version control can be costly and disruptive for creative work, where artists often explore non-linearly and manage alternatives through informal methods.Prior studies describe versions as materials to review and mix; CSTs have also explored parallel editing, microversions, and graph-based histories with annotations or screenshots.Source: §2.1 · pp. 2–3

2.2 Support for Expressive Programming

• Creative coders must map expressive concepts to low-level syntax, while visual programming, natural-language tools, and direct manipulation offer different ways to bridge that divide.Spellburst combines node-based programming, natural-language prompting with autocomplete, and direct manipulation interfaces.Source: §2.2 · p. 3

2.3 AI Augmentation for Creative Tasks

• Prompt-based AI can support fast iteration and combination, but creative coding tools must account for limited low-level control and the cognitive load of evaluating suggestions.Spellburst focuses on iterative and non-linear prompts.Source: §2.3 · p. 3

3 FORMATIVE INTERVIEWS WITH EXPERTS

• Ten expert creative coders spanning generative art, software engineering, design, game development, and education took part in semi-structured interviews, project walkthroughs, and a live p5.js task.The team analyzed transcripts and video using inductive coding, affinity clustering, and thematic analysis.Source: §3 · pp. 3–4, §3.1 · p. 4

3.2.1 Parameter- and accident-driven exploratory programming.

• Many participants explored by tweaking parameters, drawing on prior experience and parameter ranges from earlier work; the study recommends fine-grained control over manipulated parameters.One pattern in the task constrained circular movement to −π/4 to π/4 radians.Source: §3.2.1 · p. 4
• Participants also described trial-and-error as accident-driven development, with unexpected connections inspiring ideas and rapid feedback loops supporting exploration.They wanted parameter values to be easy to read and manipulate while readily seeing the output.Source: §3.2.1 · p. 4

3.2.2 Version control practices.

• Participants found Git’s planning and commit practices mismatched to unplanned creative exploration, and many instead duplicated files, commented out code, or relied on memory and notes.The study recommends integrating versioning and tracking with rapid exploration.Source: §3.2.2 · p. 4
• Participants saved visual outputs through screenshots or video, but those outputs were often detached from their code and useful history metadata.The study recommends tracking emergent outputs alongside the code that produced them.Source: §3.2.2 · pp. 4–5

3.2.3 Interest in AI Augmented Creative Exploration.

• Some interviewees were wary of AI in personal projects or preferred post-processing, while participants welcomed expressive prompts for inspiration and emphasized that humans should lead the creative process.Participants described AI as a source of suggestions for new directions, and the study recommends different support at different creative stages while preserving artist control.Source: §3.2.3 · p. 5

4 USER EXPERIENCE

• Sketch and operator nodes form hierarchical layouts on a zoomable canvas, with operations supporting modification, merging, duplication, diffing, and extraction.Sketch nodes represent rendered p5.js programs; operator nodes link explorations and output new sketches from source sketches and prompts.Source: §4 · p. 5

4.1 Set-up

• Artists can create a root sketch by adding a node, then author or paste code and view its rendered output.Spellburst also offers starter code for basic shapes.Source: §4.1 · pp. 5–6

4.2 Focused Exploration

• Duplication and branching let artists preserve a base sketch while exploring line-thickness and color variations in separate paths.Branches are generated through duplicating and iterating on sketches and prompts rather than arbitrary freeform edges; artists prefer direct control for familiar attributes.Source: §4.2 · p. 6

4.3 Exploring Large Creative Jumps

• Natural-language modification turns an initial sketch into a planetary-orbit concept, while autocomplete suggestions offer additional directions such as adding planetary rings.Ash selects the rings suggestion and continues with a new sketch.Source: §4.3 · p. 7

4.3.1 Iterating on generated outputs through code editing.

• For finer refinement, artists can open generated code, adjust parameters such as spacing, and see the output update instantly.Spellburst provides code comments for prompt-generated code.Source: §4.3.1 · p. 7

4.3.2 Iterating on prompts using Semantic and Global Variable Slid-

• Prompt-linked sliders map phrases to global variables such as circleSize, numCircles, and noiseStrength for real-time sketch adjustments.The sliders support exploration of divergent creative paths alongside non-deterministic autocomplete.Source: §4.3.2 · pp. 7–8

4.4 Merging Nodes and Extracting Properties

• Artists can merge sketches from separate branches, extract an attribute into a separate sketch, and compare versions through natural-language and code-level diffs.Generated code may occasionally fail to run or conflict with expectations, requiring manual debugging or regeneration; structured auto-layout visualizes the version-history tree without canvas cleanup.Source: §4.4 · p. 8

5 SPELLBURST

• Spellburst composes OpenAI API calls with context from code, prior prompts, and operator nodes to generate p5.js sketches.Its graph operations support semantically meaningful merges, not only line-by-line code combination.Source: §5 · p. 8, §5.1.4 · p. 9

5.1.1 Code Generation.

• Few-shot examples, output templates, rendering constraints, and explanatory code comments guide generated sketches toward usable, more interpretable code.Prompt design was iteratively explored using visual output as a guide; comments were added to support finer syntactic edits.Source: §5.1.1 · pp. 8–9, §5.1 · p. 8

5.1.2 Prompt Auto-complete.

• Because natural-language prompts are ambiguous, Spellburst autocomplete offers nondeterministic directions using the current prompt, connected sketch, and five completion examples.The aim is to support exploration rather than only task execution or search.Source: §5.1.2 · p. 9

5.1.3 Semantic Parameter Adjustment.

• Semantic parameter adjustment extracts prompt phrases, maps them to related global variables, then generates code from the prompt and map.The authors describe the map as resembling an attention heatmap, which they cannot obtain from the closed-source model.Source: §5.1.3 · p. 9

5.1.4 Merging Operations.

• Semantic merging uses an LLM to combine sketch elements meaningfully, such as taking physics from one sketch and a palette from another.Few-shot examples, templates, and code comments guide the merge output.Source: §5.1.4 · p. 9

5.1.5 Developing a Taxonomy for Creative Coding Transformations.

• A three-category transformation taxonomy covers objects or marks, canvas composition, and relationships between objects; over 40 works and crowd-sourced queries informed its examples.Curated crowd-sourced responses became seed data for few-shot prompts.Source: §5.1.5 · pp. 9–10

5.2 Implementation Details

• Reactflow supports the branching-and-merging node interface, while d3.js manages hierarchical tree visualization and graph auto-layout.Separate iframes run sketches, and reloading them on each code-editor keystroke supports immediate visual feedback.Source: §5.2.1 · p. 10

5.2.2 Backend and Language Model API:.

• The backend uses Node.js, Express.js, and ChatGPT; graph edits create new nodes, while reruns update connected sketches only one layer deep.Deleting a node reattaches its descendants to its parent, which does not preserve the original causal structure.Source: §5.2.3 · p. 10

6.1 Method

• In the within-subject study, 11 expert creative coders completed small-change, large-shift, and open-ended tasks with Spellburst and their typical IDE alongside ChatGPT.The task order was randomized; participants also completed questionnaires and a discussion.Source: §6.1 · p. 10

6.2.1 Prompt Based Editing.

• Participants welcomed prompts for larger creative shifts, but views on small syntactic edits were mixed; auto-complete suggestions inspired ideas, while some wanted more interpretability.Several participants found generated code close to their intent, while others found prompts less useful when they already knew what to change.Source: §6.2.1 · p. 11

6.2.2 Comparison of Spellburst with ChatGPT baseline.

• Participants valued Spellburst’s visual branching and version organization over ChatGPT’s more linear process, though one participant found ChatGPT less frictional because it was familiar.Some participants said Spellburst helped them reason about versions and explore more ideas per minute.Source: §6.2.2 · pp. 11–12

6.2 Findings

• Participants explored through multiple pathways, varied branch depth and breadth, and often made several modifications before merging; Figure 6 visualizes these patterns.Some participants also left prompt-modified nodes unexplored, while others used modifications as terminal nodes.Source: §6.2 · pp. 10–11, Figure 6 · p. 12

6.2.3 Usability of Spellburst’s Interface.

• Spellburst’s overall PSSUQ usability score was 7.67; NASA-TLX responses rated it most mentally demanding (mean=9.9, SD=4.8) and least physically demanding (mean=2.6, SD=1.2).The PSSUQ Information Quality sub-scale was 6.76, with the passage identifying error messages, documentation, and information organization as improvement areas.Source: §6.2.3 · p. 12
• Participants appreciated branching and merging without Git-style commands and saw potential to use Spellburst as a sandbox, though some wanted more control over visual focus.One participant also valued running prompts simultaneously because the delay left time to think about further variations.Source: §6.2.3 · p. 12

6.2.4 Perceptions of impact on their creative practice.

• Participants saw Spellburst as useful for early-stage ideation and fast prototyping, even when generated code did not exactly match their expected output.One participant said abstract instructions could succeed without being highly specific.Source: §6.2.4 · pp. 12–13
• Participants raised concerns about accidental plagiarism, attribution, bias, and limited understanding of AI-generated work; the authors call for source citation and attention to responsible use.The discussion notes differing views on ownership and artists’ rights, and recognizes that model biases may appear in generated code or descriptions.Source: §6.2.4 · pp. 13–14

7.1 Utility of Spellburst

• The section frames Spellburst’s broader utility as extending the expert workflow to creativity and learning use cases.Source: §7.1 · p. 14

7.1.1 Learning Version Control for Creative Coding Novices.

• Spellburst could help novices explore a wider design space and save, reflect on, and debug their creative process through intuitive version control.The discussion notes that novices may get stuck on either semantic or syntactic constructs and may not know how parameter ranges affect outcomes.Source: §7.1.1 · p. 14

7.1.2 Debugging.

• For debugging, users can duplicate a sketch, incrementally revise it with code inspection or AI prompting, then merge the fix back into the main branch.Source: §7.1.2 · p. 14

7.1.3 Sharing and Documenting Work.

• Shared Spellburst links can expose prompts, code, and earlier branches, letting peers give feedback and beginners trace an artist’s process.The imagined peer can locate an earlier iteration, while a beginner can follow the process step by step.Source: §7.1.3 · p. 14

7.2 Limitations and Future Work

• Semantic merging can yield playful discoveries, but its ease-over-control design limits precision for small, deliberate creative jumps.The authors suggest allowing users to specify and manipulate the emphasis placed on aspects of each input.Source: §7.2 · p. 14
• Effective prompts remain hard to predict, so semantic prompts do not map perfectly to syntactic changes and may frustrate users.Proposed directions include taxonomy-based fine-tuning, prompt constraints, conversational common ground, and multimodal links between canvas and text.Source: §7.2 · p. 14
• The evaluation involved 10 expert coders and limited interactions and tasks, leaving medium-sized jumps and long-term in-situ use for future study.The authors also call for larger, more diverse samples and approaches to scale interface performance and usability as nodes and edges grow.Source: §7.2 · pp. 14–15
• Future work should examine collaborative Spellburst use for teams combining preferred elements from separate sketches.Source: §7.2 · p. 15

8 CONCLUSION

• Spellburst integrates creative exploration with history tracking to reduce management burden and support combinatorial and transformative creativity.Source: §8 · p. 15
• An LLM informed by crowdsourced data supports semantic and syntactic exploration, balancing prompt-driven shifts with dynamic controls and direct code editing.Source: §8 · p. 15
• Evaluation with expert generative artists indicates that Spellburst supports rapid exploration while reducing the overhead of managing it.Source: §8 · p. 15

A.1 JSON Schemas for Graph Data

• The appendix provides example JSON objects for a graph node and an edge.Source: §A.1 · p. 17

A.2 Prompts for LLMs

• LLM requests use a system configuration, dynamic context, and user input; shared restrictions require raw code output and efficient implementations.Source: §A.2 · p. 17, §A.2.1 · p. 17

A.2.2 Modification.

• The modification route configures a p5.js assistant and demonstrates a natural-language variation prompt applied to existing sketch code.Source: §A.2.2 · p. 17

A.2.3 Merging.

• The merging route combines two p5.js snippets, with a merge prompt specifying which features to combine and the desired result.Source: §A.2.3 · p. 18
• The merge example combines a rotating-line animation with bouncing-ball behavior in the resulting sketch.Source: §A.2.3 · p. 18

A.2.4 Autocomplete.

• Autocomplete is configured to provide suggestions for incomplete prompts, including continuations for particle and tree-art requests.Source: §A.2.4 · p. 19
• Autocomplete examples return up to three continuations, such as suggestions for particle behavior and visual styles.Source: §A.2.4 · p. 19

A.2.5 Extraction. / A.2.6 Diffing.

• The extraction and diffing prompts frame the assistant as a p5.js expert; diffing asks for similarities and differences in sketch content, properties, and code.Source: §A.2.5 · p. 19, §A.2.6 · p. 19

• Eligibility targeted U.S.-based adults with English as a first language, sufficient art experience, and desktop access.A total of 370 participants completed the initial prescreening survey.Source: §A.3.1 · p. 19
• The tasks asked participants to describe image transformations in natural language without seeing the underlying code.Each task supplied a starter image and a final image.Source: §A.3.1 · p. 19
• The study collected 12 prompts for each of 20 images, with contributions from 52 unique participants.Incomplete and timed-out tasks were discarded.Source: §A.3.1 · p. 19
• Successful combined descriptions commonly referenced both a taxonomy term and a simile or metaphor.Source: §A.3.1 · p. 20
• Responses were categorized as semantic, syntactic, or combined semantic-and-syntactic descriptions.Source: §A.3.1 · p. 20
• Example prompts paired visual transformations, such as rippling circles, with concrete editing instructions.One example specified repeated layers and progressively lowered image levels.Source: §A.3.2 · p. 20

A.4 Example Spellburst Evaluation Output

• The evaluation examples show P6’s variations on concentric-circle starter code and P8’s provisional slider interface.Source: Figure 10 · p. 21, Figure 11 · p. 21

A.5 System Architecture

• The system architecture connects the front-end Canvas to a server that calls the ChatGPT API.Source: Figure 12 · p. 22

A.6 Demographic Information of Participants

• The appendix includes separate demographic tables for the generative-study and expert-evaluation participants.Source: Table 1 · p. 22, Table 2 · p. 22