JarvisHub: An Open Harness for Canvas-Native Multimodal Creative Agents

• Creative projects accumulate references, drafts, alternatives, edits, failures, versions, and feedback that agents need to understand and update across an extended workflow.The challenge is to give agents a readable, editable workspace that preserves relationships among materials, accepted or rejected candidates, and unfinished artifacts.Source: §1 · pp. 1–2
• Prompt-to-output, chat-based, and node-based systems support parts of creative production but do not provide a general runtime for agents to inspect, revise, and verify project state.Figure 1 contrasts these systems with JarvisHub’s shared canvas workspace.Source: Figure 1 · p. 2, §1 · p. 2
• Commercial products signal a shift toward multi-stage creative workflows, but their closed architectures prevent researchers from inspecting state representation, tool scheduling, feedback use, and failure repair.Examples include Claude Design, Google Stitch, TapNow, LibTV, and MiniMax Hub.Source: §1 · p. 2
• JarvisHub formalizes long-horizon creation as an agent process over an editable project graph, representing multimodal materials, dependencies, versions, edits, and feedback as canvas nodes and links.Agents inspect the canvas, call tools, and write created or modified nodes back to the same traceable project state.Source: §1 · p. 3
• JarvisHub unifies canvas state, a protocol bridge for checked and logged operations, and an agent runtime for tool use, synchronization, recovery, and trajectory recording.The harness also exposes canvas-editing, media-generation, native-execution, recovery, and MCP-backed extension tools, with skills, memory, and sub-agents supporting longer workflows.Source: §1 · p. 3
• JarvisHub is demonstrated on long-horizon cases in narrative media generation, interactive web development, and presentation deck generation.Source: §1 · p. 3

2.1 Overview

• Each turn observes the canvas, selects a permitted action, invokes a capability, and writes the returned evidence back to the shared project state.The architecture combines canvas state, a protocol bridge, and an agent runtime, and records trajectories, evaluator signals, human edits, and checkpoints.Source: §2.1 · p. 3, Figure 2 · p. 4

2.2 Canvas-Native Project State

• The canvas represents artifacts and interaction context in a typed graph, making candidates addressable and reusable, dependencies inspectable, and artifact status visible.It separates editable contents, provenance and runtime metadata, user selections and feedback, and spatial layout; status distinguishes completed or selected artifacts from planned, running, failed, or unverified ones.Source: §2.2 · p. 4

2.3 Protocol-Constrained Canvas Interaction

• The protocol bridge checks proposed actions against available capabilities and per-request grants, then records accepted mutations and observations on the canvas for auditability and recovery.Recorded updates can create nodes, change fields, connect dependencies, attach artifacts, log failures, create checkpoints, or request human correction.Source: §2.3 · p. 5

2.4 Agent Runtime for Creative Orchestration

• The runtime grounds each action in the observed canvas, available manifest, and execution grant, while skills, memory, and subagents help sequence capabilities across creative tasks.Skills encode reusable procedures, memory preserves preferences and decisions across turns, and subagents can explore independent subtasks for integration into the shared canvas.Source: §2.4 · pp. 5–6

2.5 Feedback and Traceability

• Feedback from users, evaluators, or subagent critics guides whether the agent continues, repairs a node, seeks clarification, or stops when evidence is insufficient.The trajectory records the request, canvas state, capabilities, grant, action, observation, feedback, repair or follow-up decision, and updated canvas state.Source: §2.5 · pp. 6–7

3.1 Experimental Setup

• All tasks use GPT-5.5 as the agent backend, GPT Image 2 for image generation, Seedance 2.0 [41] for video generation, and Gemini 3.1 Pro for multimodal evaluation.The paper directs readers to the open-source repository for further backend configuration details.Source: §3.1 · p. 7

3.2 Long-Horizon Creative Tasks

• The evaluation covers narrative media generation, interactive web development, and presentation deck generation, each requiring planning, reference organization, intermediate artifacts, feedback, and local revision.The tasks develop open-ended goals into coherent deliverables.Source: §3.2 · p. 7

3.3 Qualitative Results

• Each task is shown through a canvas-centered workspace trace and a final artifact, making plans, references, assets, dependencies, and agent progress inspectable alongside the output.The paired views illustrate how accumulated canvas state supports coherent deliverables.Source: §3.3 · p. 7
• The narrative case turns a short-drama prompt into a visual sequence whose canvas trace exposes story planning, references, dependencies, and generation progress while supporting continuity across shots.The final artifact presents key frames with recurring characters, setting cues, and action continuity across shots.Source: §3.3 · p. 7, Figure 4 · p. 8, Figure 5 · p. 8
• The web-development case converts an aesthetic and interaction brief into a photography website, with the canvas tracking references, implementation, previews, and revisions while preserving visual direction and interface consistency.The final screens show consistent typography, image placement, page structure, and visual direction.Source: §3.3 · pp. 8–9, Figure 6 · p. 9, Figure 7 · p. 9
• The presentation case turns a machine-learning lecture topic into a structured slide deck, with the canvas exposing content planning, visual assembly, previews, and progress while preserving topic structure and style.The task requests a decision-trees lecture deck styled like a Stanford lecture.Source: §3.3 · p. 10

• Typed, addressable canvas nodes and links make the canvas shared external memory and action space for agents and humans.Users can inspect and guide the state the agent reads and modifies, enabling artifact reuse, local updates, dependency maintenance, and continuation of unfinished work.Source: §4 · p. 11
• An open creative-agent harness can enable project-state benchmarks, process-level evaluation, and structured trajectory data for future model training.With consent, anonymization, copyright filtering, and quality control, trajectory data can train agents in planning, tool selection, multimodal state tracking, local repair, and feedback-guided revision.Source: §4 · p. 11
• Full trajectories reveal creative-agent behavior and process failures that final artifacts alone cannot characterize.They record context, actions, tool evidence, feedback, and canvas updates, exposing failures such as ignored references, lost accepted choices, overwritten variants, or unnecessary global regeneration.Source: §4 · p. 11

5 Conclusion and Limitations

• JarvisHub represents creative projects as editable canvas graphs, with a shared runtime for context inspection, tool use, artifact updates, feedback, and state recording.The runtime integrates canvas operations, media generation, native tools, repair, checkpointing, and trajectory capture.Source: §5 · p. 11
• Qualitative demonstrations show JarvisHub can support more inspectable and reusable agentic workflows, and recorded trajectories can inform benchmarks, evaluation, and agent training.The experiments are qualitative demonstrations, not a completed benchmark or leaderboard.Source: §5 · p. 11
• JarvisHub does not guarantee semantically correct creative decisions, and final artifact quality depends on the external models and tools used by its runtime.The protocol bridge makes canvas actions explicit and recoverable but does not ensure semantic correctness.Source: §5 · p. 11
• Raw trajectory records require quality filtering, consent, anonymization, and copyright review before use as research data.Source: §5 · p. 11