PaperlayerView exact revision on arXiv

Source-linked AI summary

Agentic Artifact Creation: Systems, Evaluation, Principles, and Opportunities

Tianfu Wang, Zhezheng Hao, Xilin Xia, Lixin Liu, Mengkang Hu, Hongzhang Liu, Xi Chen, Ziyan Liu, Xiankun Lin, Weijia Zhang, Nicholas Jing Yuan, Hui Xiong

arXiv:2608.28122v1cs.MM

TL;DR

Dependable deliverables require more than generating isolated components because artifact decisions are interdependent and validity is only partially observable. This survey reviews agentic artifact creation as stateful construction linking representation, policy, and runtime verification across 230 systems and 29 benchmarks. It finds recurring challenges in coupling, failure visibility, repairability, evaluation independence, and maintaining coherent state through revision and evolution.

  • Problem

    Complete artifacts are harder than isolated outputs because interdependent decisions can affect downstream constraints, while checks may be late, partial, or stale after revision.

  • Method

    The survey defines agentic artifact creation as stateful construction and synthesizes systems and benchmarks across six artifact families, applications, evaluation, and design principles.

  • Results

    Across artifact families, construction challenges reflect decision coupling, late or unclear failures, and edits that are too coarse for local repair.

  • Takeaways & Limitations

    Effective construction control requires explicit commitments, feasible targeted repair, and revalidation of affected state after changes.

  • Takeaways & Limitations

    Evidence is strongest for delivered artifacts and bounded executions, while trajectories, use outcomes, and validity beyond tested cases remain less consistently covered.

Abstract

from arXiv · show

Generative models can turn natural-language prompts into images, text, code, and other content, lowering the cost of producing drafts and components. Their practical impact increasingly depends on whether those pieces can become complete, dependable deliverables. This survey examines agentic artifact creation, which we define as stateful construction in which an AI system materially constructs or revises a deliverable and intermediate observations redirect later work. Functionally, the process links an operational representation of the artifact, a construction policy, and runtime verification whose feedback can redirect later actions. We reviewed 259 works available through August 20, 2026: 230 systems meeting this definition and 29 benchmarks of agentic artifact construction. We compare six artifact families, then analyze application settings and evaluation practice as separate dimensions. Across families, construction challenges reflect not only modality but also how tightly decisions are coupled and whether failures become visible while they remain repairable. Decomposition can reduce local complexity while increasing coordination and reassembly costs. Learned judges may add little independent evidence when they share the generator's preferences or blind spots. We formulate principles for keeping commitments and responsibility explicit, turning feedback into targeted repair, and revalidating affected state after change. We also identify opportunities for sustaining coherent, accountable control as artifacts, creator intent, and construction systems evolve. A curated paper list is available at https://github.com/GeminiLight/awesome-agentic-artifact-creation.

1 Introduction

Agentic artifact creation addresses the limits of direct generation for dependable deliverables by using state, construction decisions, and runtime feedback to redirect work. The survey defines this paradigm, scopes its review, and organizes four questions around construction, context, evaluation, and principles.

  • From Direct Generation to Agentic Creation: Complete deliverables require coordinated parts to satisfy multiple acceptance criteria, which isolated outputs or scalar quality scores cannot establish.Poster construction must preserve source content, fit a constrained page, and communicate the paper clearly.
  • From Direct Generation to Agentic Creation: Interdependent decisions and partially observable validity make revisions difficult because changes can affect downstream constraints and checks may become stale.Software agents illustrate the need to edit repository state and interpret test or execution feedback rather than emit isolated code.
  • Conceptual Scope: Agentic artifact creation materially constructs or revises a deliverable while intermediate observations redirect later artifact-level actions or stopping.The survey distinguishes this episode-level construction boundary from generic tool-using or problem-solving agents.
  • Agentic Construction Model: The paradigm links an Operational Representation, Construction Policy, and Runtime Verification so feedback can guide changes, localization, and continued construction.These functions are most useful when they describe, edit, and assess the same construction units, supporting composability, traceability, and revisability.
  • Review Scope and Contributions: The review retained 230 systems and 29 construction benchmarks after searching multiple scholarly sources, deduplicating candidates, and screening for stateful construction.The review asks how construction is organized, how contexts shape it, how it is evaluated, and which principles guide it.
  • Our Contributions and Organization: The survey contributes a stateful-construction model, a coded corpus comparing six artifact families, separate analyses of applications and evaluation, and four inspectable-control principles.Its organization proceeds from foundations and the paradigm through families, applications, evaluation, principles, challenges, and opportunities.

2 Background

The background positions agentic artifact creation at the intersection of foundation-model content operations, agentic control, and artifact-production research. Its artifact-centered lens differs from adjacent surveys that primarily organize architectures, models, media, processes, domains, or evaluation.

  • Foundation Models: Foundation models provide reusable operations for generating, editing, and converting content, but do not maintain construction state or decide later actions and completion.These responsibilities motivate the agentic control mechanisms reviewed in the survey.
  • Agentic AI Architectures: LLM-based agents add planning, state or memory, tool use, observation, and stopping around foundation models, with control distributed across single agents, orchestrators, or specialized peers.For artifact creation, these mechanisms must preserve artifact state and carry observations into later artifact-level actions or stopping.
  • Artifact Production Processes: Earlier artifact-production research made representations, actions, feedback, and human authority explicit through procedural generation, program synthesis, and related domain mechanisms.These traditions connect specifications or constraints with candidate artifacts, completions, or feasibility checks.
  • Positioning Against Adjacent Surveys: The survey organizes its comparison around the Delivered Artifact, making acceptance criteria, editable state, observations, and repair scope comparable across artifact families.This differs from neighboring surveys centered on agents, model families, media, creative refinement, software repair, trajectories, human feedback, or evaluation.

3 The Agentic Creation Paradigm

Agentic Artifact Creation is stateful artifact construction in which represented state and intermediate observations redirect later work. Its architecture links an operational representation, construction policy, and runtime verification into a recurrent process supporting composability, traceability, and revisability.

  • Conceptual Definition: The paradigm is defined at the level of a single artifact-creation episode rather than a particular model family, system architecture, or artifact type.Creative artifacts allow multiple goal-satisfying realizations, while the definition specifies minimum control behavior.
  • Conceptual Definition: Agentic Artifact Creation requires material construction or revision, persistent artifact or process state, and observations that redirect subsequent artifact-related work.The redirection can change the next action, revision target, active branch, or stopping decision.
  • Functional Roles: The Operational Representation maintains current artifact-side state through an Intermediate Form and Edit Interface, determining what persists and which changes can be expressed.The state update is represented as R_t+1 = U(R_t, a_t).
  • Functional Roles: The Construction Policy selects actions from task requirements, current representation, and feedback, including revision targets, operations, work allocation, branches, or stopping choices.Runtime Verification produces feedback by evaluating observations of updated state against acceptance criteria.
  • System Design: Decomposition reduces local complexity and exposes specialist interfaces but introduces coordination costs, dependency-bearing handoffs, and responsibility for preserving global commitments.The key concerns are conflict resolution, commitment ownership, and which state survives each boundary.
  • System Affordances: Composability, traceability, and revisability emerge when artifact structure, decisions, interfaces, records, and runtime evidence are aligned.Direct generation can remain appropriate for simple artifacts whose failures are cheap to detect and regenerate because agentic control adds overhead.

4 Landscape of Artifact Families

The survey organizes agentic construction into six artifact families and 16 analytical profiles, examining how acceptance criteria determine representations, repairs, dependencies, and observation modes. Across families, control becomes harder as commitments extend beyond the immediately edited unit and relevant judgments remain only partly observable.

  • Family Organization: The survey groups works by six independently accepted artifact forms: textual, 2D visual, audio, video, spatial, and behavioral.Behavioral artifacts are defined by executable, state-dependent responses as the delivered form.
  • Observability Landscape: Figure 5 contrasts dominant failure-observation points across static inspection, temporal playback, and runtime consequences, while noting these modes are tendencies rather than exclusive.The shared grammar is state–edit–observe.
  • Creative Writing: Creative writing carries long-range plot, character, voice, payoff, timing, and anticipated-reception commitments that may fail many passages after the originating decision.Narrative systems use planning, revision, or persistent character and social state; performative systems retain timing and performance structure.
  • Professional Documents: Professional documents are accepted against source, domain, audience, or workflow obligations, with evidence links, rules, and task performance defining repair units.Informational reports emphasize coverage, traceability, and evidence organization, while functional documents emphasize reliable task performance.
  • Scholarly Manuscripts: Scholarly manuscripts use contribution-level acceptance criteria linking claims to methods, results, prior work, and contributions, with arguments, claim–evidence links, and method–result inconsistencies as repair units.PaperDebugger supports targeted in-editor revision, while The AI Scientist provides an end-to-end research-to-manuscript case.
  • Cross-Family Pattern: Textual control becomes harder as commitments extend beyond the edited passage, while reader and domain judgment remain necessary because coherence, novelty, persuasion, and professional fitness are only partly observable.Repair is bounded when those judgments can be traced to the responsible event, claim, source, rule, or argument.

4.2 2D Visual Artifacts

2D visual artifacts are accepted through rendered surfaces, but dependable construction requires preserving the data, semantic, source, and cross-page links that make defects repairable.

  • 2D visual artifacts range from raster layers and vectors to chart code, layout trees, and deck structures, with edits spanning pixels, objects, pages, and cross-page commitments.Rendering provides a common feedback channel, while different profiles authorize the surface through data, perceptual, semantic, or source evidence.
  • Data visualizations: Data visualizations must communicate effectively while keeping every visual mark faithful to source data and recorded transformations.Systems preserve executable transformation paths, inspect renders, and synchronize data, chart specifications, and conversational revisions.
  • Illustrative graphics: Illustrative graphics require editable structure beneath the rendered surface so feedback can target perceptual compositions, regions, objects, and relations.Images emphasize localized appearance, whereas diagrams depend more heavily on recoverable entities and relations.
  • Visual documents: Posters balance source fidelity, content selection, hierarchy, legibility, and one-page capacity, while presentations add narrative order and cross-slide consistency.Construction systems address these coupled requirements through staged workflows, incremental editable actions, role coordination, references, and rendered feedback.
  • 2D profiles differ in what authorizes the rendered surface and in their repair units, so local visual validity does not establish that larger dependencies survived an edit.Direct surface edits can sever data or source links; structured code and object state preserve links but may make visual changes indirect.

4.3 Audio Artifacts

Audio artifacts combine editable symbolic or production state with rendered sound, requiring different checks for structure, alignment, timbre, expression, and scene-level perception.

  • Audio artifacts are symbolic or rendered sound organized over time, with editable state in notation, lyrics, production graphs, stems, or waveforms.Construction can revise notes, phrases, synthesis, effects, mixes, and other time-organized components.
  • Music: Music systems coordinate melody, harmony, arrangement, lyrics, notation, and audio through shared signals, aligned state, explicit grammars, and iterative listening.Symbolic checks can redirect structural repair early, while timbre and expression still require listening after rendering.
  • Spoken audio: Spoken audio must keep source content, scripts, speakers, timing, and rendered segments aligned, but the mapped corpus contains no dedicated spoken-audio construction system.Without linked source units, scripts, voices, and clips, late listening feedback tends to trigger whole-segment regeneration.
  • Soundscapes: Soundscape systems represent acoustic events, sources, temporal placement, edit programs, and rendered clips to support diagnosis, retrieval, correction, and local revision.Executable scene plans can link source selection, temporal layout, rendering, metadata, and human-editable state.
  • Audio separates inspectable symbolic or production state from qualities heard only after rendering, so perceptual repair remains local only when linked to an editable unit.Structural checks can redirect upstream edits, but perceptual judgments require correspondence to a revisable component.

4.4 Video Artifacts

Video artifacts are ordered visual or audiovisual sequences whose construction must preserve communicative commitments across shots, channels, and time before assembled consequences become visible.

  • Video construction uses scripts, storyboards, shot lists, timelines, event graphs, and cross-shot memory to revise scenes, clips, transitions, motion, sound, and temporal constraints.Acceptance criteria extend across ordered shots and time rather than isolated frames.
  • Expository videos: Expository videos must accurately cover external subject matter while coordinating visuals, narration, subtitles, and timing after assembly.Systems ground scenes in aligned channels or executable plans and inspect rendered results.
  • Narrative videos: Narrative videos must carry story intent, identity, causal order, and cinematic decisions consistently from one shot to the next.Hierarchical representations organize story, scene, shot, and modality plans, including long-video orchestration.
  • Promotional videos: Promotional videos depend on message fidelity, recognizable identity, audiovisual appeal, and fit to their intended audience or channel.Their purpose includes presenting destinations, brands, personas, or performers, with direct advertising as one case.
  • Video systems expose clip- and tool-level edits that map visual, audio, contextual, or physical observations to bounded revisions across modalities and post-production stages.Construction mechanisms differ in the timeline state and edit operations they expose, even when profiles are classified by communicative purpose.

4.5 Spatial Artifacts

Spatial artifacts preserve persistent geometry and relations across assets, scenes, simulations, fabrication, and applications; repair remains local only when observations trace to responsible structure.

  • Spatial artifacts are defined by persistent geometry and spatial relations in meshes, scene graphs, asset libraries, engine scenes, parametric histories, or CAD programs.Feedback may arrive through rendering, geometry operations, traversal, simulation, or fabrication checks.
  • 3D assets: 3D assets require semantic parts, appearance, and edit handles to remain correspondingly recoverable through revision.Procedural histories and progressive mesh construction preserve modeling operations and geometric state, although plausible surfaces may lack recoverable design intent.
  • Parametric models: Parametric models preserve dimensions, ordered features, constraints, editable variables, and precise boundary geometry for engineering-oriented revision.Cross-backend construction graphs can expose dependencies, constraints, topology, and diagnostics.
  • 3D scenes: Spatial worlds combine assets into layouts that must remain relationally consistent, traversable, task-compatible, and sometimes simulation-ready.Engineered models additionally make component interfaces and dimensional constraints part of acceptance.
  • Rendering exposes appearance, while traversal, simulation, and fabrication reveal failures missed by static inspection; local repair requires tracing observations to responsible structure.Responsible units include parts, relations, features, constraints, and assets.

4.6 Behavioral Artifacts

Behavioral artifacts are accepted through state-dependent responses rather than static surfaces, requiring increasingly extended execution horizons and references that may be virtual specifications or external processes. Across profiles, effective repair depends on tracing observed behavior to bounded editable state and revalidating affected conditions.

  • Behavioral artifact scope: Behavioral artifacts are defined by state-dependent responses to inputs, actions, and environment interaction rather than static appearance alone.Software realizes functionality or experience, while simulation models approximate a target environment or process.
  • Validation horizons: Software validation expands from repository execution and integration checks to browser-visible action–state sequences and sampled gameplay trajectories.Longer horizons expose failures that code-only feedback can miss, but they require broader and more expensive observation.
  • Simulation references: Simulation models differ from games because they are accepted against either specified virtual-world rules or observations and laws governing an external process.Virtual simulators require legal, reproducible transitions, while physical models must agree with observations, constraints, or governing laws over the tested regime.
  • Repairability: As behavioral horizons grow, execution covers less of the relevant state space and failed trajectories become harder to localize to a responsible file, component, rule, or parameter.Repair therefore requires tracing behavior to the responsible editable unit and revalidating affected conditions.
  • Shared comparison: Useful behavioral evaluation compares whether observations remain linked to persistent, editable state so failures can be repaired at bounded units.Later-stage observations broaden coverage only when the representation preserves actionable links to the affected state.

5 Applications

Applications reconfigure shared artifact families around different objectives, artifact bundles, acceptance criteria, and authorities. Across creative, commercial, educational, professional, and scientific settings, workflows coordinate construction with domain-specific feedback and responsibility, sometimes across independently accepted deliverables.

  • Application dimensions: Application context determines the domain objective, participating artifacts and tools, and authority over consequential use, while artifact family identifies the delivered object and revised state.The same artifact can serve several domains, and one workflow can coordinate several artifact forms.
  • Creative production: Creative production coordinates interdependent narrative, visual, auditory, and interactive decisions using rendering, playback, and interaction to redirect later construction.Interactive entertainment adds persistent branching state, runtime validity, and playability to narrative and stylistic criteria.
  • Brand communication: Brand and customer-engagement workflows preserve shared message or identity commitments while adapting assets and communication to channels, context, and observed audience response.Response signals can redirect later actions but remain noisy proxies for commercial goals.
  • Educational support: Educational systems align instructional resources or learning trajectories with goals while using assessment, interaction, and learner-state feedback to adapt later construction.Curriculum development emphasizes alignment and structure, whereas adaptive learning maintains evolving learner context across interactions.
  • Professional and scientific work: Professional and scientific workflows connect changing sources, tools, experiments, analyses, and review to reports, decisions, hypotheses, or presentations.In regulated settings, qualified professionals remain responsible for consequential interpretation and action.
  • Cross-output dependencies: Some application objectives require several independently accepted deliverables to remain aligned through shared paper-derived or other cross-output state.This dependency pattern is distinct from both artifact family and application context.

6 Evaluation and Benchmarking

Evaluation separates delivered artifacts, construction trajectories, and agentic systems, linking each target and criterion to explicit evidence channels, evaluators, metrics, aggregation rules, and protocols. The review finds that task-stratified and complementary evidence is more informative than aggregate scores alone, while trajectories and beyond-tested validity remain comparatively undercovered.

  • Evaluation targets: Evaluation targets delivered artifacts, construction trajectories, or agentic systems, and these targets are related but not interchangeable.Artifact evaluation concerns requirements and intended use; trajectory evaluation concerns detection, repair, preservation, and cost; system evaluation probes reusable behavior beyond task capability.
  • Delivered artifacts: Artifact evaluation combines requirement satisfaction, quality, intended-use effectiveness, and internal consistency, because functional checks or aggregate scores can hide unmet requirements.Representative measures include hard-constraint pass rate, rubric scores, task completion, and cross-unit violation rates.
  • Evidence and protocols: An evaluation signal pairs an evidence channel with an evaluator, after which metrics and aggregation rules determine the reported quantity for a stated requirement and stage.The same observation can support different claims under different evaluators, so the pairing must be explicit.
  • Evidence complementarity: Complementary signals are preferable to merely numerous signals because multiple judges can share a blind spot, while requirement-to-check maps can connect requirements, states, repairs, and errors.This structure supports more inspectable evidence across source, structural, runtime, rendered, and human channels.
  • Benchmark reporting: Task-stratified benchmark results reveal capability differences that a single aggregate rank conceals, so evaluations should name task categories, harnesses, and environments.For example, GPT-5.6 Sol ranges from the 87.5th percentile on slides to the 34.2nd percentile on mobile apps.
  • Coverage limitations: Evidence is strongest for delivered artifacts and bounded executions, while trajectories, use outcomes, and validity beyond tested cases remain less consistently covered.Only three protocols evaluate a system property beyond task capability, each covering one property, and inconsistent reporting limits comparison.

7 Principles of Agentic Creation

The paper synthesizes four qualitative principles for keeping acceptance criteria, artifact state, responsibility, evidence, repair, and revalidation inspectably connected. These principles are relations rather than a mandatory pipeline or maturity ladder, and their implementation should remain proportionate to artifact- and application-specific risks.

  • Qualitative Difficulty Lens: The qualitative difficulty lens examines decision interdependence, failure observability, and repairability as recurring conditions of construction difficulty.These dimensions concern dependency scope, timely attribution, and whether diagnoses map to bounded actions that preserve accepted work.
  • Externalize Commitments: Externalize Commitments connects consequential requirements and prior decisions to addressable artifact state that later actions and checks can preserve, change, or verify.Relevant state may include criteria, source links, constraints, semantic edit units, dependency edges, provenance, or review status.
  • Define Control Boundaries: Define Control Boundaries assigns delegation, handoff, tool-permission, and review responsibility where dependencies or decision consequences change.Decomposition helps only when modules follow meaningful dependencies and handoffs preserve the state needed downstream; otherwise recomposition can lose shared constraints.
  • Make Feedback Actionable: Make Feedback Actionable links a criterion to timely evidence, diagnosis, affected state, and a feasible bounded repair or escalation.A global score can support selection or stopping but cannot identify a local repair, while precise diagnosis has little value if actions regenerate the entire artifact.
  • Revalidate Affected State: Revalidate Affected State treats acceptance evidence as version-scoped and invalidates dependent evidence or approvals when sources, requirements, tools, state, or authorization change.Selective revalidation avoids discarding accepted work and repeating unaffected checks, but requires explicit change dependencies.
  • Cross-artifact coherence: Changes to claims, assets, style decisions, or audience assumptions can invalidate several coordinated deliverables, motivating versioned dependency and evidence graphs.Current cross-family evidence is too sparse to establish one shared mechanism for invalidation, regression, rollback, and renewed review.
  • Proportionate control: The four principles specify inspectable control relations rather than a maturity ladder or mandatory pipeline, favoring the lightest process that preserves commitments and re-establishes validity at acceptable cost.Their realization remains artifact- and application-specific.

8 Challenges and Opportunities

The section identifies six control problems spanning dependency coherence, diagnosis and repair, system evolution, creator intent, delegated authority, and evaluation of open-ended outcomes. It proposes explicit state, targeted repair, scoped memory, authority-aware delegation, versioned evolution, and plurality-preserving, budget-matched evaluation as responses.

  • Challenges and Opportunities: Six unresolved control problems concern coherence, diagnostic repair, system evolution, creator intent, delegated authority, and evaluation of multiple valid outcomes.These challenges remain hypotheses, so comparative claims should match model, tool, human, and resource budgets.
  • Coherence across Dependent Decisions: Dependency information should become adaptive control state through typed commitment graphs linking requirements, artifact units, decisions, owners, and evidence.The graph can materialize only links touched by changes or judged high-risk.
  • Targeted Repair under Self-Diagnosis Limits: Shared generator, inspector, and reviser weaknesses can hide defects, while critics may detect symptoms without locating causes or safe edits.The resulting failure chain runs from observation to diagnosis to action, so strengthening only one link is insufficient.
  • Targeted Repair under Self-Diagnosis Limits: Diagnostic diversification should use independently grounded evidence, escalate unresolved disagreement, localize faults, and evaluate detection, repair, regression, and recovery separately.Separate reporting prevents accurate localization from concealing weak detection or repairs that introduce regressions.
  • System Self-Evolution beyond Task-Specific Gains: System self-evolution persists changes across episodes, but apparent gains may reflect task order, extra inference, evaluator bias, retrieval, or unobserved regressions.Versioned patches with provenance, scope, rollback paths, held-out shadow testing, and independent authorization are proposed before promotion.
  • Creator Intent under Change: Scoped preference memory should distinguish stable preferences, project commitments, provisional choices, and uncertain inferences while remaining inspectable, correctable, and project-specific.Conflicting feedback should prompt clarification rather than silently overriding retained intent.
  • Authority under Delegation: Authority-aware delegation assigns permission to decisions and actions, with review following ownership and consequence rather than relying on a single human-in-the-loop flag.Action-level provenance should expose the source, decision, approval, and tool involved.
  • Evaluation of Open-Ended Outcomes: Plurality-preserving evaluation should version fixed constraints, evolving preferences, evidence, and evaluator roles without forcing valid outcomes into one ranking.Trajectory comparisons also require matching tools, model calls, feedback, compute, and human effort, since matching model calls alone is insufficient.

9 Conclusion

The survey defines agentic artifact creation as stateful construction in which observations of an evolving artifact redirect later work through an operational representation, construction policy, and runtime verification. Across six artifact families, it identifies coupled decisions, late or unexplained failures, and overly coarse edits as recurring challenges, while emphasizing explicit commitments, accountable control, targeted repair, and revalidation.

  • Conclusion: Agentic artifact creation links an Operational Representation, Construction Policy, and Runtime Verification so observations can redirect later construction.Their alignment determines whether evidence can localize failures and direct feasible edits.
  • Conclusion: Across six artifact families, recurring difficulties include tightly coupled decisions, late or causally unclear failures, and edits too coarse for local repair.The survey treats these challenges as extending beyond modality alone.
  • Conclusion: Four principles address these conditions by externalizing commitments, defining control boundaries, linking feedback to feasible repairs, and revalidating affected state.Implementation remains domain-specific, and direct generation is preferable when little accepted state must be preserved or control costs exceed regeneration.
Loading 2608.28122v1…