PaperlayerView exact revision on arXiv

Source-linked AI summary

From Static Templates to Dynamic Runtime Graphs: A Survey of Workflow Optimization for LLM Agents

Ling Yue, Kushal Raj Bhandari, Ching-Yun Ko, Dhaval Patel, Shuxin Lin, Nianjun Zhou, Jianxi Gao, Pin-Yu Chen, Shaowu Pan

arXiv:2603.22386v1cs.AIcs.CL

TL;DR

LLM systems increasingly use executable workflows that combine model calls, tools, retrieval, code, memory, and verification, creating a need to understand how workflow structure is designed and optimized. This survey frames these systems as agentic computation graphs, organizes methods by structure determination, optimized component, and evaluation signal, and distinguishes templates, realized graphs, and traces. It concludes that workflow structure should be evaluated alongside task outcomes, including cost, robustness, and structural variation.

  • Problem

    Existing workflow-optimization methods target different artifacts and operate at different times, making their designs and results difficult to compare consistently.

  • Method

    The survey introduces an agentic computation graph framework that separates reusable templates, run-specific realized graphs, and execution traces while organizing methods by structural timing, optimization target, and evaluation signal.

  • Results

    The survey identifies a spectrum from static and node-level optimization to subgraph selection, workflow generation, and in-execution editing, with structural plasticity matched to task needs and available signals.

  • Takeaways & Limitations

    Workflow structure is a first-class design object whose evaluation should include task quality, execution cost, reliability, and structural comparability.

  • Takeaways & Limitations

    Node-level prompt tuning can compensate for weak topology, potentially making poor scaffolds appear competitive while increasing cost and reducing robustness.

Abstract

from arXiv · show

Large language model (LLM)-based systems are becoming increasingly popular for solving tasks by constructing executable workflows that interleave LLM calls, information retrieval, tool use, code execution, memory updates, and verification. This survey reviews recent methods for designing and optimizing such workflows, which we treat as agentic computation graphs (ACGs). We organize the literature based on when workflow structure is determined, where structure refers to which components or agents are present, how they depend on each other, and how information flows between them. This lens distinguishes static methods, which fix a reusable workflow scaffold before deployment, from dynamic methods, which select, generate, or revise the workflow for a particular run before or during execution. We further organize prior work along three dimensions: when structure is determined, what part of the workflow is optimized, and which evaluation signals guide optimization (e.g., task metrics, verifier signals, preferences, or trace-derived feedback). We also distinguish reusable workflow templates, run-specific realized graphs, and execution traces, separating reusable design choices from the structures actually deployed in a given run and from realized runtime behavior. Finally, we outline a structure-aware evaluation perspective that complements downstream task metrics with graph-level properties, execution cost, robustness, and structural variation across inputs. Our goal is to provide a clear vocabulary, a unified framework for positioning new methods, a more comparable view of existing body of literature, and a more reproducible evaluation standard for future work in workflow optimizations for LLM agents.

1 Introduction

The survey treats LLM agent workflows as structures to be optimized and compares methods by when structure is determined, what is changed, and what evidence guides updates. It introduces a unified vocabulary and evaluation perspective spanning reusable templates, realized graphs, execution traces, and workflow-level costs and properties.

  • Motivation: LLM systems increasingly coordinate calls, retrieval, tools, code, memory, and verification through executable workflows whose structure governs dependencies and information flow.The survey focuses on workflow structure rather than isolated model-call quality.
  • Static and dynamic optimization: The survey distinguishes static optimization of reusable templates from dynamic selection, generation, or editing of realized graphs at inference time.Static methods include scaffold, topology, and node-level optimization; dynamic methods include subgraph selection, pre-execution generation, and in-execution editing.
  • Unified representation: Agentic computation graphs unify executable LLM-centered workflows, while templates, realized graphs, and traces separate reusable design, per-run structure, and runtime behavior.This separation clarifies which artifact a method optimizes.
  • Taxonomy: The taxonomy organizes methods by graph determination time and plasticity, covering fixed templates, pre-execution generation or selection, and in-execution editing.These dimensions also help position offline-trained generators and runtime subgraph selection.
  • Cross-cutting synthesis: The survey synthesizes optimization targets, evidence sources, and update mechanisms to explain what changes, which signals authorize changes, and how quality–cost trade-offs are managed.The evidence sources include metrics, verifiers, preferences, and traces.
  • Evaluation: Its evaluation protocol separates downstream task performance from graph properties, execution cost, robustness, and structural variation across inputs.The survey compiles 77 in-scope works and 27 workflow-relevant evaluation assets.

2 Conceptual Framework and Taxonomy

The survey models executable LLM workflows as agentic computation graphs and distinguishes reusable templates, run-specific realized graphs, and execution traces. Its taxonomy organizes optimization by when structure is determined and whether node parameters, graph structure, or both are updated.

  • Core abstraction: Agentic computation graphs represent LLM-centered workflows as nodes performing actions and edges encoding control, data, or communication dependencies.The abstraction covers retrieval, code execution, validation, memory, tool use, and message passing.
  • Core abstraction: An ACG template is a reusable executable specification containing nodes, edges, node parameters, scheduling or routing, and admissible activation or edit actions.It defines the structural and parametric space available before a concrete input is observed.
  • Template, realized graph, and trace: A realized graph is the workflow structure used for one run, while an execution trace records states, actions, observations, failures, retries, verifier outputs, and accumulated cost.A query-conditioned generator may reuse only an operator vocabulary or graph-generation policy while producing different realized graphs per query.
  • Quality–cost view: Workflow optimization balances task quality R(τ; x) against execution cost C(τ), with expectations accounting for stochastic execution and run-specific graph generation.For in-execution editing, graph edits become part of the evolving state and trace.
  • Optimization targets: Node-level optimization updates local parameters, graph-level optimization changes structural variables, and joint optimization updates both through simultaneous or alternating procedures.Structural changes can affect topology, routing, branching, scheduling, communication, and admissible edits.
  • Static and dynamic determination: Static methods fix structural degrees of freedom in a reusable template, whereas dynamic methods construct, select, or edit realized-graph structure at pre-execution or during execution.Graph determination time and graph plasticity mode describe when and how runtime structure can vary.

3 Static Optimization of Agent Workflows

Static workflow optimization improves reusable templates or fixed collaboration scaffolds before deployment, spanning graph search, operator-library design, node-level tuning, and joint structure–configuration optimization. Its effectiveness depends on constrained search spaces and trustworthy evaluation, while frozen structures can remain brittle under distribution shift or unanticipated branching.

  • Static workflow optimization: Static methods optimize reusable templates or fixed collaboration scaffolds offline, making them easier to inspect, constrain, ablate, and benchmark under stable budgets.Their fixed structural assumptions can become brittle under distribution shift, tool drift, or unanticipated branching.
  • Graph and template search: AFlow, ADAS, evolutionary multi-agent design, and VFlow search over executable workflow representations using MCTS, code-space proposals, populations, or verifier-guided objectives.These methods differ in representation but all require executable candidates and reliable evaluation.
  • Graph and template search: Learning or refining operator libraries makes the vocabulary itself an optimization target rather than assuming fixed primitives.A2Flow extracts abstract operators from demonstrations, clusters reusable patterns, and refines them for later search.
  • Node-level optimization: Fixed-scaffold methods such as DSPy and black-box prompt optimizers improve node-level prompts, demonstrations, or collaboration behavior without changing the underlying graph.DSPy compiles prompts or demonstrations against a user-specified metric, while other methods use textual-history or evolutionary search.
  • Node-level optimization: Node-level tuning is practical and often assumption-light, but stronger prompts can mask weak topology while increasing cost and reducing robustness.The survey therefore treats static node tuning as a necessary baseline rather than a complete explanation of workflow effectiveness.
  • Joint optimization: Joint methods alternate or combine topology, prompt, and workflow-level updates because local configuration and structure affect one another.Maestro uses graph edits and node-level updates with numeric scores or reflective textual feedback, while related work relaxes communication graphs for gradient-based optimization.
  • Verification and boundary: Static optimization works best when candidate workflows are constrained enough and evaluation is trustworthy enough to distinguish good structure from bad.When these conditions fail, optimization shifts toward deciding how much structure should remain plastic at inference time.

4 Dynamic Optimization and Runtime Adaptation

Dynamic optimization moves structural decisions from fixed templates toward run-specific selection, generation, and revision. Greater plasticity increases expressivity but also raises validation, credit-assignment, stopping, and budget-control demands.

  • Selection and pruning: Runtime pruning selects task-specific subgraphs from a fixed super-graph, preserving validity while producing sparse, adaptive topologies.This approach can capture substantial cost savings, but cannot add tools, verifiers, or decomposition strategies absent from the super-graph.
  • Pre-execution generation: Pre-execution methods allocate depth, operators, routing, roles, or edges for each query before execution, with preference, reinforcement, and data-centric generation as alternative routes.Hybrid systems may draft workflows before execution and revise them online, combining planning-time expressivity with runtime refinement.
  • In-execution editing: In-execution methods interleave workflow construction with execution, revising sub-goals, operators, roles, edges, or topologies using intermediate feedback.Other approaches use testing, lightweight failure signals, evolutionary search, debate, or reflection to modify workflows during inference.
  • Design spectrum: Dynamic methods form a spectrum from subgraph selection to pre-execution generation and full in-execution editing.Moving toward greater plasticity increases structural expressivity while increasing validation, stopping, and budget-control burdens.

5 Feedback Signals and Update Mechanisms

Workflow optimizers use scalar metrics, verifiers, preferences, and trace-derived textual feedback to accept, rank, or revise structures. The appropriate signal and update mechanism constrain how aggressively structure can be changed.

  • Scalar metrics: Scalar metrics drive black-box search and query-conditioned controllers, but their sparsity, noise, and expense make evaluator quality and action granularity important.Examples include success, accuracy, F1, pass@k, and application-specific rewards, sometimes combined with cost.
  • Verifier signals: Verifiers provide hard constraints, checkpoints, or dense rewards for workflow repair and validity checking.Strong static checks permit more aggressive generation or mutation, whereas weak or expensive verifiers can reward brittle solutions or consume the optimization budget.
  • Preference signals: Preference signals rank workflows or traces rather than assigning independent scalar rewards, helping stabilize optimization when score magnitude is informative or rankings are reliable.Score-aware preferences and semantic-consistency preferences are used to train or select workflow generators.
  • Trace-derived feedback: Textual trace feedback supplies semantic explanations for failures and proposes prompt or graph edits, but can drift, hallucinate, or overfit to individual trajectories.The safest systems use textual feedback for proposals while metrics or verifiers determine whether changes survive.
  • Signal–algorithm matching: The trusted feedback signal often determines safe action granularity: verifiers support mutation, preferences stabilize noisy-score generation, and textual critiques benefit from external validation.Reinforcement learning is most natural when workflow construction is genuinely sequential, while evaluation should report the signals and guards authorizing structural changes.

6 Evaluation and Reporting for Workflow Optimization

Workflow evaluation should treat structure as a first-class output alongside downstream task performance. A reproducible protocol therefore reports graph properties, resource use, robustness, structural variation, and benchmark conditions.

  • Evaluation goals: Structure-aware evaluation asks whether a method generated or selected a good graph, while downstream validation asks whether the resulting system solved the task.These goals answer different questions and should be separated whenever possible.
  • Evaluation goals: Reporting only final task success cannot distinguish better structure from excessive computation, hidden scaffolds, retries, or fallback behavior.A plausible graph may execute poorly, while a task may be solved without analyzing the workflow that produced it.
  • Efficiency: Efficiency reporting should accompany task metrics, including tokens, LLM and tool calls, latency, wall-clock time, monetary cost, and cost-per-success.Cost-per-success normalizes resource use by utility.
  • Graph properties: Graph-level reporting should cover template size, depth, width, critical path, communication volume, edit count, editing fraction, and structural variation.Dynamic methods especially require variation measurements across repeated samples or paraphrases.
  • Robustness: Robustness tests should include paraphrases, tool failures, retrieval noise, API or schema drift, unseen tools, larger registries, and strict budget caps.Dynamic replanning can otherwise degenerate into uncontrolled cost growth.
  • Benchmark design: Benchmarks need clear task specifications, stable harnesses, budget controls, documented tools and side channels, canonicalization rules, and structure-comparison procedures.These requirements reduce inconsistent scoring of semantically equivalent workflows.

7 Synthesis: Design Trade-offs and Practical Guidance

The survey frames workflow optimization as a trade-off between reusable static search and increasing levels of runtime plasticity. Practical guidance is to add structural flexibility only when task heterogeneity or environmental uncertainty requires it, while reporting stronger structural and cost baselines.

  • When static is enough: Static optimization is often sufficient when the operator space is constrained, evaluators are trustworthy, and workloads are repetitive.Under these conditions, reusable templates can be cheaper, easier to debug, and easier to benchmark than runtime generation.
  • When static is enough: Static methods become limited under tool drift, environment shift, or strongly heterogeneous task structure because a fixed design cannot determine when another design is needed.Their strength in optimizing a fixed design does not provide run-specific structural adaptation.
  • Choosing plasticity: Runtime selection or pruning fits workloads that vary mainly in difficulty, communication budget, or required model strength within a known motif library.Pre-execution generation is preferable when queries require genuinely different decompositions, communication patterns, or operator sequences.
  • Choosing plasticity: In-execution editing is reserved for interactive environments where partial execution reveals information that changes the appropriate next steps.Its additional expressivity comes with larger search spaces, stronger validity dependence, and harder structural credit assignment.
  • Graph-level intervention: Prompt optimization cannot replace missing verification, decomposition, communication structure, or control flow; prompt and topology updates should inform each other.Graph-level intervention is higher leverage when failures are structural rather than purely instructional.
  • Verification: Verifiers are most valuable when cheap and semantically meaningful, such as unit tests, schema checks, executability checks, and synthesizability checks.Their value declines when they are weak proxies for downstream success or too expensive.
  • Practical recipe: A practical recipe begins with a constrained scaffold, adds graph search for structural failures, prefers selection before generation, reserves editing for environmental uncertainty, and then compresses communication.The survey presents this as a recurring empirical pattern rather than a universal rule.

8 Open Problems and Future Directions

The survey identifies unresolved challenges in structural credit assignment, representation design, continual adaptation, benchmark quality, and theory for workflow optimization.

  • Structural credit assignment: Structural credit assignment remains difficult because gains may arise from changed edges, verifiers, role prompts, or increased computation.The survey calls for counterfactual replay, efficient ablation estimators, and critics combining traces with graphs.
  • Expressivity versus verifiability: Expressive workflows improve flexibility but are difficult to compare and validate, while constrained representations improve executability and reproducibility but may exclude powerful solutions.The central tension is balancing expressivity with verifiability.
  • Continual adaptation under tool and environment drift: Tool and environment drift requires static templates to be repaired and dynamic generators to use drift-aware policies, but adaptation efficiency is rarely reported.The survey proposes controlled drift benchmarks to study recovery costs rigorously.
  • Data and benchmark quality for workflow research: Workflow optimization depends heavily on evaluator and artifact quality because leakage, inconsistent reference workflows, and brittle evaluator incentives can be exploited.Structure-aware evaluation additionally requires coordination among canonicalization, semantic equivalence, and execution validity.
  • Toward theory for workflow optimization: The field lacks a theory explaining when dynamic generation is necessary, when static templates suffice, and how sample complexity scales with structural plasticity.Such theory could clarify when runtime edits justify their added complexity relative to fixed scaffolds.

9 Conclusion

The conclusion frames LLM agent systems as executable workflows and unifies static and dynamic optimization through a structure-centered taxonomy. It distinguishes templates, realized graphs, and traces while positioning workflow structure as central to capability, cost, reliability, and comparability.

  • Conclusion: The survey conceptualizes agentic systems as executable workflows and distinguishes static templates from dynamic structures conditioned on inputs or runtime feedback.This taxonomy unifies prompt compilation, topology pruning, workflow generation, and runtime editing within a quality–cost framework.
  • Conclusion: Workflow optimization is organized around reusable templates, run-specific realized graphs, and execution traces.Graph determination time and graph plasticity mode refine the static–dynamic distinction, including gray cases such as offline-trained generators and runtime subgraph selection.
  • Conclusion: Metric search, verifier guidance, preference learning, and trace-derived feedback can be understood as learning better local policies, better structures, or both.
  • Conclusion: Workflow structure should be treated as a first-class design object because it directly affects capability, operational cost, reliability, and scientific comparability.The survey provides foundations for analyzing, optimizing, and evaluating these structures.

A.1 Supporting Tables

The supporting tables catalog methods and resources used to compare workflow optimization approaches, covering node-level prompt optimization, adjacent routing and pruning, and background frameworks.

  • A.1 Supporting Tables: The appendix collects catalog tables as supporting material so the main workflow-centered argument remains focused.It also records the classification-card convention used throughout the survey.
  • A.1 Supporting Tables: Table 6 catalogs node-level prompt optimization methods commonly used inside larger workflows.
  • A.1 Supporting Tables: Table 7 catalogs adjacent methods for selection, pruning, sparsification, and routing.
  • A.1 Supporting Tables: Table 8 catalogs background frameworks, scaffolds, and strong baselines that anchor workflow experiments.

A.2 Literature Classification Convention

The literature classification convention uses a stable classification card and explicit terminology to make comparison tables internally consistent.

  • A.2 Literature Classification Convention: Main-text tables use a stable classification card instead of paper-specific column designs.
  • A.2 Literature Classification Convention: The convention operationalizes terms including ACG template, GDT, GPM, and the scope tags core, adjacent, and background.
  • A.2 Literature Classification Convention: The section makes coding rules explicit so the main comparison tables remain internally consistent.
Loading 2603.22386v1…