Avatar: Toward Autonomous End-to-End Orchestration of Scientific Workflows using LLMs

I. INTRODUCTION

• Without shared boundaries for agent observation, decisions, and action validation, agent integrations can be costly to adapt and difficult to compare across systems and conditions.Mature WMSs are operationally complex, making substantial architectural changes costly and time-consuming.Source: §I · p. 1
• Avatar separates orchestration into an orchestrator, executor, and provenance component, with pluggable conventional, LLM-backed, or hybrid policies.This provides a common basis for studying where agentic reasoning belongs and how it interacts with conventional workflow mechanisms.Source: §I · p. 1
• Avatar is implemented as an extensible Academy prototype for integrating and comparing agents, tools, monitoring interfaces, and validation strategies.The framework supports systematic comparison of different integration choices.Source: §I · p. 2
• Up to 55% less wasted computation and 40% less GPU-busy time are reported for agentic reasoning across evaluated workloads and objectives.The paper evaluates diverse workloads, orchestration structures, and objectives.Source: §I · p. 2

II. RELATED WORK

• Many systems confine agents to particular workflow stages, retaining validation, scheduling, and execution authority in established WMS components.This limits the consequences of incorrect agent decisions but restricts continuous participation in orchestration.Source: §II · p. 2
• General-purpose reasoning frameworks do not by themselves address scientific workflows’ dependencies, heterogeneous resources, failures, provenance, and runtime integration.Connecting their reasoning to workflow state, decisions, and execution remains necessary.Source: §II · p. 2
• Existing AI-workflow integrations are fragmented and system-specific, leaving each WMS to decide independently where agents observe, act, and undergo validation.This repetition requires architectural modification and complicates systematic evaluation across models, workflows, and infrastructure conditions.Source: §II · p. 2

A. Deriving the Reference Architecture

• Scientific workflow systems separate logical abstractions and decisions from runtime mechanisms that realize the computation.The framework survey organizes representative systems around these abstractions, decisions, runtime realizations, and exposed attributes.Source: A. Deriving the Reference Architecture · p. 2, Table I · p. 3
• Avatar maps workflow evolution, concrete execution, and decision-relevant evidence to an orchestrator, executor, and provenance component.The orchestrator manages logical abstractions; the executor realizes decisions on infrastructure; provenance records outcomes and emits triggers for reconsideration.Source: A. Deriving the Reference Architecture · p. 3
• These are normalized responsibilities, not necessarily modules that existing workflow systems expose under the same names.Systems can implement the shared progression from logical dependency reasoning to runtime realization through different abstractions.Source: A. Deriving the Reference Architecture · pp. 3–4
• Each responsibility can use conventional, LLM-backed, or hybrid policies behind common interfaces and validation boundaries.This makes Avatar a common basis for comparing where agentic reasoning is introduced and what trade-offs it creates.Source: A. Deriving the Reference Architecture · p. 4

B. Actor-based Realization

• Avatar implements orchestrator, executor, and provenance as actors whose agents can use conventional algorithms, LLM reasoning, or hybrids.Existing planners, executors, and provenance managers can be wrapped and compared with agentic implementations without changing the surrounding architecture.Source: B. Actor-based Realization · p. 4
• Actors exchange messages through a shared fabric that decouples components and makes interactions explicit.Message schemas, routing policies, and access controls can vary without modifying component internals or the overall architecture.Source: B. Actor-based Realization · p. 4
• Adapters translate incoming data and validate outgoing actions, limiting what agents observe and can do in the runtime.They can filter state, normalize provenance, validate action formats, reject unsafe calls, or require checks before execution.Source: B. Actor-based Realization · p. 4
• Varying actor implementations, exposed state, tools, and message-fabric policies enables comparisons from conventional baselines to more autonomous orchestration loops.Explicit observation, reasoning, action, and validation boundaries also provide a practical blueprint for agentic workflow orchestration.Source: B. Actor-based Realization · p. 4

Actors

• The orchestrator releases ready tasks and maps failure diagnoses to validated recovery decisions, while the executor dispatches tasks and provenance tracks attempts and lineage.On failure, the provenance actor emits a trigger carrying its policy's diagnosis; the orchestrator maps it to a recovery decision and enacts it after validation.Source: §IV · p. 4

Policies, adapters, and modes

• Each actor uses a pluggable rule-based or LLM-backed policy, and an adapter validates proposed actions against one fixed catalog.The catalog includes retry, migrate, blacklist-worker, replicate-task, scale-out/in, batch, throttle, and campaign-steering actions.Source: §IV · p. 5
• M0 uses rules throughout; M1 adds LLM diagnosis in provenance while rules act; M2 also lets the LLM decide in the orchestrator.The executor continues to actuate by rule in M2.Source: §IV · p. 5

Scope

• LLM-driven executor placement across heterogeneous resources is left for future multi-node tests because this paper uses single-node, homogeneous testbeds.Those testbeds offer no real placement choice.Source: §IV · p. 5

Objectives and criteria

• Avatar is assessed for reproducing conventional WMS behavior, supporting diverse workflows with unchanged actors and interfaces, and identifying when agentic reasoning is beneficial.Representativeness uses correctness and behavioral agreement; generalizability uses execution of diverse workloads with the same actors and interfaces.Source: A. Methodology · p. 5

Criteria and system configuration

• Evaluation measures workload-specific costs and outcomes, including retries and wasted worker time, backlog and utilization, and molecular-design outcomes and GPU usage.Experiments run on the University of Chicago RCC midway3 cluster, with E3 using one GPU.Source: A. Methodology · pp. 5–6

Workloads and baselines

• The resilience workload is a fault-prone fan-out/fan-in DAG with unreliable map tasks, transient or permanent failures, and recovery-history diagnosis.By default, p = 0.35 and x = 1/3; Avatar and native TaskVine use the same retry ceiling in this setup.Source: A. Methodology · p. 5
• The scaling workload is a streaming analyze-and-steer workflow whose inputs arrive in four phases, each triggering preprocessing, parallel analysis, and aggregation.Each input creates four parallel analysis tasks; the phases are low, burst, high, and recover.Source: A. Methodology · p. 5
• The active-learning workload steers a Colmena campaign that selects QM9 molecules by predicted HOMO–LUMO gap and evaluates batches using a deterministic lookup.It uses 5,000 candidates, a 100-molecule seed set, and batches of 32; steering proposals are catalog actions validated before execution.Source: A. Methodology · p. 5

Baselines

• Each workload is paired with its conventional management mechanism: TaskVine for resilience, Parsl for scaling, and Colmena for active learning.The E2 Parsl setup uses fixed pools of 4, 12, or 32 workers; the E3 Colmena campaign has a fixed 12-round budget.Source: A. Methodology · p. 5

Representativeness

• Avatar M0 matches native TaskVine’s retry, completion, and permanent-failure counts for every tested (N, seed) pair, preserving the recovery behavior relevant to the experiment.At N=256, both produce 88 retries, 231 completions, and 26 permanent failures; M0 uses approximately 19% fewer worker-seconds because its executor avoids some dispatch overhead.Source: B. Experimental Results · p. 6

Generalizability

• Across E1 and E2, Avatar M0 completes the same number of tasks as native Parsl, with consumed worker-seconds differing by less than 2%.Together, the tested workflows contrast a batch-oriented, fault-prone DAG with a streaming workload with bursty arrivals and changing resource demand.Source: B. Experimental Results · p. 6
• Across E1–E3, one unchanged Avatar core uses a shared action catalog, with workloads exercising largely disjoint resilience, scaling, and scientific-steering actions.E1, E2, and E3 connect through workload-specific interfaces; noop is their only shared action.Source: B. Experimental Results · pp. 6–7, Figure 4 · p. 6

E1: Resilience

• M2 uses failure history to identify permanently failing tasks and give up after one retry, making wasted computation effectively invariant to retry budget R.Its advantage over blind retry grows as failure probability p or permanent-failure fraction x increases; the greatest benefit occurs when failures are frequent and unrecoverable.Source: B. Experimental Results · p. 7

E2: Scaling

• For E2, M2 reaches a mean backlog of approximately 99, while the rule-based autoscaler reaches approximately 21; reasoning latency is costly in this high-frequency control loop.Among fixed pools, w4 has mean backlog ≈171, while w32 maintains near-zero backlog by provisioning approximately 2.5× the compute it uses.Source: B. Experimental Results · p. 7

E3: Active Learning

• At round 7, M2 reaches the same best HOMO–LUMO gap (0.3997) as the 12-round baseline and reduces GPU-busy time by approximately 40%.For E3, M2 detects convergence from learning progress; the authors contrast this infrequent decision with E2’s fast scaling loop.Source: B. Experimental Results · p. 7

• Avatar reproduces native TaskVine and Parsl execution in rule mode while supporting static fault-tolerant DAGs, dynamic streaming workflows, and GPU-based active learning.Source: §VI · p. 7
• LLM policies provide benefits for workflow decisions that require inference.Source: §VI · p. 7
• Future work will extend Avatar to multi-node, heterogeneous environments spanning CPUs, GPUs, and multiple sites.Source: §VI · p. 7
• Planned extensions include executor resource selection and targeted recovery actions based on error types rather than simply retrying failed tasks.Source: §VI · p. 8