PaperlayerView exact revision on arXiv

Source-linked AI summary

ECAS: An Edge-Controlled Agentic System for Validation-Gated Scientific Application Execution

Baixi Sun, Mingze Xia, Huihuo Zheng

arXiv:2609.14211v1cs.DCcs.AI

TL;DR

HPC deployment requires translating high-level goals into system-specific artifacts, while cloud access and one-shot generation pose trust and adaptation challenges. ECAS separates cloud reasoning from edge control and gates target-scale execution behind validation. In the reported repetitions, the one-shot baseline resolved no faults, while the closed-loop agent resolved every fault.

  • Problem

    HPC deployment requires translating high-level goals into system-specific artifacts, while cloud access raises trust concerns and one-shot generation cannot adapt to runtime failures.

    Source: §I · p. 1

  • Method

    ECAS uses a cloud LLM for proposals and repairs, a user-controlled edge for credentials, policy, and execution authority, and validation-gated promotion to target scale.

    Source: §I · pp. 1–2, C. Validation-Gated Execution · p. 3

  • Results

    One-shot baseline resolves 0/6 faults, while the closed-loop agent resolves every fault in every repetition (30/30 runs).

    Source: C. E2: One-Shot vs. Closed-Loop Agent… · p. 6

  • Takeaways & Limitations

    ECAS provides credential isolation and policy-mediated execution for LLM-driven HPC automation while keeping credentials, state, skills, and execution authority on the user-controlled edge.

    Source: §VII · p. 6

    The preliminary evaluation uses two related ALCF systems, deterministic fault injection, small target scales, and fixed repair, submission, and time budgets.

    Source: §VI · p. 6

Abstract

from arXiv · show

Scientific applications increasingly rely on high-performance computing (HPC), yet translating a scientist's high-level goal into a correct target-scale execution remains brittle and labor-intensive. Large language model (LLM) agents promise to automate this, but two obstacles remain: granting a cloud-hosted model direct HPC access exposes credentials and execution authority, while withholding it demands continuous human supervision; and one-shot generation cannot adapt when generated artifacts fail in a site-specific HPC environment. We present \textsc{ECAS}, an \textbf{E}dge-\textbf{C}ontrolled \textbf{A}gentic \textbf{S}ystem for closed-loop execution of scientific computing campaigns with limited human intervention. \textsc{ECAS} separates \emph{reasoning}, \emph{control}, and \emph{execution}: a cloud-hosted LLM proposes plans, artifacts, and repairs; a user-controlled edge agent retains credentials, workflow state, and execution authority while enforcing policy and resource constraints; and the HPC system computes. Its core mechanism is \emph{validation-gated execution}: generated artifacts pass static checks and small-scale validation, failures trigger repairs from sanitized execution feedback, and target-scale execution is permitted only after validation and policy checks pass. \textsc{ECAS} also draws on an edge-resident library of expert-distilled, site-specific skills that is never disclosed to the cloud. In preliminary experiments with three scientific applications on two production ALCF systems under six injected fault types, closed-loop repair improves application success from 0/6 to 6/6 over one-shot generation, validation gating prevents all three observed target-scale failures, and skill conditioning improves success from 4/6 to 6/6. These results show the feasibility of delegating adaptive reasoning to the cloud while retaining execution control at the edge.

I. INTRODUCTION

HPC campaigns require translating high-level goals into system-specific artifacts, while cloud access and one-shot generation pose trust and adaptation challenges. ECAS separates cloud reasoning from edge control, gates target-scale runs behind validation, and uses expert-distilled site skills to support automated deployment.

I. INTRODUCTION

  • HPC deployment translates a user goal into system-specific source or configuration, software environment, launch command, and batch script, where runtime mismatches can waste queue time and allocation.Examples include incompatible libraries, an incorrect launcher, and paths unavailable on compute nodes.Source: §I · p. 1
  • ECAS keeps credentials, workflow state, execution authority, and site-specific skills at a user-controlled edge while the cloud proposes plans, artifacts, diagnoses, and repairs.The edge mediates access and sanitizes feedback crossing the trust boundary.Source: §I · pp. 1–2
  • Validation-gated execution requires static checks and small-scale validation before policy-approved promotion to target scale, with failures repaired from sanitized feedback.This is intended to catch expensive scale-out failures at small scale before they consume allocation.Source: §I · pp. 1–2
  • Expert–agent interactions are distilled offline into reusable edge-private skills that condition runtime generation and repair, reducing repeated intervention as coverage grows.Expert review remains for genuinely novel failures.Source: §I · pp. 1–2
  • The prototype evaluates MPI-IO and deep-learning I/O workloads on two production ALCF systems under deterministically injected faults, measuring repair, validation gating, and skill conditioning.The evaluation examines the benefits of those three mechanisms.Source: §I · p. 2

II. BACKGROUND AND RELATED WORK

Prior work covers HPC domain question answering, retrieval, agentic execution and repair, scientific-workflow architectures, reusable agent experience, and I/O benchmarks. ECAS positions its contribution as an edge-controlled, validation-gated execution system with private, expert-sourced site skills.

II. BACKGROUND AND RELATED WORK

  • HPC-GPT targets domain knowledge, while HyCE retrieves system-specific answers and predefined commands rather than executing workflows, [4].HyCE also discusses data-privacy and command-execution risks.Source: §II · p. 2
  • Foam-Agent and Foam-Agent 2.0 generate OpenFOAM cases and HPC scripts, monitor and repair runs; ECAS instead uses a user-controlled edge device as trust and control plane and validation-gates scale-up for resource-waste reduction [1], [2],.Source: §II · p. 2
  • ECAS follows the reason–act paradigm of ReAct [6].Source: §II · p. 2
  • Broader agentic-workflow work is largely conceptual or provenance-focused; ECAS contributes a concrete edge-controlled prototype and execution-time evaluation [7],.Source: §II · p. 2
  • Unlike self-generated agent experience, ECAS skills are expert-sourced, encode site-specific operations, and remain private on the edge because they contain site identifiers [9]–[11].The library serves as both a performance mechanism and part of the trust boundary.Source: §II · p. 2
  • The workloads draw on IOR for MPI-IO, DLIO for deep-learning I/O, E3SM-IO for climate I/O, and Darshan for run characterization –.Source: §II · p. 2

III. SYSTEM DESIGN

ECAS divides scientific-computing automation among cloud reasoning, edge control, and HPC execution. The user-controlled edge is the trusted core that materializes and checks cloud proposals and mediates system access.

A. Three-Plane Architecture

  • The cloud LLM returns plans, artifacts, and abstract, site-agnostic repair diagnoses; the edge uses private skills to resolve them into concrete site-correct fixes.The cloud holds no credentials and issues no commands.Source: A. Three-Plane Architecture · p. 2
  • The user-controlled edge owns credentials, workflow state, artifacts, private skills, policy checks, log sanitization, the sole system connection, and execution authority.Every cloud proposal is materialized and checked at the edge, and only sanitized information crosses outward.Source: A. Three-Plane Architecture · p. 2, Figure 1 · p. 3
  • The HPC execution plane runs validation and target-scale jobs through its batch scheduler, trusting only the edge agent to authenticate as the user.Source: A. Three-Plane Architecture · pp. 2–3

B. Threat and Failure Model

The threat model treats the cloud LLM as fallible but non-adversarial and relies on edge-resident checks to bound errors. Target-scale execution follows static checks, small-scale validation, repair from sanitized feedback, and independent policy and allocation checks.

B. Threat and Failure Model

  • The threat model assumes a fallible, non-adversarial cloud LLM, a trusted user-controlled edge, and a trusted HPC site; compromised edges, malicious sites, and sanitized-text side channels are out of scope.The edge checks artifacts and resource requests and sanitizes logs before they cross the boundary.Source: B. Threat and Failure Model · p. 3

C. Validation-Gated Execution

  • Artifacts pass static checks and a minimal-scale validation run; failures trigger bounded repair from sanitized output, and only successful, policy-approved jobs are promoted to target scale.Success is judged by an application-specific oracle rather than scheduler completion alone.Source: C. Validation-Gated Execution · p. 3
  • The edge authorizes target-scale execution only when the validation oracle passes, the request meets node, walltime, queue, and account policy, and the job fits the remaining node-hour budget.The resource footprint includes node count, wall-clock request, queue, and charging account.Source: C. Validation-Gated Execution · p. 3

D. Skill-Building Methodology

ECAS builds site-specific expertise offline through expert-agent work, then reuses distilled skills during automated deployment. The edge-private library reduces repeated expert involvement for covered tasks.

D. Skill-Building Methodology

  • The skill-building phase is offline and human-in-the-loop, while the deployment loop is automated and reuses the resulting site-specific knowledge.ECAS relies on this build-time knowledge because a general-purpose cloud model lacks it.Source: D. Skill-Building Methodology · p. 4
  • An HPC expert corrects stalled or site-nonconforming tasks, and the edge distills each correction into a versioned skill keyed to workload, site, and error signature.Skills can encode compatible modules, launcher templates, queue/account rules, or known-good environment recipes; the library remains on the edge.Source: D. Skill-Building Methodology · p. 4
  • During deployment, the edge retrieves relevant skills to guide cloud generation and repair; covered tasks run without human involvement, while novel failures can return to the expert.A novel task that exhausts its budget escalates, and the expert’s correction seeds a new skill.Source: D. Skill-Building Methodology · p. 4

IV. IMPLEMENTATION

ECAS places credentialed workflow control and policy enforcement at a Python edge controller, while cloud reasoning is accessed through an institutional gateway. Its implementation covers two ALCF environments and three I/O workloads.

IV. IMPLEMENTATION

  • The Python edge controller holds credentials, operates over SSH and PBS, versions artifacts, and applies static, policy, and sanitization steps.The hosted reasoning plane is accessed through an institutional inference gateway.Source: §IV · p. 4
  • The implementation deploys on Crux and Sophia and exercises IOR, DLIO, and E3SM-IO, with E3SM-IO serving as a cross-system portability case.The passage identifies the workloads and their original references: IOR, DLIO, E3SM-IO, and ADIOS2 BP.Source: §IV · p. 4
  • Static checks and the policy gate run on the edge, which sanitizes usernames, absolute paths, and project identifiers before sending logs to the cloud.The edge library stores structured skills keyed by workload and site; retrieved guidance conditions cloud prompts without sending raw library content.Source: §IV · p. 4

V. EVALUATION

The evaluation studies ECAS across two production ALCF systems, three workloads, and injected faults, using application-specific success criteria and fixed experimental budgets. It also defines how validation gating changes the scale at which failures incur wasted node-hours.

A. Experimental Setup

  • The study uses Crux, a 256-core-per-node CPU system with PBS Pro, Cray MPICH, and PALS mpiexec, and Sophia, a GPU system with OpenMPI and eight A100 GPUs per node.Crux hosts end-to-end validation and target-scale runs; Sophia exercises a distinct site environment.Source: A. Experimental Setup · p. 4
  • IOR, DLIO, and E3SM-IO are tested at one-node validation scale before promotion to four-node targets for IOR/DLIO or a 512-process, two-node target for E3SM-IO.E3SM-IO validation uses a 16-process map on one node; the target configuration is map-pinned.Source: A. Experimental Setup · pp. 4–5, Table II · p. 5
  • Six injected fault types include two statically detectable faults, four runtime faults, and two site-specific faults; repair is limited to three attempts per task before escalation.Each fault is run five times per condition in experiments E2 and E4, using Claude-opus-5 for diagnosis and an edge mapping to site-correct fixes.Source: A. Experimental Setup · p. 5
  • Four research questions examine end-to-end completion, repair versus one-shot generation, gating’s avoided target-scale waste, and skill conditioning’s effect on repeated repairs and expert intervention.Experiments use fixed repair, submission, and time budgets and report queue-excluded time alongside counts to control for scheduler-queue variability.Source: A. Experimental Setup · p. 5
  • A run succeeds only when scheduler completion, zero application exit, a nonzero application metric, and a well-formed expected artifact all pass; the study also reports autonomous completion, repairs, and submissions.The workload-specific oracle checks IOR bandwidth, DLIO throughput, or E3SM-IO write bandwidth, plus artifacts such as a Darshan log or an ADIOS2 BP variable listing.Source: A. Experimental Setup · p. 5
  • Under direct submission, a fault fails at target scale, whereas validation gating detects it at validation scale, reducing each failure’s wasted node-hours by the validation-to-target scale ratio.Node-hours are attributed by stage, and the gating advantage grows with target scale and fault-detection latency.Source: A. Experimental Setup · p. 5

B. E1: End-to-End Feasibility (RQ1)

All three workloads completed the Crux end-to-end pipeline from small-scale validation to their target runs. E3SM-IO also passed its artifact check at target scale, and its software stack was built and link-verified on Sophia.

B. E1: End-to-End Feasibility (RQ1)

  • All three workloads—IOR, DLIO, and E3SM-IO—completed on Crux after one-node validation and promotion to their target runs.IOR and DLIO used four-node targets; E3SM-IO used a map-pinned 512-process target on two nodes.Source: B. E1: End-to-End Feasibility (RQ1) · p. 5
  • At the 512-process target, E3SM-IO reached 1112 MiB/s aggregate write bandwidth and its BP output passed the artifact oracle with all 162 F-case variables listed.The E3SM-IO F-case history files were written through the ADIOS2 BP backend at both scales.Source: B. E1: End-to-End Feasibility (RQ1) · p. 5
  • E3SM-IO and its ADIOS2/PnetCDF stack were built and link-verified on Sophia under OpenMPI, exercising the pipeline across an incompatible MPI ABI.The Crux end-to-end runs are reported in Table II.Source: B. E1: End-to-End Feasibility (RQ1) · p. 5

C. E2: One-Shot vs. Closed-Loop Agent (RQ2)

Closed-loop repair resolves the injected HPC faults that one-shot generation leaves unresolved. The comparison covers six statically or runtime-detected faults, with repeated runs for the closed-loop condition.

C. E2: One-Shot vs. Closed-Loop Agent (RQ2)

D. E3: Direct vs. Validation-Gated Scale-Up (RQ3)

Validation gating shifted failures on three runtime faults from target-scale execution to one-node validation. The comparison used 15 runs per strategy.

D. E3: Direct vs. Validation-Gated Scale-Up (RQ3)

E. E4: Cold vs. Skill-Conditioned Agent (RQ4)

Skill conditioning resolved site-specific faults that the cold agent escalated, while the paper limits its claims to preliminary, narrowly scoped evaluation. ECAS concludes that edge control, validation gating, and reusable skills support closed-loop automation with minimal human intervention.

E. E4: Cold vs. Skill-Conditioned Agent (RQ4)

  • 30/30 skill-conditioned runs succeeded versus 20/30 cold runs, including 10/10 versus 0/10 resolutions of site-specific faults.Both conditions averaged 1.00 repairs; the skill library converted two otherwise-unresolvable faults into autonomous fixes.Source: E. E4: Cold vs. Skill-Conditioned… · p. 6

VI. DISCUSSION AND LIMITATIONS

  • Evaluation is preliminary: it uses two related ALCF systems, deterministic faults, small target scales, and skills built by one expert on a small task set.The authors also note that skill retrieval can misfire and generalization across sites, schedulers, experts, and models remains future work.Source: §VI · p. 6

VII. CONCLUSION

  • ECAS combines cloud reasoning, edge control, and HPC execution, gating target-scale runs behind checks, validation, and feedback-driven repair.The preliminary prototype suggests these mechanisms and skill conditioning improve success and reduce wasted node-hours; a full evaluation is ongoing.Source: §VII · pp. 6–7
Loading 2609.14211v1…