PaperlayerView exact revision on arXiv

Source-linked AI summary

Graph, Loop, and Harness Engineering for Zero-Trust Agentic Data Engineering and Analytical Processing

Sagar Srinivas Sakhinana, Venkataramana Runkana

arXiv:2609.29668v1cs.LGcs.AI

TL;DR

End-to-end agentic data engineering and analytical execution require reliable coordination across code, data, infrastructure, and runtime environments. The paper proposes two zero-trust frameworks that gate workflows on verification evidence and evaluates them under nominal and controlled-failure conditions. Results show model-dependent verified completion and recovery alongside consistently high authorization enforcement and correct recovery-exhaustion termination.

  • Problem

    Agentic data workflows require reliable coordination across code, data, infrastructure, and runtime environments, while failures can arise across these domains and in analytics.

    Source: §1 · p. 1

  • Method

    Two zero-trust frameworks use evidence-gated workflow progression, bounded recovery, and an agent harness to generate cloud data-engineering solutions and verify analytical answers over governed data products.

    Source: §1 · p. 2, §2 · p. 2, §2.1 · p. 4

  • Results

    Verified completion and repair vary by model, while DEZTER remains between 0.98 and 1.00 across models and recovery-exhaustion termination reaches 1.00 for every model.

    Source: §4.1 · pp. 8–9

  • Takeaways & Limitations

    Terminal success is determined by independently established verification evidence rather than agent-reported success.

    Source: §1 · p. 2, §5 · p. 9

Abstract

from arXiv · show

Large language model agents increasingly automate data workflows, but end-to-end cloud data engineering and analytical execution require reliable coordination across code, data, infrastructure, and runtime environments. We present two zero-trust frameworks. Zero-Trust Agentic Data Engineering generates, deploys, and verifies complete cloud data-engineering solutions from natural-language tasks, with completion conditioned on repository, deployment, runtime, and policy evidence. Zero-Trust Agentic OLAP combines governed Data Preparation with verified Online Analytical Processing (OLAP), permitting production promotion only after validation and evidence-bound approval, and releasing analytical answers only after Same-Snapshot Execution, Exact Result Equivalence, deterministic grounding, and reflection. Both frameworks share three abstractions: graph engineering for evidence-gated workflow structure, loop engineering for bounded recovery, and agent-harness engineering for zero-trust execution. We evaluate both frameworks under nominal execution, controlled failures, bounded recovery, and policy-constrained conditions, measuring verified completion, recovery, authorization enforcement, production promotion, and verified OLAP execution.

1 Introduction

Cloud data workflows span software, infrastructure, runtime, data, and analytics, so agentic automation must verify both data engineering and analytical execution. The paper proposes two zero-trust frameworks and shared abstractions for evidence-gated workflows, bounded recovery, and controlled execution.

1 Introduction

  • Failures across software, infrastructure, orchestration, runtime, data, and analytics can disrupt services, violate security policies, increase cloud costs, or produce invalid outputs.The paper motivates reliable operation through verified Data Engineering and trustworthy analytical execution.Source: §1 · pp. 1–2
  • Zero-Trust Agentic Data Engineering generates and verifies complete cloud solutions, while Zero-Trust Agentic OLAP produces verified analytical answers over governed data products.Data Engineering completion depends on repository, deployment, runtime, and policy evidence; OLAP promotion and answer release have separate validation and verification gates.Source: §1 · p. 2
  • Graph engineering, loop engineering, and agent-harness engineering structure evidence-gated progression, bounded recovery, and zero-trust execution across both frameworks.The paper evaluates framework-specific benchmarks under nominal execution, controlled failures, bounded recovery, and policy-constrained conditions.Source: §1 · p. 2

2 Overall Framework

The frameworks turn natural-language data-engineering and analytical tasks into workflows whose execution, promotion, and completion depend on evidence. Their shared design separates agent reasoning from controlled execution and uses bounded repair when checks fail.

2.1 Zero-Trust Agentic Data Engineering

  • Zero-Trust Agentic Data Engineering generates frontend, backend, pipeline, orchestration, infrastructure, IaC, and test artifacts, then validates deployment, runtime, and policy conditions.Completion requires retained evidence for repository validity, deployment verification, runtime verification, and policy compliance.Source: §2 · p. 2, §2.1 · pp. 4–5
  • Graph orchestration coordinates specialized agents, while tenant-scoped MCP mediates tools and a per-run gVisor sandbox isolates generated workloads.The workflow uses bounded diagnosis, repair, re-planning, and retries, with termination when the recovery budget is exhausted.Source: Figure 1 · p. 3, §2.1 · pp. 4–5

DATA PREPARATION OLAP

  • Data Preparation transforms an immutable source dataset in a bounded BigQuery sandbox, then validates the candidate product before approval and promotion.Validation covers schema, constraints, reconciliation, rejected rows, row counts, resource bounds, and execution status.Source: §2.2 · p. 5, Figure 2 · p. 6

GOVERNED DATA PRODUCT

  • OLAP derives candidate and canonical queries independently from structured intent and runs both against the same fixed snapshot.Answer generation follows only after Exact Result Equivalence; the answer must also pass deterministic grounding and reflection checks.Source: §2.2 · p. 5, Figure 2 · p. 6

EVIDENCE-GATED PROMOTION

  • Evidence-Gated Production Promotion uses a separate least-privilege writer to commit validated output with lineage and approval evidence as a Governed Data Product.An Approval Binding ties the immutable source, approved artifacts, executed SQL, sandbox output, and validation result to promoted artifacts through content checksums.Source: §2.2 · p. 5, Figure 2 · p. 6

2.2 Zero-Trust Agentic OLAP

  • Across preparation and OLAP, distinct execution paths and identities separate sandbox transformations, production writes, and read-only data access.The shared harness retains workflow state and execution evidence, and repairable failures use bounded stage-local recovery.Source: §2.2 · p. 5, Figure 2 · p. 6

3 Experiments

The experiments assess six research questions across separate Data-Engineering and OLAP benchmarks. Conditions, execution environments, recovery bounds, and primary metrics are specified for evaluating nominal completion and controlled failures.

3.1 Experimental Methodology

  • Six research questions test verified completion, recovery and termination, authorization, governed preparation, verified OLAP, and bounded recovery across the two frameworks.For OLAP, the questions include evidence-gated production promotion and Verified Answer generation.Source: §3.1 · pp. 6–7, Table 1 · p. 7

3.2 Benchmark Suites

  • Each benchmark suite contains 100 tasks, and 2,400 task-condition executions cover three RQ-aligned conditions, two suites, and four LLMs.Tasks vary across workload types and requirements; OLAP failures include validation, authorization, query-result mismatch, and answer-verification cases.Source: §3.2 · p. 7

3.3 Experimental Setup

  • Four LLMs are evaluated with fixed task conditions and no model-specific prompt tuning, verification procedures, or acceptance criteria.Data-Engineering retries allow up to 20 attempts; OLAP repair allows up to three semantic attempts, with both bounded by 120 minutes, 10,000,000 tokens, and 500 tool calls.Source: §3.3 · pp. 7–8

3.4 Evaluation Metrics

  • One primary rate per research question measures nominal workflow completion or controlled-condition recovery, authorization, and failure handling.All metrics lie in [0, 1], with higher values indicating better performance.Source: §3.4 · p. 8, Table 2 · p. 8

4 Results and Discussion

Data-Engineering results show model-dependent verified completion and repair, alongside consistently high zero-trust execution and correct termination after recovery exhaustion.

4.1 Zero-Trust Agentic Data Engineering

  • VDECR rises from 0.43 for Gemini 2.5 Flash-Lite to 0.95 for GPT-5.6 Sol, while verified completion remains dependent on model capability.Completion requires repository validity, deployment verification, runtime verification, and policy compliance jointly.Source: §4.1 · pp. 8–9, Table 3 · p. 9
  • DEZTER remains between 0.98 and 1.00 across all models, supporting authorization enforcement largely independent of model capability.The framework denies unauthorized or out-of-scope operations while permitting required authorized capabilities.Source: §4.1 · p. 9, Table 3 · p. 9
  • DERTR ranges from 0.56 for Flash-Lite to 0.97 for GPT-5.6 Sol; repository repair is hardest, deployment repair highest, and exhaustion termination reaches 1.00 for every model.Repair success remains model-dependent, whereas bounded termination is consistently enforced.Source: §4.1 · p. 9, Table 4 · p. 9

5 Conclusion

The two frameworks use shared abstractions for evidence-gated, bounded, zero-trust workflows, with completion and answer release conditioned on retained verification evidence.

5 Conclusion

  • Zero-Trust Agentic Data Engineering verifies complete cloud data-engineering solutions, while Zero-Trust Agentic OLAP combines governed preparation with verified analytical execution.Both instantiate graph engineering, loop engineering, and agent-harness engineering for evidence-gated progression, bounded recovery, and zero-trust execution.Source: §5 · p. 9
  • Data Engineering completion requires repository, deployment, runtime, and policy verification; OLAP promotion and answer release each require their own verification gates.OLAP promotion follows Preparation Validation; answer release follows Same-Snapshot Execution, Exact Result Equivalence, deterministic grounding, and reflection.Source: §5 · p. 9
  • Terminal success is determined by retained evidence rather than agent-reported success.The frameworks couple autonomous generation and repair with isolated execution, explicit authorization, and independent verification.Source: §5 · p. 9

6 Technical Appendix

The appendix reports model-dependent preparation, OLAP, and recovery outcomes, examines the shared control mechanisms through ablations, and describes benchmark task construction and scope.

6.1 Zero-Trust Agentic OLAP

  • GDPCR rises from 0.62 for Flash-Lite to 0.96 for GPT-5.6 Sol, VAR from 0.47 to 0.94, and OZTRR from 0.66 to 0.97; unauthorized writes and reads are denied.Preparation promotion requires validation evidence, while OLAP verifies candidate queries against independently derived canonical queries before answer generation and checks answer grounding and reflection.Source: Table 5 · p. 12, §6.1 · p. 12
  • Query-result mismatch repair is the most difficult OLAP failure condition, followed by answer-verification repair; preparation-validation repair is comparatively easier.Repairable failures remain confined to Stage-Local Bounded Repair.Source: §6.1 · p. 12, Table 6 · p. 12

6.2.1 Ablation Protocol

  • The ablation protocol uses 600 task-condition executions per LLM for each configuration, retaining the original scoring criteria; three ablations add 7,200 executions to 2,400 primary executions.The ablations remove evidence-gated progression, bounded recovery, or online authorization and policy enforcement.Source: §6.2.1 · pp. 12–13
  • Removing bounded recovery produces the largest DERTR decreases, 0.35–0.52, and OZTRR decreases, 0.35–0.41, while removing authorization enforcement drops DEZTER to 0.45–0.61.Removing evidence-gated progression reduces VDECR by 0.12–0.17, GDPCR by 0.13–0.15, and VAR by 0.11–0.14; Table 7 reports the ablation comparisons.Source: §6.2.1 · p. 13, Table 7 · p. 13

6.3 Representative Benchmark Tasks

  • The two 100-task benchmark suites use fixed, checksummed snapshots or deterministic replay sequences held constant across models and executions.Data-Engineering tasks require generated repositories and infrastructure artifacts plus repository, deployment, runtime, and policy verification; OLAP tasks pair preparation requests with analytical questions and independent query verification.Source: §6.3 · pp. 13–14
  • Benchmark difficulty reflects increasing system, workflow, preparation, and analytical complexity rather than dataset size alone.OLAP hard tasks include temporal comparisons, ranking or window semantics, multiple dimensions, and cross-source analysis.Source: §6.3 · pp. 13–14

ANALYTICS QUESTION

The analytics workflow separates preparation, production promotion, and analytics access, then verifies queries and answers before release; representative tasks span multiple datasets and analytical demands.

PREPARATION MCP

PRODUCTION PROJECT BIGQUERY

  • Preparation runs in a per-run BigQuery Sandbox; validated outputs cross a dedicated production-writer boundary, while analytics uses a separate read-only reader identity.The distinct control, sandbox, and production projects enforce least privilege, isolate generated transformations from production data, and keep production-write authority separate from analytics.Source: Figure 3 · p. 15

GOVERNED DATA PRODUCT

  • Representative OLAP tasks pair immutable source data and a natural-language Preparation Request with an analytical question; task level reflects preparation and analytical complexity.The examples include commerce conversion and revenue by month and taxi-weather comparisons by temperature band and precipitation status.Source: Table 8 · p. 16, GOVERNED DATA PRODUCT · p. 17, Table 9
  • Other representative questions cover yearly technology-tag rankings, taxi measures by payment type and time, movie genres with qualifying ratings, and seasonal bike-station rankings.The benchmark examples specify supporting preparation semantics, including normalized question–tag relationships, validated movie ratings, and derived trip time dimensions.Source: GOVERNED DATA PRODUCT · p. 17

QUERY COORDINATOR

  • Analytics planning derives Candidate and Canonical Queries independently from Structured Query Intent, executes them on the same fixed snapshot, and gates answers on Exact Result Equivalence, grounding, and reflection.Repairable failures return to the affected stage through bounded stage-local recovery, with at most three semantic attempts.Source: Figure 4 · p. 18

Related papers

Loading 2609.29668v1…