PaperlayerView exact revision on arXiv

Source-linked AI summary

Operationalizing Governed Agentic Workflows in Regulated Finance: A Customer Zero Case Study using Zafin AIOS

Shahir Daya, Rob Dunlap, Alexandre Boulenger, Ahmed Abdelmoneim, Salvatore Vella

arXiv:2610.33253v1cs.SEcs.AI

TL;DR

Regulated software development must reconcile probabilistic AI with requirements for reliable, governed production work. This paper evaluates AIOS-assisted workflows in production codebases, reporting effort reductions while governance controls remained intact. The study’s baseline relies on expert estimates rather than a controlled human-led comparison.

  • Problem

    Agent entropy can degrade code quality and cause architectural drift over time, posing a governance challenge for enterprise AI-assisted development.

    Source: §2.3 · p. 5

  • Method

    The study assesses AIOS-assisted production workflows using estimated effort reduction, human intervention, and preservation of governed execution.

    Source: §4.6 · p. 10, §5 · p. 11, §5.6 · p. 13

  • Results

    Mean effort fell by 38% for the pricing execution platform, 76% for the client deal negotiation platform, and 53% overall, with 2.9 human interventions per request and governance controls intact.

    Source: §7 · p. 15

  • Takeaways & Limitations

    The reported effort reductions occurred while maintaining strong governance controls, including auditability, control, and human authority.

    Source: §6 · pp. 13–14

    The baseline uses expert estimates rather than a fully controlled human-led comparison, since parallel production-codebase changes are not feasible.

    Source: §4.9 · p. 11

Abstract

from arXiv · show

Regulated industries such as finance and healthcare face critical governance challenges that hamper the adoption of AI-assisted software development and hinder the transition from experimental pilots to production work. This paper presents Zafin AIOS, an end-to-end orchestration platform and control plane designed to deploy AI within regulated industries such as finance. We provide the architectural principles and an empirical comparison between AI-assisted and human-led workloads. To validate this framework empirically, we conduct a case study analysis of Zafin's internal enterprise workflows, which we refer to as Customer Zero. This paper contributes a production case study of AI-assisted software engineering in regulated industries. We evaluate the results of using Zafin AIOS with two different production codebases, one for the pricing execution platform and the other for the client deal negotiation platform. Our contribution is a customer case study for regulated industries with production codebases and governance, which includes proof of work. The results demonstrate that strictly governed agentic orchestration increases enterprise capacity without compromising control. In the pricing execution platform workflows, the platform provided complete observability and reduced effort by 38\%. In the client deal negotiation platform, it reduced reporting work by 76\% while maintaining all predefined data privacy, role-based access, and audit guardrails. This paper bridges the gap between AI-assisted software engineering theory and the operational reality of compliance. Closing this gap is important to help enterprises take advantage of AI technologies. These results are descriptive of the benefits of using governed workflows in regulated industries.

1 Introduction

Regulated enterprises need to balance generative AI’s speed with deterministic governance, privacy, and auditability. The paper presents Zafin AIOS and evaluates its use on two production codebases, reporting effort reductions while preserving controls.

1 Introduction

  • Regulated enterprises face a gap between generative AI’s speed and agility and requirements for deterministic behavior, privacy, security, auditability, and governance.The codebases at issue are valuable assets, often developed and maintained for decades.Source: §1 · p. 2
  • Zafin AIOS is an end-to-end orchestration platform and control plane designed to enable and scale generative AI in regulated industries.Its stated purpose is to increase capacity to use generative AI without compromising control and governance.Source: §1 · pp. 2–3
  • The study compares AI-assisted effort on a complex Java pricing platform and a predominantly front-end TypeScript deal negotiation platform.The pricing codebase spans many product domains and millions of lines; the second platform integrates with many other apps.Source: §1 · p. 3
  • Effort fell by 38% in the pricing execution platform and 76% in the client deal negotiation platform while privacy, RBAC, and auditability were maintained.The paper describes the study as practitioner estimates of effort saved in production workflows.Source: §1 · p. 3
  • The paper contributes an orchestration platform with regulated-industry controls and an empirical effort evaluation based on estimates agreed by developers and supervisors.It distinguishes its production-codebase study from work using theoretical, sandbox, or small-codebase settings.Source: §1 · pp. 3–4

2 Literature Review

The literature review describes a gap between AI-assisted development in pilots and the quality, security, and compliance requirements of regulated production settings. It identifies governance challenges and four architectural pillars proposed to address them.

2.1 The Tension of Artificial Intelligence in Software Development

  • AI-assisted development has shown promise, but regulated finance and healthcare remain stuck between sandbox pilots and production because of quality, compliance, and security requirements.Failures can bring regulatory penalties and reputational loss [6].Source: §2.1 · p. 4
  • AI’s probabilistic behavior conflicts with deterministic compliance requirements, motivating an explicit, executable enterprise control plane.The review states that leaving process enforcement to a stochastic LLM is unacceptable in regulated environments [11].Source: §2.1 · p. 4

2.2 Current State of AI-Assisted Software Development

  • The model context protocol (MCP) standardizes interfaces between LLMs and enterprise tools, while a central governance layer can enforce access to enterprise resources.AI-assisted development also shifts engineers toward defining intent and mediating AI interaction [16].Source: §2.2 · pp. 4–5

2.3 Governance Challenges in High-Value Production Codebases

  • High-value codebases face hard-to-review generated code, access and supply-chain vulnerabilities, prompt injection [19], and agent entropy that can degrade code quality and architecture [22].Research recommends zero-trust verification and least privilege for agent actions [21].Source: §2.3 · p. 5

2.4 Architectural Pillars for Enterprise AI Acceptance

  • Four proposed pillars are governed orchestration, human oversight, specification-driven development, and telemetry and auditing.The review links these pillars to policy-as-code [21] [23], human oversight, specifications as the source of truth, and forensic traces [33] [34].Source: §2.4 · pp. 5–6

3 The Zafin AIOS Architecture

Zafin AIOS is a control plane that governs agents, models, tools, skills, and knowledge through structured work and configured workflows. It combines scoped access, human gates, traceability, and monitoring to support auditable agent execution.

3.1 Introduction

  • AIOS’s control plane governs agents, models, tools, skills, and knowledge; agents perform work under assigned identities, scoped resources, workflows, and policies.Each action traces to an accountable identity, with least-privilege role-based access controls.Source: §3.1 · p. 6
  • Execution-time controls give agents staff-directory identities, least-privilege access, and an egress boundary; no shared or anonymous principal is used.Structured intents route work to registered agents and models, with model choice determined by task or agent behavior.Source: §3.1 · p. 6
  • The outer workflow fixes step order and gate placement, while agents choose their approach within each step and must provide its required evidence.Runs can be replayed from recorded state, and versioned workflows remain reviewable and auditable.Source: §3.1 · pp. 6–7
  • The system grounds agents in approved institutional knowledge and registered skills and tools, while monitoring captures model, token, latency, and cost information.Completed, operator-reviewed case studies can be published back to the Knowledge Fabric for later retrieval.Source: §3.1 · p. 7
  • Proof of work records plans, rationales, step transcripts, acceptance verdicts, and supporting artifacts; human gates allow review, rework, and clarification.The architecture also describes ongoing evaluation using fixed benchmark sets, weighted rubrics, and scheduled replay.Source: §3.1 · p. 8

4 Methodology

The study compares AIOS-assisted work with human effort estimates in Zafin’s regulated, live production codebases. It treats the findings as descriptive rather than as a controlled comparison, and the AIOS features studied were already over two months old.

4 Methodology

  • The comparative Customer Zero case study examines Zafin’s own internal production codebases rather than synthetic benchmarks.The study was conducted within a regulated FinTech using live production codebases.Source: §4 · p. 8
  • The comparison uses AIOS-assisted execution against estimates of how long the same design-to-production feature work would take.Source: §4 · p. 8
  • The results are descriptive productivity evidence, not a controlled human-led versus AI-assisted baseline.The paper notes that few studies examine AI-assisted software engineering in live regulated production codebases.Source: §4 · p. 9

4.1 Study Setting

  • The study setting uses live, evolving production codebases rather than synthetic benchmarks or hackathons.Zafin’s codebases and software engineering practices must adhere to regulated-industry guidelines; the study also considers proof of work, auditability, and human oversight.Source: §4.1 · p. 9
  • The AIOS features selected were over two months old while the platform and its features were still evolving.Re-running the study as the AIOS codebase evolves is identified as another option.Source: §4.1 · p. 9

4.2 Codebases

The study covers two large production codebases: a complex backend pricing platform and a less complex, integrated deal-negotiation application. It selects real backlog enhancement requests spanning products, task sizes, and complexity levels.

4.2 Codebases

  • The pricing execution platform is a complex backend system, while the client deal negotiation platform is a less complex, large user-interface application.The pricing platform handles complex banking rules and large enterprise volumes; the deal platform supports bankers negotiating pricing across banking domains.Source: §4.2 · p. 9
  • The pricing platform is primarily Java with significant SQL, while the deal negotiation platform is mainly TypeScript and user interfaces.Source: §4.2 · p. 9

4.3 Case Selection

  • The study selected 30 pricing-platform and 19 deal-negotiation enhancement requests from real backlogs, for 49 requests total.Source: §4.3 · p. 9
  • The requests represent real live work formerly done solely by human developers, rather than synthetic benchmarks.The sample provides real-world measurements.Source: §4.3 · p. 10
  • The selected cases span multiple products and codebases, with varied task sizes and complexity levels.Source: §4.3 · p. 10

4.4 Unit of Analysis

Each enhancement request is the unit of analysis, beginning with a product manager’s customer-oriented description. The study records workflow effort and timing, then compares estimated effort avoided with expert estimates of human-led work.

4.4 Unit of Analysis

  • Each enhancement request is treated as one case, beginning with the product manager describing the desired outcome in customer terms.Source: §4.4 · p. 10
  • For each request, the study records product/codebase, contribution across design-to-production work, execution and human retries, and elapsed time.These data allow results to be summarized overall and by product/codebase.Source: §4.4 · p. 10

4.5 Baseline

  • The human-led effort baseline uses expert developer estimates, with the team leader’s consensus used to double-check them.The estimates cover effort without AIOS and effort performed by automation versus human-led work.Source: §4.5 · p. 10
  • The study compares estimated effort avoided against human-led implementation for each real task across production codebases and varying sizes and complexities.Source: §4.5 · p. 10

4.6 Metrics Chosen

The study defines productivity and governance measures for enhancement requests processed through AIOS. Controls include access restrictions, human review opportunities, governed tool access, logging, and reviewable workflow artifacts.

4.6 Metrics Chosen

  • The chosen measures cover estimated effort reduction, workflow execution cycles, user-initiated interventions, and preservation of governance controls.Effort reduction is the estimated percentage of design-to-production work avoided through AIOS assistance; execution cycles count AIOS and human iterations.Source: §4.6 · p. 10
  • Governance measures ask whether privacy, role-based access control, auditability, and proof-of-work controls remained intact.Source: §4.6 · p. 10

4.7 Governance Controls

  • Agents operate under least-privilege role-based access, with defined workflows and human-reviewable handling that provides more reviews for higher-risk or more complex tasks.Source: §4.7 · p. 10
  • Governed egress and tool access accompany human review gates, clarification pauses, action logging, and artifacts including plans, tests, traces, and review records.Review gates are available at intake, planning, and result stages.Source: §4.7 · p. 11

4.8 Analysis Approach

The analysis grouped savings by codebase and used oversight and cycle counts to assess human involvement, while considering productivity alongside regulated-industry requirements. The study’s baseline and short, evolving observation period constrain interpretation.

4.8 Analysis Approach

  • Savings were grouped by codebase, while human oversight and cycle counts tracked intervention and regulated-industry requirements framed the productivity interpretation.Average AIOS savings were calculated across available ERs; human oversight and cycle counts were used to assess human intervention.Source: §4.8 · p. 11

4.9 Limitations

  • The baseline uses expert estimates rather than a controlled human-led comparison, because parallel changes in production codebases were not feasible.Expert reviews and team-leader consensus provide descriptive metrics.Source: §4.9 · p. 11
  • The codebases and AIOS evolved during the study, and ERs were captured over only two months.The authors plan a further study to update the data.Source: §4.9 · p. 11

5 Results

Across 49 real enhancement requests, AIOS reduced estimated effort in both production codebases, with human intervention still present and governance controls maintained. Results are descriptive, based on expert-estimated effort in live, evolving codebases.

5 Results

  • The results cover 49 real enhancement requests and assess estimated effort reduction, human intervention, and preservation of governed execution.The results are presented descriptively rather than as a controlled experiment.Source: §5 · p. 11

5.1 Samples Used

  • 30 requests came from the pricing execution platform and 19 from the client deal negotiation platform, representing distinct production codebase types.Pricing is described as large and complex; negotiation is UI-heavy, less complex, and has more interface rules.Source: §5.1 · pp. 11–12

5.2 Effort Reduction

  • 53% was the average estimated effort reduction across 49 requests; by codebase, reductions were 38% for pricing and about 76% for negotiation.The results varied by codebase; the authors note that differences may relate to codebase complexity and the negotiation platform’s UI intensity.Source: §5.2 · p. 12, Table 1 · p. 13

5.3 Human Intervention

  • 2.9 human interventions per request was the overall average, with a median of 2; humans remained involved in steering, clarification, and review.Pricing averaged 3 interventions per request versus 2.6 for negotiation; at least one intervention means the workflow was not fully autonomous.Source: §5.3 · p. 12

5.4 Execution Cycles

  • 7.8 cycles per request was the overall average, while pricing averaged 6.7 and negotiation 9.4; cycle count alone should not be read as inefficiency.Negotiation had both higher cycle counts and higher mean estimated effort reduction; AIOS may resolve iterations that would otherwise require human intervention.Source: §5.4 · p. 12

5.5 Governance Preservation

  • Governed execution was maintained, and productivity gains did not come from bypassing controls required in regulated financial industries.The workflow included structured intents, least-privilege RBAC, governed tool access, review gates, logged actions, proof-of-work artifacts, and reviewable traces and evidence.Source: §5.5 · pp. 12–13

5.6 Summary of Results

  • Across real production workloads, gains were larger in negotiation than pricing; human oversight remained present and governance controls stayed intact.These are expert-estimated descriptive results, not randomized control trials; the codebases and AIOS were live and evolving.Source: §5.6 · p. 13

6 Discussion

AIOS reduced estimated effort across two production codebases while maintaining governance controls. The results vary by workload, retain human involvement, and are descriptive evidence with limits on causal and broader claims.

6 Discussion

  • Mean effort fell by 38% in the pricing execution platform, 76% in the client deal negotiation platform, and 53% overall.The findings came from Zafin’s regulated production environment, not a sandbox or synthetic benchmark.Source: §6 · p. 13
  • Effort reductions were achieved while maintaining governance controls, including privacy, role-based access, and auditability.The workflows did not achieve savings by bypassing enterprise controls.Source: §6 · pp. 13–14
  • The 38% reduction came from the more complex pricing codebase, while the client deal negotiation platform showed a 76% reduction.The paper cautions that the difference is workload-specific rather than a universal productivity multiplier.Source: §6 · p. 14
  • Human intervention averaged 2.9 per request overall, shifting toward intent setting, clarification, steering, review, and oversight.Execution cycles averaged 7.8 overall; the client deal negotiation platform had both higher mean cycles and higher effort reduction.Source: §6 · p. 14
  • The discussion frames regulated AI adoption as requiring a control plane combining workflow design, access control, evidence capture, human authority, and repeatable governance.AIOS addresses concerns including stochastic behavior, auditability, security, least-privilege access, and human oversight.Source: §6 · p. 14
  • The evidence is limited by expert-estimated baselines, no randomized trial or full parallel human implementation, 49 requests, and one company.The metrics do not capture all dimensions of quality, maintainability, or long-term production impact; the authors characterize the study as operationally realistic but not definitive causal proof.Source: §6 · p. 15

7 Conclusion

The conclusion presents Zafin AIOS as a governed orchestration platform and reports estimated effort reductions across 49 enhancement requests in two production codebases. It treats the findings as descriptive operational evidence for human-supervised AI assistance, while noting that they are not controlled causal proof.

7 Conclusion

  • AIOS combines governed orchestration, proof of work, human authority, granular cost controls, role-based access, traceability, and auditability.Source: §7 · p. 15
  • The Customer Zero case study examined 49 real enhancement requests across two production codebases against expert-estimated human-led effort.Source: §7 · p. 15
  • Mean estimated effort fell by 38% for the pricing execution platform, 76% for the client deal negotiation platform, and 53% overall.Human intervention averaged 2.9 interventions per request, and governance controls remained intact.Source: §7 · p. 15
  • The authors position AIOS as a human-supervised augmentation system and conclude that productivity and governance need not be opposing goals.They emphasize observable, auditable, governable AI-assisted work as the key requirement for regulated industries.Source: §7 · p. 15
  • The results are descriptive rather than controlled causal proof, and further validation across more teams, codebases, and organizations is needed.Source: §7 · p. 15
Loading 2610.33253v1…