Source-linked AI summary
Arya Forge: Auditing Validation and Release Approval for Financial Instruction Data
Kushagra Bhatnagar, Deekshith Marla
TL;DR
Financial instruction data can contain stale content or approvals that cover different material, while strict producer contracts can reject valid alternative representations. Arya Forge combines typed records, current-content checks, and manifest-bound approval, and its case study finds enforcement and retention tradeoffs within the tested implementation and workloads.
Problem
Financial instruction records can pass schema checks despite stale answers, mismatched evidence, or approval for different content, while strict wording requirements can reject valid alternatives.
Source: §1 · p. 1Method
Arya Forge combines typed instruction records, current-content validation, and approval coverage bound to a dataset manifest.
Source: §3.1 · pp. 2–3Results
The case study found that strict native contracts reject selected stale-content faults but also remove every tested clean cash-conversion-cycle example and four equivalent DSCR representations.
Source: §8 · p. 13Takeaways & Limitations
The findings support explicit interface specifications, protected approval inputs, and reporting retention by task family within the tested implementation and workloads.
Source: §8 · p. 13The evidence is limited to synthetic, single-project data and selected internal experiments; it does not establish general correctness, authenticated approval, expert-validated quality, or downstream model benefit.
Source: §3.2 · p. 4, §6 · p. 12
Abstract
from arXiv · showhide
Financial instruction-data pipelines must distinguish content validity from permission to release a particular dataset. Arya Forge combines typed records, current-content checks, and approval coverage bound to a dataset manifest. We evaluate its release boundaries in a financial-data systems case study. An integrity audit covers 100,000 synthetic instruction records; a separate arithmetic implementation agrees with all 15,000 numeric records across twelve formula families. In 240 scenarios with 2,400 treatment rows, adding financial and release-record checks to a short-instruction filtering recipe rejects all injected numeric faults and policy holds while retaining 220 of 240 clean controls. The twenty lost controls comprise every tested cash-conversion-cycle example. Nineteen persisted-release cases show that eleven selected negative cases are blocked, four equivalent representations are rejected, and changes to external evidence after binding are outside ordinary Gold promotion's checks. Local processing measurements and controlled checkpoint resumptions characterize cost and recovery through 100,000 input rows. The results expose a tradeoff between strict producer-contract enforcement and useful-representation retention, and identify the protected inputs required for manifest-bound approval. These findings apply to the tested implementation and workloads; they do not establish general financial correctness, authenticated approval, expert-validated quality, or downstream model improvement.
1 Introduction
Financial instruction data can pass schema checks yet contain stale answers, mismatched evidence, or approval for different content; strict producer contracts can also reject valid alternative wording. Arya Forge studies how content checks and manifest-bound approval interact, including their enforcement and retention costs.
1 Introduction
- Schema validity alone can miss changed answers, period-mismatched evidence, or approval covering different content, while strict producer wording can exclude valid alternatives.The introduction frames evaluation around both invalid acceptance and useful-content loss.Source: §1 · p. 1
- A stale PASS flag can admit an edited answer; recomputing supported quantities and checking the released response addresses content validity, distinct from representation compatibility and authorization.The debt-service coverage ratio example illustrates why the stored calculation flag is not enough.Source: §1 · p. 1
- The study asks which checks and trusted inputs bind validation and approval to a release, which constructed failures are rejected or valid representations lost, and what cost and recovery are observed.Independent financial quality and downstream model utility remain separate evaluation targets.Source: §1 · p. 1
- The contribution is measured interaction among financial-record contracts, executable validation, and release gates—not a new cryptographic primitive or an inferred model benefit.It combines component diagnostics, paired filter treatments, persisted-release cases, and supporting integrity, processing, and recovery measurements.Source: §1 · p. 2
2 Related work
Prior work establishes data-processing infrastructure, validation methods, provenance, and release-control techniques, while financial instruction-data studies address datasets and model adaptation. Arya Forge focuses on financial record contracts and approval coverage, without claiming authenticated approval or downstream utility.
2 Related work
- Data-Juicer and DataTrove provide extensible processing infrastructure (Chen et al., 2025; Penedo et al., 2024), while TFDV and Deequ support schema, anomaly, and constraint checks.Forge’s filter study adds its mechanisms to two pinned DataTrove blocks; it neither compares equally informed financial validators nor ranks frameworks.Source: §2 · p. 2
- Prior release-control work includes artifact-bound supply-chain metadata, admission gates, evidence validity, and workflow release policies.These works establish release gating and evidence binding as prior art; Forge adds no signature authentication or incremental dependency analysis.Source: §2 · p. 2
- Provenance audits and revocation systems address licensing, attribution, and record-level revocation, whereas Forge evaluates eligibility at a dataset-release boundary without external revocation discovery.An approval assertion does not independently establish licensing or attribution.Source: §2 · p. 2
- FinQA and FinanceBench supply financial QA resources (Chen et al., 2021; Islam et al., 2023), while PIXIU and FinDAP study financial instruction data and adaptation (Xie et al., 2023).Prior synthetic QA validation also compares evaluated arithmetic expressions to reduce answer-representation sensitivity; Forge instead enforces a narrower native-text contract that rejects some equivalent wording.Source: §2 · p. 2
3 System and trust boundary
Arya Forge organizes typed records through staged promotion and treats Gold as an approval state rather than a quality label. Its manifest-bound gate checks current content and approval coverage, but the operational boundary depends on trusted inputs and differs across promotion paths.
3.1 Typed records and stages
- Raw, Bronze, Silver, and Gold denote source inventory, canonical migration, selected validated candidates, and releases with explicit approval coverage; Gold is not synonymous with high quality.Releases carry manifests and multiple artifact representations, whose hashes detect divergence from recorded references rather than prove source facts or approval correctness.Source: §3.1 · pp. 2–3
3.2 Approval coverage
- Approval coverage binds each retained record to an active source- or record-scoped decision for an exact Silver manifest, with no missing or ambiguous coverage.Authority, accepted identities, and evaluation time remain trusted inputs; ordinary Gold promotion trusts bound external evidence, while supplemental promotion reopens referenced evidence and checks lineage.Source: §3.2 · pp. 3–4
3.3 What is revalidated at promotion
- The shared promotion gate checks current schema, score, route, lineage, findings, sensitive-data patterns, and recognized native numeric contracts, but does not rerun every domain-specific validator.Stored assertions still inform task validation, so fresh numeric and privacy checks are not fresh expert validation of every financial statement.Source: §3.3 · pp. 4–5
3.4 Corpus processing and selection
- Streaming audit and qualification use disk-backed state, but the ordinary small-release builder materializes records in memory; operational quality scores and clean labels do not prove factual truth.Absence of a finding means only that no implemented check rejected a record.Source: §3.4 · p. 5
4 Evaluation design
The evaluation combines integrity and arithmetic audits, component diagnostics, paired filter treatments, persisted-release cases, and controlled local workload measurements. Its designs are bounded by synthetic or authored workloads, selected mechanisms, and specific recovery and deployment conditions.
4.1 Frozen local assets
- An asset audit checks ten local Gold releases, manifest-listed digests, counts, record-set and source-manifest hashes, then independently compares decoded Parquet rows and reconstructed columns with JSONL.The report binds verifier source hashes and runtime versions to observations and records exact source snapshots rather than claiming a bare Git commit reproduces the repaired state.Source: §4.1 · p. 5
4.2 Separate arithmetic implementation
- A separate arithmetic module avoids the production calculator and generator, using rational arithmetic for ratios, loss, and conversion, discounted instalments for EMI, and logarithm/exponential evaluation for CAGR.Typed constraints precede the legacy EMI reference path; agreement reduces shared-code failure modes but retains common mathematical specifications and assumptions.Source: §4.2 · p. 5
4.3 Current-content promotion checks
- Current-content promotion checks recompute supported native numeric cases and bind canonical questions and responses to results; recognized unsupported formats are refused, while unmarked arithmetic is not comprehensively found.Quarantine validation checks current training content and unresolved sensitive-data findings, but its scanner covers only implemented patterns.Source: §4.3 · p. 6
4.4 Overlap detection and near deduplication
- Overlap detection aligns recognized input envelopes while retaining substantive context, values, and prior turns; near-deduplication also requires matching ordered financial-literal signatures.The conservative policy avoids lexical-only rejection of changed values, but may retain equivalent numeric formatting and does not establish semantic equivalence.Source: §4.4 · p. 6
4.5 Targeted component diagnostics
- Component diagnostics replay pre-repair and repaired snapshots on author-selected controls and fault probes, including stale flags, sensitive findings, cross-format prompts, and financially distinct near-duplicates.These retrospective regression cases are not a random test set, independent benchmark, production incidence estimate, or complete historical-environment reconstruction.Source: §4.5 · p. 6
4.6 Controlled filter composition
- The frozen filter study pairs ten treatments across 240 synthetic scenarios, comparing a short-instruction DataTrove recipe with added numeric validation and release-record checks.The final arm checks numeric faults and selected release-record conditions, not the complete raw-to-Gold pipeline; shared templates and repeated paired treatments limit generalization.Source: §4.6 · pp. 6–7
4.7 Persisted release-boundary experiment
- Nineteen frozen persisted-release cases exercise ordinary Bronze-to-Silver and governed Silver-to-Gold promotion, including selected approval, content-mutation, representation, and evidence-change conditions.Content mutations coherently update artifact and record-set hashes and bind fresh fictional approval; two runs compare result bytes and persisted hashes, but the cases are dependent and not representative of real authority.Source: §4.7 · p. 7
4.8 Local processing and recovery experiment
- Local measurements run audit, Bronze migration, and qualification on authored 1K, 10K, and 100K-row workloads; controlled post-checkpoint resumptions compare substantive outputs with uninterrupted runs.The fixed-length synthetic workload tests execution behavior, not realistic diversity or financial correctness; recovery does not model arbitrary process kills or power loss.Source: §4.8 · pp. 7–8
5 Results
The audits find representation consistency across 100,000 synthetic records and arithmetic agreement across 15,000 numeric records, while constructed tests expose both blocked faults and lost valid representations. Persisted-release cases, local processing measurements, and recovery tests characterize boundaries of the evaluated implementation and workloads.
5.1 Asset integrity and composition
- 100,000 records had matching decoded JSONL and Parquet payloads and 1,200,000 matching projected field values; 15,000 numeric records passed separate arithmetic and native-contract checks.These checks establish consistency and agreement for the audited records, not factual validity or an external error rate.Source: §5.1 · p. 8, Table 3 · p. 8
5.2 Repaired boundaries and exact overlap
- All 15 targeted diagnostics conformed under repaired components versus 5 under preserved pre-repair components, while the refreshed overlap audit found no implemented identity, source, or exact-normalized collisions.The diagnostics were author-selected probes, not population accuracy; shared generators and templates remain outside the overlap result.Source: §5.2 · pp. 8–9, Table 4 · p. 9
- A current-content scan of 100,000 local records found no implemented-pattern or retained-flag findings, but does not estimate privacy-detector recall.The low-level release writer is a trusted construction API, not a complete approval authority.Source: §5.2 · p. 9
5.3 Incremental checks on constructed faults
- Adding the release-record gate rejected all constructed numeric faults and policy holds while the SFT recipe retained 220 of 240 clean controls; every arm rejected 240 repetition probes.The augmented arms inherited 20 clean-control rejections, all cash-conversion-cycle examples; all controls in the other eleven families survived.Source: Figure 2 · p. 9, §5.3 · pp. 9–10, Table 5 · p. 10
- All 20 rejected clean controls were cash-conversion-cycle examples, while all 20 controls in each of the other eleven families survived.Repeated derivation/substitution content triggered duplicate-line or duplicated-five-gram criteria; retention therefore concealed complete loss of one tested task family.Source: §5.3 · p. 10, Table 5 · p. 10
5.4 Release boundary: rejection and acceptance limits
- In 19 dependent persisted-release cases, three ordinary controls reached Gold and 11 selected negative cases were blocked, while all four constrained equivalent representations were rejected.Changing external evidence after binding still permitted ordinary Gold promotion; the unchanged bound registry was trusted.Source: Table 6 · p. 10, §5.4 · pp. 10–11
5.5 Processing cost and controlled recovery
- At 100K raw rows, median audit, Bronze, and qualification times were 31.36, 13.48, and 4.02 seconds; all three 10K interruption/resume comparisons matched uninterrupted substantive outputs.The measurements cover staging and rule execution, not production of approved training data; Figure 3 reports local measurements, not a scaling law.Source: Figure 3 · p. 11, §5.5 · p. 11
5.6 Reproducibility and scope
- The installed code-only distribution passed 182 shipped tests, but full research inputs remain private and exploratory AI judgments are not expert labels or population defect estimates.The data-inclusive extraction reproduced selected results on the same host; the suites do not constitute independent replication.Source: §5.6 · pp. 11–12
6 Limitations and claims boundary
The study’s evidence is bounded by synthetic, single-project data, limited verification and evaluation designs, and unavailable or unexecuted broader evaluations. These constraints support a bounded systems claim, not broad financial validity, framework superiority, or causal downstream model benefit.
6 Limitations and claims boundary
- The verified dataset is synthetic and single-project, and two implementations can share a specification error.Most nonnumeric domains lack independent expert truth labels; sensitive-data scanning and lexical deduplication also have limited coverage.Source: §6 · p. 12
- Nineteen release cases share two source records and do not test hostile deployment or authenticated approval.The comparative study uses two DataTrove blocks, and its rejection labels derive from explicit perturbations and policies rather than blinded expert truth.Source: §6 · p. 12
- Full-corpus bundles, independent human labels, natural-defect comparisons, and matched-budget model experiments remain unavailable or unexecuted.Performance observations are limited to one laptop and a repetitive authored workload; private research artifacts limit external verification, supporting only a bounded systems claim.Source: §6 · p. 12
7 Broader impact and assistance disclosure
The authors caution that correct arithmetic and a clean gate report do not establish suitability for consequential financial decisions. They disclose AI assistance in software, experiments, drafting, and exploratory review, while retaining responsibility for scientific claims and release decisions.
7 Broader impact and assistance disclosure
- A clean gate report cannot substitute for substantive financial review or certify suitability for lending, compliance, or customer decisions.Training data may still contain unsupported decisions, omitted risks, misleading precision, or jurisdictional assumptions despite correct arithmetic.Source: §7 · p. 12
- AI assistants supported software work, experimental runners, drafting, and exploratory review; human authors retain responsibility for final claims and permissions.The separate reviewer agents shared a model family and were not qualified human participants.Source: §7 · p. 12
Funding and competing interests
The work was funded by Arya.ai, and both authors are affiliated with the system’s developer. No external financial experts participated in the reported AI-assisted review.
Funding and competing interests
- Arya.ai funded the work, and both authors are affiliated with the system’s developer.No external financial experts participated in the reported AI-assisted review.Source: Funding and competing interests · p. 12
8 Conclusion
The case study links financial instruction-record checks to manifest-scoped release approval and measures their interaction. Its results show that selected stale-content faults can be rejected, while strict checks can remove useful representations; broader generality and model benefit require separate study.
8 Conclusion
- Fresh native-contract validation rejects selected stale-content faults, while the composed recipe removes every tested clean cash-conversion-cycle example.The strict contract also rejects four equivalent DSCR representations.Source: §8 · p. 13
- The case study connects financial instruction-record checks to manifest-scoped release approval and measures their interaction.Ordinary and supplemental promotion paths differ in when they read approval evidence.Source: §8 · p. 13
- The findings support explicit interface specifications, protected approval inputs, and retention reporting by task family.Integrity audits and controlled recovery provide supporting implementation evidence.Source: §8 · p. 13
- Independent examples, qualified labels, and equal-input mechanism comparisons are identified as next tests of generality.Model benefit requires a separate training study.Source: §8 · p. 13
Data and code availability
The code distribution is separate from the private research evidence, and the exploratory review uses two same-base-model agents whose judgments are explicitly not expert-validated.
Data and code availability
- The Apache-licensed code package contains the allowlisted implementation, documentation, integrity manifest, and portable tests, while research populations and supporting evidence remain private.No public repository, dataset, artifact DOI, or controlled reviewer-access arrangement is claimed.Source: Data and code availability · p. 13
A.1 Exploratory AI-assisted record review
- Two fresh agents reviewed the same 200-record retained packet using only a shared rubric and visible conversations, with labels locked separately before coordinator analysis.They lacked the coordinator’s domain mapping, pipeline scores, reference judgments, manuscript, and each other’s annotations.Source: §A.1 · p. 14
- The rubric covers numeric correctness, evidence support, jurisdiction/date applicability, and training usefulness, but the judgments are not independent expert assessments or established error rates.The model/API revision and sampling seed were not pinned, so preserved labels enable analysis replay rather than deterministic recreation.Source: §A.1 · p. 14
A.2 AI review agreement and unresolved findings
The paired AI review reports disagreement and arithmetic checks without treating agreement as a truth label; unresolved completeness, evidence, and usefulness judgments remain.
A.2 AI review agreement and unresolved findings
- At least one adverse label appears on 61 of 200 records, while the reviewers jointly label 31 not useful; these are sample descriptions, not population defect estimates.Reviewer A labels 151 useful, 34 not useful, and 15 uncertain; reviewer B labels 128 useful, 32 not useful, and 40 uncertain.Source: §A.2 · p. 15
- Both reviewers report no arithmetic mismatch, but their counts distinguish checked values from explicit comparisons: A records 395 checked values including 392 comparisons, and B records 395 comparisons.All forty numeric-label disagreements arise from differing interpretations of credit confidence scores whose numerical meaning is unspecified.Source: §A.2 · p. 15
- The reviewers flag possible omissions, unsupported decisions, and unstated assumptions as candidates for qualified review, not adjudicated findings; agreement is not accuracy.Automated arithmetic conformance therefore coexists with unresolved judgments about completeness, evidence, and usefulness rather than a single quality score.Source: §A.2 · p. 15
Appendix B. Further evaluation protocol
The appendix proposes stronger independent evaluations while documenting bounded reproduction checks, private evidence, and artifact-access limits; the reported checks do not establish broader validation or replication.
Appendix B. Further evaluation protocol
- Stronger claims require unexecuted extensions, including independently authored representation holdouts, equal-input mechanism comparisons, and clustered uncertainty across paired treatments.The proposed protocol also calls for qualified human assessment, controlled model-utility comparisons, and broader systems and recovery testing.Source: Appendix B · pp. 15–16
- The proposed human review and model-utility evaluation remain plans: no qualified-human ratings were collected, and model comparisons require fixed training conditions and independent adjudication.Systems extensions would test heterogeneous workloads, termination between checkpoints, chained recovery, concurrency, and external approval or revocation updates.Source: Appendix B · p. 16, Appendix E · p. 17
Appendix C. Detailed processing measurements
- Table 8 describes three repetitions per cell on the authored laptop workload, with qualification reading 0.9N canonical rows and time excluding interpreter startup.Peak RSS includes interpreter startup.Source: Table 8 · p. 16
D.1 Recorded reproduction checks
- Portable checks reproduced on the same host, while a separate data-inclusive extraction reproduced selected 100K audit and overlap outputs and passed 190 focused tests.The code-only wheel passed 182 shipped tests in a fresh environment, but these checks do not constitute independent laboratory or new-machine replication.Source: §D.1 · p. 17
D.2 Unavailable corpus evidence
- Primary full-corpus bundles were unavailable in the evidence inventory, so larger documented counts are excluded from verified totals and aggregate rejection cannot be read as an invalid-data rate.Exclusion for unbound rights evidence does not establish that a source legally forbids use.Source: §D.2 · p. 17
Appendix E. Research artifact documentation
- The study uses synthetic records and authored cases rather than an independently authored financial benchmark; no qualified-human labels were collected, and research artifacts remain private.The code-only distribution cannot reproduce the full empirical study without private inputs; a portable core experiment still requires cleared materials and supporting outputs.Source: Appendix E · pp. 17–18
Appendix F. Artifact map
- The confidential artifact map locates frozen experiment evidence and separate review preparations, while the identified Apache-licensed code distribution—not the surrounding archive or review kit—is the public-release candidate.The map’s evidence locations are private, not public download links.Source: Appendix F · p. 18