UnespDataLens-RM: A Reference Model for Analytical Data Engineering with Governance, Quality, Provenance, and Reproducibility

1. Introduction

• The reviewed approaches leave a methodological gap: no identified reference structure jointly relates Analytical Data Engineering processes, cross-cutting capabilities, design principles, artifacts, and evaluation mechanisms.The capabilities include governance, quality, metadata, provenance, versioning, reproducibility, and monitoring.Source: §1 · p. 3
• UnespDataLens-RM is a technology-independent reference model integrating technical-operational processes with capabilities for governance, documentation, traceability, versioning, monitoring, and reproducibility.Its operational scope includes data acquisition, integration, preparation, validation, storage, and delivery.Source: §1 · p. 3
• The study develops UnespDataLens-RM using Design Science Research to address the question of integrating Analytical Data Engineering processes with governance, quality, provenance, and reproducibility.The research question frames the goal as supporting reliable, auditable, and reusable analytical pipelines.Source: §1 · p. 3
• The model comprises eight technical-operational modules and eight cross-cutting modules, complemented by dimensions, artifacts, metrics, and validation criteria.The cross-cutting modules include metadata, governance, provenance, versioning and reproducibility, analytical representations, feature engineering, monitoring, and intelligent support.Source: §1 · p. 3
• The proposed contribution is a common structure linking production processes and cross-cutting capabilities, formalized through design principles, modules, relationships, artifacts, metrics, and validation criteria.The model is intended as a conceptual foundation for future instantiations and domain specializations.Source: §1 · pp. 3–4

2. Background and Related Work

• The approaches address distinct purposes: CRISP-DM and CRISP-ML(Q) structure analytical development, DAMA-DMBOK organizes data-management knowledge, and lakehouse describes an architecture.The paper treats these as complementary approaches rather than equivalent alternatives.Source: §2 · p. 5
• A reference model is an abstract, implementation-independent representation organizing a domain’s elements, responsibilities, and relationships as a basis for different instantiations.UnespDataLens-RM is positioned as a structure for approaches to contribute to specific responsibilities, not as their replacement.Source: §2 · p. 5
• The literature distributes lifecycle capabilities across process models, management frameworks, architectures, and operational practices with different objectives and abstraction levels.The gap is not the individual absence of these capabilities, but their fragmentation across approaches.Source: §2 · p. 5
• Table 1 compares representative approaches by whether capabilities are explicitly covered, partially or indirectly treated, or not specified.The comparison is not intended to establish superiority among approaches.Source: §2 · pp. 5–6, Table 1 · p. 6
• Among the approaches analyzed, no reference structure was identified that explicitly links Analytical Data Engineering processes with governance, quality, metadata, provenance, versioning, reproducibility, monitoring, and evaluation mechanisms.The identified gap concerns systematic articulation of these responsibilities around the lifecycle of analytical assets.Source: §2 · p. 7

3. Research Method

• The article focuses on the first three DSR activities: problem identification, definition of solution objectives, and artifact design and development.Demonstration and empirical evaluation are planned as future work.Source: §3 · p. 7
• The researchers analyzed literature on Data Engineering, governance, quality, provenance, reproducibility, DataOps, MLOps, and analytical architectures, finding lifecycle capabilities distributed across approaches.The approaches’ differing objectives and abstraction levels make articulation in a common methodological structure more difficult.Source: §3 · p. 8
• Solution objectives included integrating technical-operational processes and cross-cutting capabilities while supporting technology independence, modularity, traceability, reproducibility, and governance.These objectives guided the artifact’s design.Source: §3 · p. 8
• The requirements were translated into design principles and an architecture comprising modules, complementary dimensions, artifacts, metrics, and validation criteria.The specification was iteratively refined through component-consistency analysis and checks against needs identified in the literature.Source: §3 · p. 8

4. Architecture of UnespDataLens-RM

• The model integrates data, transformations, responsibilities, controls, and artifacts in a technology-independent structure for organizing analytical assets throughout their lifecycle.Governance, metadata, provenance, and reproducibility accompany production and evolution rather than appearing only as sequential stages.Source: §4 · pp. 8–9

4.1 Derivation and Design Principles

• The architecture links literature evidence to requirements, design principles, and components, making its rationale traceable from the investigated problem to the resulting structure.This traceability is intended to support future evaluation of the artifact against the requirements that guided its development.Source: §4.1 · pp. 9, 12
• The first principles treat data as documented analytical assets, embed governance responsibilities and controls in the lifecycle, and require quality criteria to be considered continuously.They also require continuous recording of asset origins, transformations, dependencies, and evolution.Source: §4.1 · p. 10
• The remaining principles require reproducibility, technology independence, modular responsibilities with reduced coupling, and evolution without changing the model’s structural foundations.Reproducibility depends on preserving information needed to reconstruct processes and results.Source: §4.1 · p. 10

4.2 Structure and Organization of the Modules

• The architecture has sixteen modules—eight technical-operational and eight cross-cutting—and its production flow can be iterative, parallel, or revisited rather than strictly linear.The operational sequence spans source identification to delivery, with execution shaped by validation, changed requirements, or processing problems.Source: §4.2 · p. 12
• Explicit module inputs and outputs define adaptable conceptual interfaces, while cross-cutting capabilities can exchange information across stages and accompany assets throughout their lifecycle.The model specifies responsibilities rather than prescribing a storage technology such as a warehouse, lake, or lakehouse.Source: §4.2 · p. 13
• Modules address governance responsibilities and controls, provenance and lineage, and versioning and reproducibility across analytical assets and processes.Their functions include recording origins, transformations, and dependencies, and preserving versions and reconstruction conditions.Source: §4.2 · p. 14
• Other modules structure analytical representations, manage derived features, and monitor asset and pipeline behavior, stability, and evolution.Feature Engineering applies when analytical applications require derived attributes.Source: §4.2 · p. 14
• Cross-cutting modules are not uniformly implemented; their intensity and form depend on each instantiation, and intelligent agents are optional supporting capabilities.Analytical representations depend on product needs, while Feature Engineering applies when derived attributes are needed.Source: §4.2 · p. 14
• Explicit responsibilities, inputs, outputs, artifacts, rules, metrics, and validation criteria define modular interfaces for conventional software or optional AI-agent implementations.An agent may handle a module or selected activities and interact through the model’s defined artifacts and interfaces; agent use is not required.Source: §4.2 · p. 15

4.3 Complementary Dimensions and Extensibility

• Complementary dimensions include DataOps, MLOps, Data Contracts, observability, Data Discoverability, semantic interoperability, and Schema Drift.They represent cross-component practices or environment-dependent capabilities rather than necessarily forming pipeline stages or independent modules.Source: §4.3 · p. 15
• New practices may begin as complementary dimensions until sufficient theoretical and empirical evidence supports formalizing them as structural components, preserving a stable core while adapting to change.DataOps and MLOps may affect automation, versioning, deployment, and monitoring; Data Contracts can constrain producer-consumer relations, and semantic interoperability can extend integration, metadata, and representation capabilities.Source: §4.3 · p. 15
• The architecture relates literature evidence and design requirements to principles, technical-operational processes, and cross-cutting capabilities to integrate distributed responsibilities.It preserves modularity and technology independence while allowing possible multi-agent instantiations.Source: §4.3 · p. 15

• The model formalizes an Analytical Data Engineering lifecycle spanning source identification and characterization through delivery of assets for consumption.Source: §5 · p. 16
• Governance, metadata, provenance, versioning, and monitoring are integrated as continuous lifecycle capabilities, alongside quality, feature engineering, analytical representations, and observability.Source: §5 · p. 16
• Its modules are associated with intermediate artifacts that document decisions, rules, metrics, transformations, and validation conditions, rather than abstract stages alone.Source: §5 · p. 16
• The design principles link product quality to understanding acquisition, transformation, evaluation, and delivery, supporting auditing, impact analysis, reproduction, and dependency understanding.Source: §5 · pp. 16–17
• Technology independence and modularity support different implementations and progressive adoption suited to organizational maturity, without requiring all capabilities simultaneously.Source: §5 · p. 17
• The model remains a technical-methodological proposition undergoing validation; superiority over existing approaches has not yet been empirically demonstrated.Source: §5 · p. 17
• UnespDataLens-RM’s central contribution is systematically integrating responsibilities scattered across Data Engineering models, frameworks, and practices into a common methodological structure.Source: §5 · pp. 16–17

6. Conclusions

• The authors state that analytical-asset reliability also depends on documenting origin, controlling evolution, assessing quality, and reconstructing the responsible processes.Source: §6 · p. 17
• The model relates responsibilities commonly addressed separately, providing a reference structure to specialize for an application context rather than replacing existing proposals.The authors describe it as technology-independent and adaptable across domains, infrastructures, and organizational maturity levels.Source: §6 · p. 17
• Its modules are formalized through artifacts, metrics, and validation criteria to bring the conceptual specification closer to future operational instantiations.The model components can be materialized in different infrastructures and at different organizational maturity levels.Source: §6 · pp. 17–18
• The current work presents an advanced conceptual specification, but demonstration and empirical evaluation remain to be conducted.It therefore does not establish measurable improvements in quality, governance, traceability, or reproducibility, or superiority over other pipeline-organization approaches.Source: §6 · p. 18
• Future work includes an instantiation and experimental evaluation, incremental adoption strategies and maturity levels, and cross-domain assessment of generality and specialization needs.Further directions include interoperability with standards and approaches such as W3C PROV, FAIR, DataOps, and MLOps, and instantiation architectures.Source: §6 · p. 18
• UnespDataLens-RM establishes a conceptual foundation for treating Analytical Data Engineering as a governed lifecycle of analytical assets.It integrates technical-operational processes with capabilities related to governance, quality, provenance, lineage, versioning, and reproducibility.Source: §6 · pp. 17–18
• The model’s eventual contribution to more reliable, traceable, auditable, and reproducible analytical ecosystems depends on instantiation, experimentation, and evaluation in real-world scenarios.The study did not generate or empirically analyze datasets.Source: §6 · p. 18

Funding

• The study was financed in part by Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – Brasil (CAPES), Finance Code 001.Source: Funding · p. 18

Declaration of Competing Interests

• The authors declare no competing financial interests or personal relationships that could have influenced the reported work.Source: Declaration of Competing Interests · p. 19
• The authors used ChatGPT to support text organization, linguistic revision, writing improvement, article-structure adjustment, and image production under their supervision.They state that the tool did not replace critical analysis, methodological decision-making, or researchers’ scientific responsibility, and that assisted content was subsequently analyzed, verified, and edited.Source: Declaration of Competing Interests · p. 19

Data Availability

• The study presents the design and specification of a reference model and did not generate or empirically analyze datasets.Source: Data Availability · p. 19
• No datasets are associated with the results presented in the article.Source: Data Availability · p. 19