Source-linked AI summary
Artificial Intelligence Governance for Businesses
Johannes Schneider, Rene Abraham, Christian Meske, Jan vom Brocke
TL;DR
Businesses face uncertainty about how to generate value from AI as the technology advances and governance challenges span data, models, and systems. The paper synthesizes AI and ML literature into a conceptual governance framework, highlighting practical proposals and areas needing further research.
Problem
Businesses face uncertainty about using AI effectively for value, while rapid advances and regulation complicate governance across data, models, and systems.
Source: Introduction · p. 1Method
The paper synthesizes AI and ML literature into a conceptual framework that treats AI as systems built from data, models, and system-level elements.
Source: Discussion and future work · p. 14, Conclusion · p. 17Results
The review presents a framework and fine-grained governance proposals, including separating AI development, testing, and deployment infrastructure.
Source: Discussion and future work · p. 16, Conclusion · p. 17Takeaways & Limitations
The framework highlights business-relevant governance areas, including data value, whose future performance is difficult to forecast in existing work.
Source: Conclusion · p. 17, Discussion and future work · p. 17The validity of the paper’s practical proposals has yet to be verified, and many AI governance aspects remain unspecified.
Source: Discussion and future work · p. 16
Abstract
from arXiv · showhide
Artificial Intelligence (AI) governance regulates the exercise of authority and control over the management of AI. It aims at leveraging AI through effective use of data and minimization of AI-related cost and risk. While topics such as AI governance and AI ethics are thoroughly discussed on a theoretical, philosophical, societal and regulatory level, there is limited work on AI governance targeted to companies and corporations. This work views AI products as systems, where key functionality is delivered by machine learning (ML) models leveraging (training) data. We derive a conceptual framework by synthesizing literature on AI and related fields such as ML. Our framework decomposes AI governance into governance of data, (ML) models and (AI) systems along four dimensions. It relates to existing IT and data governance frameworks and practices. It can be adopted by practitioners and academics alike. For practitioners the synthesis of mainly research papers, but also practitioner publications and publications of regulatory bodies provides a valuable starting point to implement AI governance, while for academics the paper highlights a number of areas of AI governance that deserve more attention.
Introduction
AI offers businesses potential value, but rapid technical change, regulation, and model risks make effective governance difficult. The paper addresses the lack of practical, corporate-focused guidance with a framework grounded in AI and related research.
- 90% of executives saw AI as a critical business opportunity, but only 10% reported significant financial gains, amid uncertainty about how to use AI effectively.The paper frames this gap as a challenge for organizations seeking to generate value from AI.Source: Introduction · p. 1
- Rapid AI advances and growing regulation make it difficult for companies to identify suitable governance mechanisms across data, ML models, and AI systems.AI outputs can be hard to understand, and data-driven decisions can produce bias and other risks.Source: Introduction · pp. 1–2
- The paper defines business AI governance as rules, practices, and processes intended to align AI technology with organizational strategies and objectives.Its stated aim is to support successful AI use toward objectives such as profitability and efficiency.Source: Introduction · p. 2
- Existing approaches often emphasize societal and humanistic concerns while neglecting corporate and practical needs; the paper responds with a conceptual framework synthesized from AI and ML literature.It also elaborates on model governance, including testing, data valuation, and data quality.Source: Introduction · p. 2
Methodology
The authors combine a systematic search with a narrative review because the initial literature did not cover business-relevant AI governance sufficiently. They map the identified works to IT and data governance frameworks and adapt the categories to fit the reviewed material.
- The authors added a narrative search after finding that the systematic review lacked business-relevant governance aspects and tangible ML processes.The initial works also lacked coverage of issues such as data valuation.Source: Methodology · pp. 2–3
- The systematic search retained 30 articles from 1,627 abstract hits: 13 on governance and 17 on management.The search covered peer-reviewed English-language journal articles from 2011 to May 2021 across university databases.Source: Methodology · p. 3
- The narrative review mapped literature to IT and data governance frameworks, then searched uncovered areas using broad terms and forward and backward citation searches.It included conference papers and, after quality assessment, selected gray literature to capture recent knowledge and cover the field from different angles.Source: Methodology · p. 3
- The analysis assigned reviewed works to generic framework categories, expanded subcategories, and added or dismissed categories as the context required.The resulting framework was discussed by all coauthors.Source: Methodology · p. 4
Framework outline
The framework treats business AI governance as governance across data, ML models, and AI systems, extending IT and data governance concepts. It defines governance in terms of aligning AI technology with organizational strategy and objectives.
Literature review
- The framework synthesizes literature into dimensions and concepts, including antecedents that affect governance and consequences related to corporate goals.The review refines each framework dimension from literature and presents an overview in Figure 2.Source: Framework outline · p. 4, Literature review · p. 4, Figure 2 · p. 4
Framework outline
- The framework defines AI governance as the union of data, model, and system governance, with all three dimensions treated as important.It builds on existing IT and data governance areas while adding model governance for code and procedures used in model training, evaluation, and testing.Source: Framework outline · p. 4, AI governance definition · p. 5
AI governance definition
- AI governance for business is the structure of rules, practices, and processes used to ensure AI technology sustains and extends organizational strategies and objectives.Source: AI governance definition · p. 5
- The paper emphasizes collaboration, formal AI management, strategic focus, decision rights, supporting artifacts, and compliance monitoring as six parts of AI governance.Source: AI governance definition · p. 5
- The paper uses a data-driven lens of AI, distinguishing ML models and model-building processes from AI systems that may include other components.Reinforcement learning is included as a possible source of experience data, but the paper does not discuss it in detail because it is less common in business practice.Source: AI governance definition · p. 5
Governance mechanisms
The paper groups AI governance mechanisms into structural, procedural, and relational types. Structural mechanisms establish accountability and decision authority through roles, governance bodies, and reporting structures.
Governance mechanisms
- AI governance mechanisms comprise structural, procedural, and relational forms that connect business, IT, data, model, ML, and system management.Figure 3 provides an overview of the mechanisms.Source: Governance mechanisms · p. 5, Figure 3 · p. 6
Structural mechanisms
- Structural mechanisms define reporting structures, governance bodies, accountability, roles, responsibilities, and the allocation of decision-making authority.Source: Structural mechanisms · p. 5
- Proposals include centralized hubs, centers of excellence, interdisciplinary councils, executive sponsors, and feature owners, while specific model-related roles remain under-investigated.A dedicated AI budget without performance obligations may benefit adoption in its early phase.Source: Structural mechanisms · pp. 5–6
Procedural mechanisms
Procedural mechanisms govern whether AI systems and ML models operate correctly, securely, and in line with legal and organizational requirements. The framework spans strategy, policies, standards, repeatable processes, contracts, measurement, compliance monitoring, and issue management.
Procedural mechanisms
- Procedural mechanisms address correct and efficient operation, security, and compliance with requirements for explainability, fairness, accountability, security, and safety.Explainability, fairness, and accountability primarily concern models, while safety is more holistic and may involve non-AI safeguards.Source: Procedural mechanisms · p. 6
- The procedural toolkit covers strategy, policies, standards, processes and procedures, contractual agreements, performance measurement, compliance monitoring, and issue management.These mechanisms apply to data, models, and systems individually and holistically.Source: Procedural mechanisms · p. 6
- Standards should normalize data representations, models, system architecture, and processes while supporting interoperability and fitness for purpose.Source: Procedural mechanisms · pp. 6–7
- Documented, standardized, repeatable processes can govern data work, ML development, testing, validation, assurance, documentation, and coding practices.The literature describes multiple proposed processes but still lacks agreement on the best process and has limited research on model and AI procedures.Source: Procedural mechanisms · p. 7
- Contracts can define operational boundaries and reduce liability risk, while addressing possible extraction and misuse of training-data information contained in models.Source: Procedural mechanisms · p. 7
- Compliance monitoring tracks conformance with regulations and organizational agreements, while audits interrogate complex processes for adherence to policies, standards, and regulations.Issue management identifies, manages, and resolves AI-related problems; the reviewed literature offers little detail for model and system issues.Source: Procedural mechanisms · p. 7
Relational mechanisms
Relational mechanisms support stakeholder collaboration, while the framework’s subject scope covers data and AI artifacts, including distinctions in data sensitivity.
Relational mechanisms
- Relational mechanisms comprise communication, training, and coordination of decision-making among stakeholders.Training can build AI capabilities or prepare employees for tasks automated or augmented by AI.Source: Relational mechanisms · pp. 7–8
Subject scope
- Subject scope covers data as content and models and systems as IT artifacts.Source: Subject scope · p. 8
Data subject
- Data type can affect model selection, while personal data is subject to distinct regulations and explanation rights for automated decisions.Explainability processes are less of a concern for models based on non-personal data.Source: Data subject · p. 8
Model subject
The framework treats models and systems as distinct governance subjects, with system concerns including autonomy, human interaction, and operating environment; organizational scope spans one or multiple organizations.
Model subject
- Model subject characteristics include labeled versus unlabeled training, online versus offline learning, and transparent versus black-box models.Online systems continuously learn from observed data and pose additional system-engineering challenges.Source: Model subject · p. 8
System subject
- System autonomy ranges from AI deciding without human review to AI suggesting treatment for a human decision-maker.Autonomy affects legal responsibilities and liability; systems may also augment or automate tasks.Source: System subject · pp. 8–9
- System governance also concerns human-AI interaction, safety, and whether the operating environment is dynamic or static.Dynamic environments often require online learning and highly adaptive models.Source: System subject · p. 9
Organizational scope
- Organizational scope distinguishes governance within one organization from governance between multiple organizations.Source: Organizational scope · p. 9
- Interorganizational development and reuse of open-source models raise governance questions about corporate-policy compliance and liability.The passage notes that such models may be biased, malfunction unpredictably, or be difficult to assess because of their black-box nature.Source: Organizational scope · p. 9
Targets
Governance targets span multiple goal areas; data-focused concerns include value, drift, quality, bias, lineage, and lifecycle management.
Targets
- Governance programs may address one or multiple goal areas, and ML-system quality criteria span data, model, environment, infrastructure, and system.The framework subsumes model criteria that focus on metrics relevant to governance.Source: Targets · pp. 9–10
Data targets
- Data valuation supports investment decisions, but estimating value is difficult when potential uses and benefits are uncertain.Valuation becomes more relevant when acquisition, labeling, or processing incurs costs; active learning can identify samples that improve a given model most.Source: Data targets · p. 10
- Concept drift changes the data distribution over time, potentially making training data unrepresentative of the relationship a model should capture.Real drift changes classifications; virtual drift changes sample distributions between training and operation.Source: Data targets · p. 10
- Data-quality improvement trades potential model-performance gains against cleansing costs, while filtering may replace cleansing when abundant mostly accurate data exists.Source: Data targets · pp. 10–11
- Biased data may produce unfair ML models that violate regulations.Source: Data targets · p. 11
- Data lineage tracks origin, movement, and processing, including links between data, preprocessing, and models trained on it.Governance should propagate relevant data updates to stakeholders; the data lifecycle also includes defining, collecting, creating, using, maintaining, archiving, and deleting data, with retention requirements tied to business, regulatory, and accountability demands.Source: Data targets · p. 11
Model targets
Model targets encompass quality, fairness, explainability, efficiency, and infrastructure, while model-management practices address reproducibility, sharing, and pipeline complexity.
Model targets
- Model governance targets differ from data targets: model security includes operational attacks, while data security concerns handling; model metadata supports accountability.Source: Model targets · p. 11
- Model assessment dimensions include correctness, efficiency, fairness, interpretability, robustness, security, and privacy.Robustness concerns functioning with invalid inputs; security concerns resilience to manipulation or illegal access, and privacy concerns confidentiality.Source: Model targets · p. 11
- Fairness is assessed at the model level because decisions are made by models and data bias’s impact on decision fairness is difficult to anticipate.The passage notes that model-level mechanisms may produce a fair model despite biased data.Source: Model targets · p. 11
- Explainability supports understanding model behavior and can aid user trust, model testing, and validation; efficiency concerns construction and prediction effort.Source: Model targets · pp. 11–12
- Model infrastructure affects training, testing, and deployment efficiency and scalability, including growth in predictions, complexity, and model count.Distributed training on specialized hardware can reduce training time compared with ordinary personal computers.Source: Model targets · p. 12
- Model-management practices support pipeline tracking, reproducibility, auditing, reusability, and model sharing, but face conceptual, data-management, and engineering challenges.Challenges include versioning, customization and reuse, retraining decisions, pipeline abstraction, metadata access, and multilingual codebases.Source: Model targets · p. 12
System targets
AI systems may need controls at the system level as well as model-level governance, particularly where people and AI work together. Governance adoption also depends on internal organizational conditions and external requirements, including customer readiness.
System targets
- Human-AI systems may substitute for, augment, or combine work, so governance can address accountability, ethics, security, privacy, and operator intervention.Source: System targets · p. 12
- AI governance may seek societal benefits while limiting risks such as devaluing human skills and reducing human control.Source: System targets · p. 13
Antecedents
- Internal antecedents include AI capability and development procedures, alignment, strategy, resources, culture, management support, and cross-unit collaboration.Source: Antecedents · p. 13
- External legal and regulatory requirements, industry-specific deployment, and customer readiness—including understanding AI limitations—also shape governance.Customers are among the stakeholders to be governed.Source: Antecedents · p. 13
Consequences
The framework extends IT-governance concepts to AI’s data, models, and systems, and connects governance mechanisms with antecedents and consequences. It also proposes areas for further theory development and practice, while noting unresolved research needs.
Consequences
- AI governance consequences are categorized as performance effects, risk management, orchestration, and effects on people, including employees and customers.Source: Consequences · p. 13
Final framework
- Unlike static IT artifacts, models derive behavior from data; the framework excludes architecture as an overarching mechanism and emphasizes structural, procedural, and relational mechanisms.Source: Final framework · p. 14
- The framework governs data, models, and systems through what, who, and how dimensions, alongside antecedents and consequences.Source: Final framework · pp. 13–14, Discussion and future work · p. 14
Discussion and future work
- The authors call for studying AI-governance cube cells and note that many need more research as AI evolves and practical proposals await validation.Source: Discussion and future work · p. 16
- Proposed structural roles adapt data-governance structures for models, with separate ML development and testing roles and a possible AI risk manager.Source: Discussion and future work · p. 16
- Separate development, testing, and deployment infrastructure can limit sensitive production-data access and keep development outages from affecting deployed services.Source: Discussion and future work · p. 16
- The authors identify stakeholder governance as an area needing depth, particularly for employees and customers exposed to AI risks and involved in data and interdisciplinary work.Source: Discussion and future work · pp. 16–17
- Data-value governance remains difficult because current approaches lack predictive capability, while concept drift and improved sensors can devalue data over time.Source: Discussion and future work · p. 17
Conclusion
The conclusion presents the review as illuminating AI governance’s building blocks, including a conceptual framework and finer-grained business-oriented proposals, and identifies future research opportunities.
Conclusion
- The study reviews AI and ML literature through a conceptual framework to illuminate AI governance’s building blocks.Source: Conclusion · p. 17
- It offers finer-grained proposals motivated by technological developments, including data quality and value, and analogies to related disciplines.Source: Conclusion · p. 17
- Future work could investigate governance roles and data value by selecting cells in the cube’s “What,” “How,” and “Who” dimensions.Source: Conclusion · p. 17
Note
This end matter includes a note about computing AlphaGo’s cost, contributor biographies, and an ORCID identifier.
Note
- The note says AlphaGo’s cost can be computed from Silver et al. (2017) and points to an online calculation.Source: Note · p. 17
Notes on contributors
- Contributor notes profile Johannes Schneider, Rene Abraham, Christian Meske, and Jan vom Brocke through their academic positions and research backgrounds.Source: Notes on contributors · p. 17
ORCID
- Christian Meske’s ORCID is 0000-0001-5637-9433.Source: ORCID · p. 17