Source-linked AI summary
Operationalizing Regulations into Code: A Model to Enhance Governance and Compliance in LLM Selection for Software Engineering
Jonysberg Quintino, Hermano Moura, Filipe Calegário
TL;DR
LLM selection for software engineering lacks operational criteria that integrate regulatory, security, privacy, and governance obligations. The paper develops a DSR-based, three-layer model with MCDA and PAG-LLM, and a 20-scenario pilot provides preliminary evidence that K.O. logic distinguishes risk profiles and can exclude models with unacceptable compliance risks.
Problem
Regulatory obligations are difficult to translate into measurable technical criteria for selecting LLMs in software engineering.
Method
The paper develops a DSR-based three-layer model combining regulatory requirements, governance capabilities, MCDA criteria, PAG-LLM scenarios, and a regulatory feedback loop.
Results
The pilot revealed distinct risk profiles: LLM A mitigated 15 of 20 adversarial scenarios but had unresolved K.O.5, while LLM B failed K.O.3 and was eliminated from high-risk selection.
Takeaways & Limitations
K.O. criteria can prevent selection of technically competitive models whose compliance risks would otherwise be overlooked by overall scoring.
Takeaways & Limitations
The pilot used two models and 20 scenarios, with human consensus scoring and author-defined weights requiring broader validation.
Abstract
from arXiv · showhide
Integrating Large Language Models (LLMs) into the Software Development Life Cycle (SDLC) can improve developer productivity, but it also introduces security, privacy, and compliance risks during model selection. Regulations and frameworks such as the EU AI Act, the NIST AI Risk Management Framework (RMF), the General Data Protection Regulation (GDPR), the Lei Geral de Proteção de Dados (LGPD), and ISO/IEC 42001 establish obligations that are often difficult to translate into operational criteria for technical decision-making. This paper proposes a model to support governance and compliance in LLM selection for software engineering projects. The model is developed through Design Science Research (DSR) and is structured in three layers: (i) regulatory requirements, (ii) organizational governance capabilities, instantiated by a multi-criteria decision matrix with knock-out and weighted scoring criteria, and (iii) productivity and sustainability outcomes, operationalized by the LLM governance assessment protocol (PAG-LLM). A regulatory feedback loop connects operational results back to the normative layer, enabling iterative refinement of the model. A pilot evaluation with 20 adversarial scenarios based on Common Weakness Enumeration (CWE) and the OWASP Top 10 suggests distinct risk profiles between commercial cloud-based LLMs and local open-source LLMs. The results provide preliminary evidence that regulatory disqualification logic, particularly K.O. criteria, can prevent the selection of technically competitive models that nonetheless pose unacceptable compliance risks, demonstrating the feasibility of governance-oriented LLM selection in software engineering projects.
1 Introduction
LLM adoption in software engineering creates security, privacy, and compliance risks, while regulations remain difficult to operationalize for model selection. The paper proposes a governance-oriented decision-support model and pilots it across 20 adversarial scenarios.
- LLM adoption in the SDLC supports repetitive software engineering tasks but introduces risks including insecure code generation and sensitive-information exposure.
- Regulations and standards establish governance, transparency, oversight, and data-protection obligations that remain difficult to translate into integrated selection criteria.
- The proposed DSR-based model combines regulatory dimensions with LGPD requirements in a multi-criteria framework for comparing cloud and local LLMs.
- The prescriptive artifact operationalizes governance and compliance requirements into criteria applied before LLM integration and supports comparison beyond performance benchmarks.
- 20 adversarial CWE/OWASP-mapped scenarios pilot the model with two LLMs having contrasting deployment profiles and risk profiles.
2 Theoretical Background
The theoretical foundation combines legal, organizational, and technical perspectives on AI governance. It uses regulatory requirements for gatekeeping and data protection, ISO/IEC 42001 for organizational maturity, and NIST for technical risk indicators.
- The EU AI Act frames high-risk AI obligations around data governance, technical documentation, transparency, and human oversight.
- GDPR and LGPD share data-minimization, purpose-limitation, and transparency principles that inform the model’s data-management dimension.
- ISO/IEC 42001 evaluates AI life-cycle controls, risk management, and impact assessment as indicators of provider operational maturity.
- The NIST AI RMF and generative-AI profile provide technical indicators for hallucinations, secret leakage, vulnerabilities, and adversarial attacks.
3 Methodology
The study uses Design Science Research to construct an iterative governance model with three connected layers and a regulatory feedback loop. Its artifacts include a mapped DSR cycle and regulatory requirements.
- The model follows a five-phase iterative DSR cycle designed to maintain traceability between activities and produced artifacts.
- Phases 1–4 are complete, while Phase 5 will expand the protocol and validate expert-based weights in subsequent cycles.
- The architecture comprises three sequential layers connected by a regulatory feedback loop that updates requirements using operational results.
- The first layer consolidates obligations derived from five regulatory frameworks.
- The DSR process maps regulatory obligations and documents gap diagnosis as produced artifacts.
1. Awareness
The awareness stage diagnoses the governance gap and begins translating normative obligations into operational criteria. This establishes the basis for the model’s subsequent decision-support artifacts.
- 1. Awareness: It synthesizes normative obligations into operational criteria and defines a three-layer architecture with a regulatory feedback loop.
- 1. Awareness: The resulting architecture connects regulatory requirements, governance capabilities, and iterative feedback.
2. Suggestion
The proposed model combines a conceptual governance structure with a multi-criteria decision matrix for LLM selection.
- The conceptual model uses knock-out criteria and an integrated risk-prioritization structure.
- The MCDA matrix contains five pillars with assigned weights and representative regulatory criteria.
3. Development
The model’s implementation connects MCDA governance assessment with a PAG-LLM pilot comparing cloud and local LLM deployment profiles.
- Layer 2 applies the multi-criteria decision analysis, while Layer 3 uses the PAG-LLM protocol for operational assessment.
- The pilot compares LLM A in the cloud with LLM B locally across 20 scenarios evaluated by two independent raters.
4. Demonstration
The demonstration phase analyzes validity threats and feeds the findings into the model’s feedback loop to guide subsequent research cycles.
- The evaluation includes a validity-threat analysis to qualify the emerging findings.
- Results are integrated into the Feedback Loop, supporting planning for subsequent research cycles.
5. Evaluation & Communication
The evaluation and communication layer translates regulatory obligations into governance criteria, applies K.O. and weighted scoring logic, and produces operational evidence and iterative updates.
- Regulatory foundation: The model synthesizes obligations from the EU AI Act, NIST AI RMF, GDPR, LGPD, and ISO/IEC 42001 into its regulatory foundation.Together, these frameworks address legal, technical, data-protection, and organizational governance concerns.
- Governance capabilities: Organizational capabilities provide policies, assigned responsibilities, and AI governance processes for applying the selection matrix.This foundation situates the matrix within an organizational governance context.
- Knock-out criteria: K.O. criteria are non-negotiable regulatory constraints that exclude failing LLMs regardless of technical performance.The gatekeeping logic is intended to anticipate regulatory sanctions and critical ethical failures before pipeline integration.
- Risk scoring: Weighted soft constraints are aggregated with a Weighted Sum Model to produce the Weighted Risk Index.The IRP provides a risk score after the hard-constraint gate has been applied.
- Risk scoring: P1 and P2 receive 25% weights each, while P4 and P5 receive 15% each as a preliminary author-assigned baseline.Domain experts are expected to validate these weights in future work.
- Decision outputs: Models passing all K.O. criteria can be compared across trade-offs including data sovereignty, adversarial robustness, and operational maturity.
- Decision outputs: Software engineering practices such as code-quality metrics, static analysis, and SDLC checkpoints make governance criteria verifiable.
- Operational assessment: PAG-LLM uses CWE- and OWASP-mapped adversarial scenarios to generate technical evidence for model selection and recommend complementary controls.The third layer also includes classic productivity benchmarks and risk-profile recommendations.
5 Model Pilot Evaluation
The pilot evaluates two LLMs using 20 CWE/OWASP-mapped adversarial scenarios and shows that deployment profiles produce distinct governance risks. K.O. criteria can exclude models despite technical competitiveness.
- Evaluation Design: 20 adversarial scenarios mapped to CWE and OWASP Top 10 were evaluated through code-generation prompts scored by two independent evaluators.Responses used a 1–5 security scale, with discrepancies exceeding one point resolved by consensus.
- Pilot Results: LLM A natively mitigated 15 of 20 adversarial scenarios and outperformed LLM B on Regulatory Compliance, Transparency, and Operational Maturity.LLM A was a commercial cloud solution with a managed API, public documentation, and auditable terms of use.
- Pilot Results: LLM A nevertheless presented an unresolved K.O.5 concerning Brazilian data suitability under LGPD, creating immediate regulatory risk for Brazilian organizations.The issue involved processing in a European jurisdiction without explicit suitability for Brazilian citizens’ data.
- Pilot Results: LLM B offered full sovereignty and no retraining but failed K.O.3 after system instruction injection manipulated its output in scenario S07.That failure automatically eliminates LLM B from selection in high-risk contexts regardless of its calculated IRP.
- Interpretation: K.O. gatekeeping can prevent selection of LLM B despite its IRP of 7.13, while feeding the observed risk back into Layer 1 for iterative refinement.The pilot therefore exposes trade-offs that traditional performance benchmarks may not capture.
- Validity and Scope: The pilot’s validity is constrained by human consensus scoring without retained individual ratings, two evaluated models, 20 scenarios, and preliminarily defined pillar weights.Future work proposes inter-rater reliability analysis, automated static analysis, broader evaluation, and independent weight validation.
6 Related Works
The related literature addresses LLM selection and security in a fragmented manner. This proposal distinguishes itself by integrating technical, legal, and organizational dimensions into a prescriptive operational cycle.
- Related Works: Existing literature addresses the problem in a fragmented manner across multiple dimensions and related software engineering traditions.The paper positions its model against five primary related works across eight discriminating dimensions.
- Distinctive Contribution: The proposed model is prescriptive and architectural, structuring a complete cycle from regulatory norm to operational result for strategic LLM selection.It combines technical, legal, and organizational dimensions with continuous feedback in a single artifact.
7 Conclusion and Future Work
The paper presents a DSR-developed, three-layer model that translates regulatory and organizational requirements into operational LLM-selection criteria. Its pilot provides preliminary feasibility evidence, while future work targets broader validation and applicability.
- Conclusion: The model uses three interconnected layers to translate regulatory and organizational requirements into operational criteria and evaluation procedures.The artifact is intended to support governance and compliance in selecting LLMs for software engineering.
- Conclusion: The MCDA matrix and PAG-LLM pilot provided preliminary evidence that the model distinguishes risk profiles not captured by traditional performance-oriented selection criteria.The evaluation highlighted trade-offs involving data sovereignty, transparency, robustness, and operational maturity.
- Practical Implication: The model operationalizes regulatory obligations as technical criteria applied during LLM selection rather than as a retrospective compliance exercise.Its intended scope is governance-oriented decision support for AI-assisted software engineering.
- Future Work: Future work will expand PAG-LLM to additional models, validate pillar weights with security and legal experts, and conduct a Brazilian technology-organization case study.These activities are intended to refine the artifact and strengthen applicability across organizational and regulatory contexts.