Risk management in the Artificial Intelligence Act

I. Introduction

• The AI Act is the first comprehensive attempt to regulate AI in a major jurisdiction, and may influence regulation beyond the EU.The article notes its potential as a benchmark for the US and UK and possible de facto effects elsewhere through the “Brussels Effect.”Source: §I · pp. 1–2
• Identifying, assessing, and responding to AI risks matters especially in high-stakes settings, while attention to AI risk management has grown among practitioners.The article cites critical infrastructure as an example and notes that standard-setting bodies were developing voluntary frameworks as of 2022.Source: §I · p. 3
• Despite existing enterprise frameworks and practitioner reports, academic literature on AI risk management is limited, particularly on Article 9 itself.The article therefore conducts a doctrinal analysis using literal, systematic, teleological, and historical interpretation.Source: §I · p. 4
• The analysis relies on the original Commission proposal and subsequent institutional drafts, so future changes could make parts of it obsolete.The article argues that its purpose analysis is particularly robust to change and recommends further amendments.Source: §I · pp. 1, 5

II. Regulatory concept

• Article 9 is a backup measure: providers must identify residual risks from high-risk systems and take additional measures to reduce them to an acceptable level.The AI Act assumes that compliance with other requirements will not eliminate all risks.Source: §II · p. 6
• Article 9(1) requires providers of high-risk AI systems to implement a risk-management system, while paragraphs 2–7 specify its process, measures, and testing procedures.Paragraphs 8 and 9 contain special rules concerning children and credit institutions.Source: §II · p. 6
• Harmonised standards can support compliance through a presumption of conformity, but AI risk-management standards were not yet available.The European standardisation bodies had been tasked with developing them, a process that might take several years.Source: §II · p. 7
• Until European standards are available, providers can use international frameworks such as NIST’s or ISO/IEC’s as rough guidance, without gaining a presumption of conformity.The article expects some compatibility with future European standards but notes there is no guarantee.Source: §II · p. 7

III. Purpose

• Determining Article 9’s purpose matters because it influences the permissible range of interpretations of the provision.Source: §III · p. 7
• The article identifies improving the internal market and protecting public interests as purposes of Article 9.The cited public interests include health, safety, and fundamental rights.Source: §III · p. 8
• Whether Article 9 also protects individuals remains open, with the answer potentially affecting some tort claims.The article contrasts the possibility of protection for users and affected persons with a view of risk management as primarily an organisational requirement.Source: §III · p. 8

IV. Scope of application

• Article 9 applies to high-risk AI systems; unacceptable-risk systems are prohibited, while including low- or minimal-risk systems is not legally required.The article argues that voluntary inclusion can support holistic risk management and may add only marginal operating costs because many implementation costs are fixed.Source: §IV · pp. 9–10
• The article argues that organisations should initially include risks relevant to them, including litigation and reputation risks, even when systems fall below the high-risk threshold.After identifying and assessing risks, organisations can still choose not to respond.Source: §IV · p. 10
• Article 9 applies to providers of high-risk AI systems, subject to exclusions for certain public authorities and international organisations.The article relies on Article 16(a) and Article 2(4) to determine this personal scope.Source: §IV · p. 10
• The AI Act’s regional scope includes providers placing systems on the EU market, putting them into service there, or using their outputs in the EU, regardless of where providers are established.Source: §IV · p. 10
• Whether Article 9 should extend to general-purpose AI systems is disputed and beyond the article’s scope.Proposals ranged from extending the requirements to those systems to excluding them from the Act.Source: §IV · p. 10
• Providers must have a risk-management system in place 24 months after the AI Act enters into force; the French and Czech Presidencies proposed 36 months.The article notes that the Act’s entry into force date was then uncertain.Source: §IV · p. 11

• The section offers a comprehensive interpretation of the specific risk management requirements set out in Article 9.Source: §V · p. 11

• Article 9(1) makes establishing, implementing, documenting, and maintaining a risk management system the central requirement.Source: §1 · p. 11
• The system comprises the risk management process in paragraphs 2–4 and the testing procedures in paragraphs 5–7.Source: §1 · p. 11
• Policies, procedures, and instructions must be created and approved, put into practice, documented for authorities, and regularly reviewed and updated as needed.Source: §1 · p. 12
• The required system covers risks from high-risk AI systems, not low- or minimal-risk systems, though providers may address the latter voluntarily.Source: §1 · p. 12

• Providers identify and analyse risks, estimate and evaluate them, then adopt risk management measures in an iterative process.Source: §2 · p. 12, c. Adoption of risk management… · pp. 16–17

a. Identification and analysis of known and foreseeable risks, Article 9(2), sentence 2, point (a)

• Risk identification uses available information to find hazards; known and foreseeable risks include those discoverable with reasonable effort, with more effort warranted for higher-impact risks.Source: §2 · pp. 13–15

b. Estimation and evaluation of risks that may emerge from intended uses or foreseeable misuses, or risks identified during post-market monitoring

• Risk estimation assesses the probability and severity of harm for intended uses and reasonably foreseeable misuses, while post-market monitoring also brings identified risks into scope.Source: §2 · pp. 15–16

c. Adoption of risk management measures, Article 9(2), sentence 2, point (d)

• Providers reassess after measures and continue reducing risks until residual risks are acceptable; if they cannot be reduced to an acceptable level, development and deployment must stop.Source: c. Adoption of risk management… · pp. 16–17
• The process runs throughout the system lifecycle; the article gives no fixed cadence, but two iterations—during development and before deployment—are described as the bare minimum.Source: c. Adoption of risk management… · p. 17

• Providers must consider interactions among Chapter 2 requirements and the generally acknowledged state of the art when adopting risk measures.Source: §3 · pp. 18–19
• Risk measures must reduce residual risks to an acceptable level, but normative thresholds remain an open problem and harmonised standards or common specifications are not yet available.Source: §3 · p. 19
• The measures follow three steps: reduce risks through design, mitigate remaining risks with controls, and provide users with information or training where appropriate.Source: §3 · p. 20
• The article specifies risk-reduction measures for the design phase but not for the use phase.Source: §3 · p. 20
• When selecting measures, providers must consider users’ expected knowledge and training and the environment in which the system will be used.Source: §3 · p. 21

4. Testing procedures, Article 9(5)-(7)

• Testing serves three purposes: identifying suitable risk measures, checking consistent performance for the intended purpose, and assessing compliance with Chapter 2 requirements.It can help detect likely poor performance when the use environment differs from the training environment.Source: §4 · pp. 21–22
• Testing procedures must be suitable for their intended purpose and must not go beyond what is necessary to achieve it, restating proportionality.The article notes that proposed amendments would replace this provision with a reference to rules on real-world testing.Source: §4 · p. 22
• Metrics and probabilistic thresholds make sense for technical requirements, but not for governance requirements such as documentation or record-keeping.The author identifies this as a mismatch because Article 9(5) refers to both types of Chapter 2 requirements.Source: §4 · p. 22
• Testing is required as appropriate throughout development and, in any event, before placing the system on the market or putting it into service.Unlike the risk-management process, testing is not required to run through the system's entire lifecycle.Source: §4 · p. 23
• Providers must use preliminary metrics and probabilistic thresholds suited to the intended purpose, but their appropriateness is context-specific and best practices are lacking.Testing may be outsourced, provided the provider remains responsible for meeting the requirements.Source: §4 · p. 23

5. Special rules for children and credit institutions, Article 9(8), (9)

• A provider remains exposed to administrative and civil enforcement if an outsourcing company fails to perform required testing, and may seek recourse from that company.Outsourcing the testing therefore does not shift the provider's regulatory responsibility.Source: §5 · p. 23
• Providers must give specific consideration to whether a high-risk system is likely to be accessed by or affect children, who the article identifies as particularly vulnerable.The author concludes that providers must take special measures to protect children.Source: §5 · pp. 23–24
• For credit institutions, Article 9's AI-specific requirements form part of their existing risk-management systems rather than replacing them.The Czech Presidency proposed extending this rule to any provider already required to implement a risk-management system.Source: §5 · p. 24

VI. Enforcement

• Up to €20 million or 4% of worldwide annual turnover, whichever is higher, may be imposed as an administrative fine; Council presidencies proposed a 2% turnover cap for SMEs.The Act provides only high-level penalty guidelines, leaving details to member states.Source: §VI · p. 24
• Authorities will usually ask providers to demonstrate Article 9 conformity before penalties, and incorrect, incomplete, or misleading information can trigger further fines.Source: §VI · p. 25
• Civil claims may involve contractual, tort, or internal liability, depending respectively on contractual duties, individual-protection rules, or managers’ duties of care.Source: §VI · p. 25
• Article 9 is not directly enforceable through criminal law, although violating it may form part of an offence in some member states.Source: §VI · p. 25

VII. Conclusion

• Article 9 requires providers of high-risk AI systems to manage risks that remain after other requirements are met, through a system whose details are set out in paragraphs 2–7.The conclusion also notes special rules for children and credit institutions.Source: §VII · pp. 25–26
• The author proposes adding an organizational dimension, evaluating system effectiveness, and clarifying that risk management covers individual, collective, and societal risks.Source: §VII · p. 26
• The article’s interpretation is intended to help providers comply, inform possible amendments and harmonized standards, and potentially inform policy makers beyond the EU.Source: §VII · pp. 26–27
• The conclusion identifies penalties, administrative fines, and civil liability as enforcement routes for Article 9 requirements.Source: §VII · p. 26