Source-linked AI summary
Managing Inherent Risk: On the Conceptualization of Risk in Defense Systems
NIklas Braun, Leon J. Brettin, Marvin Loba, Markus Maurer
TL;DR
Defense systems can create inherent risks in civilian environments, challenging public acceptance because they must cause harm to fulfill defensive purposes. The paper compares civilian and military risk concepts, formulates three defense-system risk categories, and argues that proactive risk management and communication can support societal acceptance while recognizing unresolved taxonomy, legal, and communication questions.
Problem
Defense systems require a risk concept that captures their distinct combination of unintended harm, intended harm, and external threats while addressing the gap between technical risk criteria and societal acceptance.
Method
The paper analyzes existing automotive and defense standards and literature, then conceptualizes a three-category risk balance for defense systems.
Results
The paper finds that civilian safety approaches are partially applicable to defense and formulates a risk balance that includes external threats to account for physical security.
Takeaways & Limitations
Proactive risk management and differentiated communication may contribute to societal acceptance of defense systems novel to their operating environments.
Takeaways & Limitations
Future work is needed on a unified physical-security taxonomy, context-dependent risk perception and acceptance criteria, legal aspects, and integration with systems engineering.
Abstract
from arXiv · showhide
Certain defense systems are, by nature, deployed in a civilian environment in order to serve a defensive function for that environment. However, the risk involved with their deployment and operation poses a challenge for the public acceptance of these systems. Compared to safety engineering for civilian systems, the risk constellation is quite different. While reducing risk of safety-critical systems is generally desirable, defense systems are required to cause harm in order to be useful. In both cases, not all risk can be eliminated. The complex risk constellation of defense systems has implications for the systems' designs. We compare the concepts of risk in existing standards from both the civilian and the military domains. An extended constellation of risks needs to be considered, including risk caused by external threats to physical security. By including this risk constellation in public communication about defense systems, we aim to stimulate a productive debate.
I. INTRODUCTION
Defense systems can cause unintended harm even when operating as intended, making risk reduction and societal acceptance central challenges in civilian environments. The paper compares automotive and defense safety engineering, explicitly considers defense-specific risks, and develops three risk categories.
- I. INTRODUCTION: Defense systems may cause unintended harm during intended operation, such as property or personal damage from missile debris.A GBAD system illustrates how kinetic force and explosives create inherent danger without malfunction.
- I. INTRODUCTION: Risks caused by defense systems in civilian environments need reduction to a tolerable level, while residual risks may be accepted in purely military contexts.Public expectations about safety and security are relevant to societal acceptance.
- I. INTRODUCTION: Novel defense systems require safety-by-design that explicitly considers risks and stakeholder expectations before deployment decisions are publicly defended.The paper connects proactive expectation management with public discourse and acceptance.
- I. INTRODUCTION: The paper contributes an automotive perspective to defense safety engineering while explicitly considering defense-specific kinds of risk.It situates the contribution alongside ongoing defense safety research and standardization, including IEC 63187.
- I. INTRODUCTION: The paper first reviews publicly available risk concepts from automotive and defense domains, then conceptualizes three defense-system risk categories.The analysis proceeds through background concepts, risk categories, and concluding future research directions.
II. BACKGROUND
The background frames safety and risk as linked engineering concepts and compares their treatment across standards and literature. ISO Guide 51 provides a cross-domain reference for these concepts.
- II. BACKGROUND: Engineering standards and codes address safety and risk across defense and automotive domains, enabling comparison of their core concepts.The paper analyzes existing literature and standards as its background approach.
- II. BACKGROUND: Safety is tied to risk and harm in engineering, with ISO Guide 51 serving as an established cross-domain reference.The guide informs both cross-domain and domain-specific safety standards.
- II. BACKGROUND: ISO Guide 51 defines safety as freedom from risk that is not tolerable, with tolerability accepted in a given social context.IEC 61508, ISO 26262, and ISO 21448 use closely related formulations involving unacceptable or unreasonable risk.
1) Safety:
Safety and risk are technically defined through harm, probability, severity, and tolerability, but operational risk criteria may not fully capture societal judgments. The section highlights a translation gap between moral concepts and usable acceptance criteria.
- 1) Safety:: Safety is treated as an open signifier whose meaning can vary by stakeholder and context.This flexibility integrates diverse viewpoints but complicates negotiation of specific safety issues.
- 1) Safety:: Risk combines the probability and severity of harm, where harm includes injury or damage to people, property, or the environment.The scope of harm varies between standards, although ISO Guide 51 gives this broad formulation.
- 1) Safety:: Civilian safety processes require risk acceptance criteria, but translating societal moral concepts into operable criteria can leave them unclear, incomplete, or unsuitable.The resulting criteria may not adequately evaluate the risk estimated during analysis.
4) Acceptance:
Technical risk thresholds do not by themselves establish societal acceptance, and aggregated risk can obscure unacceptable behavior under specific conditions. Defense-system safety standards also reveal a mismatch between absolute safety definitions and tolerable-risk management.
- 4) Acceptance:: Risk below an acceptance criterion does not imply societal acceptance, which depends on system properties, perceptions, attitudes, opinions, and stakeholder interactions.Acceptance therefore requires contextualization beyond technical risk measurement.
- 5) Aggregated and Event-Level Risk:: Aggregated risk spans extended temporal and spatial contexts, unlike event-level risk associated with a specific harm.It is relevant when open contexts make it impossible to foresee and treat every event separately.
- 5) Aggregated and Event-Level Risk:: Managing risk only in aggregate can create risk subsidy, where positive net risk balance coexists with unacceptable behavior in specific conditions.The paper identifies this as a limitation of aggregated risk management.
- B. Risk Management in Defense Engineering: Defense systems are treated as designed to counter external threats and contribute to physical security, although their use may not be fully foreseeable.The paper acknowledges an open debate over whether defensive and offensive weapon systems can be defined consistently.
- B. Risk Management in Defense Engineering: Defense standards define safety absolutely while relying on tolerable residual risk for risk management, creating an internal inconsistency.The standards also distinguish safety-relevant mishaps from intended harm.
- B. Risk Management in Defense Engineering: Defense-system safety standards identify safety with unintended mishaps, which matters because affected stakeholders may differ from decisionmakers.IEC 63187 is identified as an emerging systems-engineering approach for defense-system safety.
C. Risk Management in Automotive Engineering
Automotive risk management treats innovative systems as introducing inherent risk into open, complex operating contexts where requirements and assurance cannot be complete. Risk can be treated but not eliminated, motivating explicit context-based risk representation and scenario communication.
- Open, uncertain road traffic combined with automotive complexity produces necessarily incomplete requirements, verification, and validation.
- Automated Driving Systems introduce inherent risk into road traffic despite their potential contribution to road safety.
- Inherent risk can be treated but never eliminated, challenging unrealistic expectations such as vision zero.
- Explicitly representing risk through operational context supports adequate requirements at an early design stage.
- Scenarios communicate risk between stakeholders across multiple abstraction levels.
D. Security, Cybersecurity and Physical Security
Security terminology is contested across cybersecurity and political-science discussions. The paper adopts physical security as protection against threats to a state and its constituents, while acknowledging that this terminological work remains incomplete.
- Cybersecurity lacks a consensual definition, with one proposal framing it as resources, processes, and structures protecting cyberspace and cyberspace-enabled systems.
- Security is an open concept in political science, and national security is criticized for its overuse and diverse meanings.
- The paper defines physical security as protection against threats to a state and its constituents, including territory, people, and goods.
- The authors acknowledge that their terminological treatment of physical security is not complete.
- Across civilian and military domains, risk definitions and the general notion of safety are compatible despite inconsistent use of safety terminology.
III. RISK CONCEPTUALIZATION
The paper models risk as caused by an entity within a context and applies this distinction to defense systems, including external threats. This supports structured risk categories and communication of risks that are not fully controllable.
- Development frameworks that incorporate stakeholder values and communicability may support defense-system acceptance amid design conflicts.
- The proposed risk-management structure aims to resolve design conflicts ethically and reduce ethical and legal uncertainty for deployment decisions.
- Risk is conceptualized as caused by an entity in a particular context.
- Automated Driving Systems illustrate how newly introduced systems become part of a socio-technical context that already exhibits risks.
- Malicious attacks can create hazardous system behavior and risks that are not attributable to the system but still require developer protection measures.
- For defense systems, developers are responsible for achieving tolerable levels of unintended-harm risks, while responsibility for externally threatened secondary risks is less clear.
- The cause–context distinction depends on system boundaries and stakeholder perspectives, while legal accountability is linked but not identical to risk-management responsibility.
- Despite its limitations, the model distinguishes three defense-system risk categories and clarifies that not all risk is entirely controllable.
B. Risk Categories
The paper distinguishes three defense-system risk categories: unintended harm, intended defensive harm, and harm from external threats. Unlike civilian safety engineering’s lower-is-better orientation, defense risk management must balance risks against defensive purpose and uncertain threats.
- The first category is system-caused risk linked to unintended harm, including risks to military personnel and workers.
- The second category is system-caused risk linked to intended harm against an object posing a threat.
- The third category is risk caused by external threats, meaning harm from an object posing a threat.
- Defense risk reduction involves design trade-offs, while civilian safety engineering generally assumes that lower risk is better.
- External-threat risks are necessary for an operable risk balance because reducing them may justify or mandate introducing other defense-system risks.
- External-threat risks are difficult to calculate because threats are partially observable and depend on predictions about future external behavior.
C. Risk Acceptance Criteria
Risk acceptance criteria differ between civilian and military approaches, and translating societal expectations into criteria remains unresolved. Engineering definitions based on harm probability and severity omit contextual and causal factors that shape risk perception.
- Civilian risk acceptance criteria remain difficult because attitudes are inconsistent, perception is context-dependent, and acceptability changes before versus after incidents.
- Military standards require the procuring authority to accept residual risk, whereas manufacturers must design systems that the authority will ultimately accept.
- Quantitative criteria connect empirical data from similar systems to requirements for novel systems and can be allocated across architecture and operating conditions.
- Engineering risk criteria use harm probability and severity but omit the context and causal chains that influence individual and societal risk perceptions.
D. Balancing Risks for Safety and Physical Security
Defense-system design and operation require balancing risks to safety and physical security. These objectives can conflict with performance capability when design resources are limited.
- Defense-system decisions should balance three risk categories addressing safety, defense against external threats, and risks reduced through deterrence.
- Safety is an emerging system property expected by stakeholders, while physical security depends on defending against threats and deterring attacks.
- Safety assurance and performance capability can conflict when design resources such as moving-object mass or kinetic-unit volume are limited.
E. Risk Communication
Risk communication may support societal acceptance and physical security, but disclosure must remain partial because full transparency can increase threats. The paper identifies taxonomy, perception, legal, and integration work for the future.
- Partially transparent risk communication may support physical security by proactively managing public expectations where deployment depends on political decisions backed by society.
- Full transparency can deteriorate the overall risk balance because publicly disclosing deliberate information may increase the level of threat.
- The proposed risk categories are intended to stimulate differentiated debate, while adequate concepts and language for stakeholder communication remain future work.
- Civilian safety approaches are only partially applicable to defense, and the proposed three-category risk balance is intended to support proactive management and societal acceptance.
- Future work includes a unified physical-security taxonomy, risk-perception studies across wartime and peacetime, general defense risk-acceptance criteria, legal research, and integration of the multicategorical balance.