Source-linked AI summary
On Being Prepared: Automated Vehicle Incident Management Exercise Practices
Laura Fraade-Blanar, Jackson Graves, Aaron Clark-Ginsberg, Sam Cooper, Jeremy Kaufman, Jackie Lindsey, Piet van Os
TL;DR
AV organizations need practical exercise guidance because existing IM materials provide limited AV-specific direction for managing incidents across a distributed, socio-technical fleet environment. The paper adapts established IM practices into company-neutral frameworks covering exercise types and stages, concluding that varied, well-designed exercises can strengthen readiness and coordination while remaining subject to realism and transfer limitations.
Problem
AV operators lack sufficiently specific guidance connecting incident-management exercise practices with the unique socio-technical context of automated vehicles.
Method
The paper tailors established IM research, policy, exercise types, and exercise stages into company-neutral frameworks for AV incident-management practitioners.
Results
The frameworks support common understanding and improve intercompany and inter-organization coordination and collaboration for AV incident management.
Takeaways & Limitations
AV practitioners should build exercise programs that use quality, diversity, and volume to practice readiness across hazards and strengthen operational response capability.
Takeaways & Limitations
Exercises cannot fully reproduce real-world trade-offs and their results cannot be broadly transferred to similar incidents without qualification.
Abstract
from arXiv · showhide
Incident management (IM) has evolved over recent decades to cover an ever-expanding array of hazards and systems. Barring real-life experience, exercises are a key tool in developing an effective IM program. As part of enterprise resilience and operational readiness, IM practitioners design exercises to understand and build capabilities for efficiently and effectively responding to incidents. Combining research and policy on an emerging transportation technology, automated vehicles (AVs), with established practices for IM, we describe what makes exercises effective and how they can be used to identify gaps and develop capacity. A range of exercise types exist, from workshops, to tabletops, to drills, and to full-scale exercises. Every stage of an exercise - preparing, setting up, facilitating, closing, and assessing - is interconnected and should further the exercise's objectives. AV IM practitioners can maximize the utility of a mature exercise program through exercise quality, diversity, and volume. The need to respond to AV incidents, from collisions to natural hazards, is inevitable. This paper tailors existing practices to a series of frameworks, predicated on the unique socio-technical nature of AVs to provide first-of-its-kind guidance for AV IM exercises. Additionally, these frameworks are company-neutral to enable intercompany and inter-organization collaboration.
1. Introduction
AV incident management must address transportation-specific and broader hazards while prioritizing safety, compliance, public trust, and continuity. The paper responds to limited AV exercise guidance by adapting established IM practices into a company-neutral framework for collaboration.
- Exercises develop, assess, and validate plans and capabilities by rehearsing responses, pressure-testing procedures, identifying weak spots, and building response capacity.
- AV incident management coordinates responses to collisions, cybersecurity attacks, natural hazards, fires, and other events while prioritizing people, compliance, public trust, and business continuity.
- The paper primarily addresses Level 4 and 5 AVs, with concepts focused mainly on fleet-based operations.
- Existing IM guidance offers useful foundations but was largely written for government audiences or remains too broad for AV operators.
- AV-focused technical literature discusses disaster-response applications but does not explain the institutional systems and processes that enable them.
- The paper provides tailored, company-neutral exercise guidance to address limited AV exercise literature and support intercompany and inter-organization collaboration.
2. Incident Management
Incident management combines multidisciplinary coordination across hazards and response phases with exercises that turn plans into practiced capabilities. Effective exercises require realistic objectives, structured planning, active participation, evaluation, and continual improvement while preserving interoperability.
- Incident management addresses emergencies across hazards and the mitigation, preparedness, response, and recovery phases.
- Its functional domains include situational awareness, action planning, resource management, coordination, collaboration, feedback, and continuous quality improvement.
- Vague scenarios limit evaluators’ ability to judge the effectiveness and completeness of response procedures.
- Exercises can pressure-test plans, identify weak spots, and build capabilities for responding to varied hazards.
- Effective exercises use realistic context, clear objectives, logistical planning, defined roles, active teamwork, and post-exercise evaluation.
- Exercise standardization supports interoperability and a common readiness baseline, while implementation remains contingent on the incident, organizations, and objectives.
3. IM and exercises in the context of AV fleet operations
AV fleets require all-hazards IM because distributed systems face transportation-specific and broader hazards across regions and operating contexts. Exercises build readiness by integrating prevention, preparation, response, and recovery, coordinating stakeholders, and matching exercise types to objectives.
- AV fleets face transportation hazards such as collisions and intentional interference, alongside earthquakes, floods, wildfires, and other geographically distributed risks.
- AV IM programs integrate prevention, preparation, response, and recovery activities across internal and external teams.
- Exercises provide controlled settings to test extreme scenarios, refine cross-functional workflows, and identify operational defects and training gaps before real incidents.
- Exercise programs should mature with the organization’s understanding, operational design domain, fleet size, and geographic footprint.
- Stakeholder exercises can validate escalation protocols and clarify information sharing and AV participation in government emergency operations.
- An all-hazards approach develops broad capabilities for novel incidents rather than dedicated processes for only specific threats.
- Summative exercises assess performance across knowledge, decision-making, communication, resources, coordination, and supporting infrastructure; functional exercises span multiple teams and processes.
- As exercises move from formative to confirmatory, complexity and realism increase, while all-hazard transferability and generalizability may decrease.
4. Stages in an exercise
The exercise process has five interconnected stages, from preparation through standing up, activity, closure, and evaluation. Each stage relies on and strengthens incident-management domains while advancing the exercise objectives.
- The framework adapts standardized exercise guidance into five stages: preparing, standing up, conducting activities, closing, and evaluating.
- Each stage uses and strengthens the functional domains needed for incident management in different ways.
- Effective exercises tailor objectives to organizational response needs and measure capability improvements within the exercise type’s constraints.
- Using multiple exercise types can provide a more comprehensive signal of organizational readiness.
4.1 STAGE 0: Preparation
Preparation begins by identifying a need and defining objectives, then shapes the exercise’s level, scenario, participants, roles, resources, and evaluation approach. AV exercises must adapt established guidance to differences between government incident management and non-government AV operations.
- Preparation starts with a needs assessment that informs the exercise’s potential type, size, and complexity.
- Exercises may address strategic decision-making, operational task execution, tactical resource management, or multiple levels simultaneously.
- Objectives should be clear, achievable, narrowly defined, and action-driven, guiding the choice of exercise type, event, and organizational factors.
- Roles should be defined across planning, execution, and evaluation, including planning-team and controller responsibilities.
- Scenarios should fit the objective and be plausible, while accounting for hazard, warning time, duration, geographic breadth, complexity, and responsibility.
- AV practitioners must scrutinize the scope of responsibility because government guidance may not match AV companies’ legal powers, jurisdictions, protocols, and resources.
- Exercises should provide a safe space for participants to make mistakes, ask questions, and express uncertainty without negative consequences.
4.2 STAGE 1: Standing up the team
Standing up the team begins when participants are notified of an event and instructed how and where to assemble. This stage can test internal notification systems and participant response behavior.
- Participants are notified of an event and given instructions for how and where to assemble.
- The stage may test company notification software and protocols as well as participants’ response behavior.
4.3 STAGE 2: Assessing and responding
During the response stage, participants assess the situation and act according to the scenario. Activities may involve the incident-management domains shown in the exercise framework.
- Participants assess the situation and respond according to the scenario, with the stage varying widely by exercise design.
- Activities may cover specific incident-management domains, including but not limited to those represented in Figure 3.
4.4 STAGE 3: Closing the exercise
Closing an exercise requires predefined criteria for when it ends and who has authority to end it. These parameters should be established before the exercise rather than improvised during it.
- Exercise closure may occur after all possible actions, when information gaps require longer work, when allotted time ends, or when objectives are achieved.
- The controller, participants, or another entity may hold authority to decide that the exercise is over.
- The basis and authority for ending an exercise should be identified in Stage 0.
4.5 STAGE 4: Evaluation and Continuous Improvement
Evaluation examines both participant performance and exercise quality, using objective-specific evidence and multiple feedback sources to support improvement. Findings are synthesized into ratings, after-action discussions, and corrective priorities, while preserving realism and acknowledging evaluation trade-offs.
- Evaluation purpose: Evaluation analyzes participant actions and exercise quality to determine whether activities produced the right outcomes at the right time, speed, and order.
- Evaluation process: Hotwashes and debriefs provide immediate feedback, but insufficient time for them is a common exercise-planning mistake.
- Objective-based evaluation: Evaluation focuses on each objective’s capabilities and targets, measuring whether required tasks were completed within the desired timeframe.
- Discovery: Evaluators should remain open to previously unidentified weaknesses, assumptions, dependencies, and uncertainties beyond predefined performance measures.
- Evidence sources: Evaluation may combine participant feedback, interviews, debriefs, hotwashes, documentation reviews, surveys, evaluator observations, and external or cross-exercise comparisons.
- Evaluation trade-offs: Pausing an exercise can support reflection but may diminish realism and prevent mistakes from playing out.
- Continuous improvement: After-action meetings and reports compare intended and actual performance, identify improvements, prioritize actions, and consider consequences of inaction.
- Example scenario: The evaluator assigned Performed with Some Challenges (S) because teams achieved the objective while showing an opportunity to improve effectiveness or efficiency.
5. Exercise opportunities and limitations
AV incident-management exercises can strengthen response capabilities, but their artificiality and limited transferability constrain what they demonstrate about real incidents. Practitioners can reduce these limitations through higher-quality, more diverse, and more numerous exercises, alongside broader safety investments.
- Limitations: Exercises cannot fully transfer lessons from one scenario to similar real-world events, even under an all-hazards approach.
- Structural artificiality: Structural artificiality limits realism, difficult trade-offs, decision making, and improvisation in AV incident-response exercises.
- Psychological factors: Artificial exercise stress may not mirror real-world response intensity, masking performance under pressure.
- Mitigation: Exercise quality can be increased through the recommendations developed for exercise design and implementation.
- Mitigation: Using different exercise types over time can provide a more balanced picture of organizational readiness.
- Broader safety system: Exercise systems are only one part of AV safety, alongside safety culture, safety-management frameworks, and learning from real incidents.
6. Discussion and Conclusions
The paper establishes company-neutral frameworks for AV incident-management exercises by adapting general IM practices to AVs’ unique socio-technical environment. It emphasizes institutional commitment, exercise diversity and analysis, while identifying scope and geographic boundaries for future refinement.
- Exercise role: AV companies should treat exercises as a core IM capability for preparing to address diverse hazards efficiently and effectively.Exercises support readiness for events whose occurrence is essential and inevitable in transportation.
- Program development: A mature exercise program requires institutional commitment, progression with company maturity, varied exercise types, and analysis of strengths and gaps.The paper also calls for internal exercise-development abilities and sufficient time and resources for exercise execution and closure.
- Future development: The paper presents its guidance as a baseline that should be refined, validated, and improved as AV incident management matures.It also identifies opportunities for standards bodies to develop AV-specific practices for exercise design and after-action reviews.
- Scope boundaries: The study focuses on fleet-based L4 and L5 AVs, while L3 vehicles, private ownership, and international settings may require different procedures or considerations.Much of the underlying literature is U.S.-focused, motivating additional research on international and multi-country practice.
- Framework contribution: The frameworks align general IM practices with AV-specific elements to support intercompany and inter-organization coordination.They promote a common understanding across AV companies, partners, and other IM practitioners.