When Should a Human Take Back Control? Optimal Delegation under Turbulent AI Risk

Turbulent AI risk

• The paper calls incident-driven dependence across time turbulent AI risk: past incidents can raise the likelihood of further same-type or cross-type incidents, while effective supervision can weaken that dependence.The formulation treats decaying incident influence and supervision weakening propagation as assumptions, and fixes incident severity while increasing outcome uncertainty with incident intensity.Source: §1 · p. 2

Motivation

• Deployment examples motivate reassessing delegation after incidents, but do not establish self-excitation or intervention effectiveness; permissions and environment separation remain the first line of defense.The framework addresses residual oversight decisions rather than replacing those safeguards.Source: §1 · p. 2

Research gap

• Existing oversight and delegation formulations do not jointly model incident-history clustering, its suppression by supervision effort, and the delegation decision.The paper argues that supervision can affect future incidents and their downstream effects, making incident history relevant to both oversight duration and redelegation.Source: §1 · p. 2

Framework

• The framework combines human project actions and verification effort with switching between a costly human-assisted regime and a frozen black-box AI controller.The manager controls regime transition rates rather than directly choosing transition times, and switching costs induce hysteresis.Source: §1 · p. 2

Contributions

• Hawkes-PPO adapts PPO using a bank of exponential filters over observed incident times, learning from states, rewards, and incident history without being given the kernel or model parameters.The contribution is evaluated numerically in a synthetic environment with model-generated incidents against fixed regimes and an approximate full-information oracle.Source: §1 · p. 3

Results

• From matching starts, Hawkes-PPO improves on both fixed regimes by point estimates and significantly closes the corresponding pure-AI-to-oracle and pure-human-to-oracle gaps.The learned policy balances the extremes: pure human has the lowest volatility and highest terminal quality at the highest operating cost, while pure AI has the reverse pattern.Source: §1 · p. 3

Failure propagation in language-model agents

• Studies report propagation, feedback and guardrail interventions, and multi-agent cascade attenuation, but none jointly optimizes sustained delegation and costly mitigation under explicit incident-history excitation.The cited work includes variable gains from external feedback, possible degradation under intrinsic self-correction, and factual-information loss alongside attenuation across evaluated multi-agent cascades.Source: §2 · p. 3

Agent safety evaluations and control protocols

• Safety protocols use trusted monitoring, auditing, or action resampling, while oversight scaling studies examine more capable systems; oversight level remains a protocol parameter rather than an optimized response to incidents.The cited benchmark work also examines unsafe tool behavior and harmful tasks under jailbreaks, with the latter outside this paper's scope.Source: §2 · p. 3

Human–AI delegation

• Instance-level deferral and sequential delegation work allocate control between humans and autonomous agents, but do not explicitly model incident-history excitation or let effort suppress it.This paper adds those features and lets the human choose effort while holding control.Source: §2 · p. 3

Cyber risk management; Optimal switching and Hawkes-driven control

• Cyber-risk research studies resource allocation, contagion, and Hawkes control, while relaxed-switching and Hawkes-control work supplies related methods; this paper adds supervisory regimes, turbulent project volatility, and switching costs.Prior work on self-exciting financial activity supports estimability of the reduced form, not that it describes language models.Source: §2 · pp. 3–4

3.1 Project State, Information, and Control Regime

• The project state records evolving project quality, while regime 0 denotes human–AI-assisted control and regime 1 denotes full AI delegation.Under human control, the human chooses project actions and monitoring effort; the AI action is generated by a controller not optimized by the manager.Source: §3.1 · p. 4

3.2 Cascade AI-Turbulence Modeling

• State and action incidents are modeled as mutually exciting counting processes, so incidents of either type can raise the future occurrence rate of either type.The kernel entries encode self-excitation and cross-excitation; monitoring effort is assumed to reduce the baseline intensities and excitation effects.Source: §3.2 · p. 5

3.3 Relaxed Formulation and Project Dynamics

• In relaxed switching, the controller chooses regime-transition intensities rather than switching times, while project dynamics include operating actions, incident losses, and turbulence-induced volatility.Higher incident intensity raises project uncertainty, and greater human monitoring reduces it; effort lowers incident frequency rather than severity.Source: §3.3 · p. 6

3.4 Objective

• The objective trades terminal and running rewards against regime operating costs, incident-related quadratic variation, and switching costs.Human effort is typically costlier than AI delegation, while positive switching costs discourage costless high-frequency switching and generate hysteresis.Source: §3.4 · p. 6

4.1 Model and objective specification

• The model uses mean-reverting project dynamics with action-dependent baseline volatility and a bivariate Hawkes process approximated by a 20-term exponential mixture.Effort reduces excitation amplitudes; operating and switching costs are quadratic and symmetric, respectively.Source: §4.1 · pp. 6–7

4.2 Black-box AI modeling

• The black-box AI optimizes project value without accounting for turbulence generated by its actions, and its autonomous action is pretrained without turbulent risk, monitoring, or switching.Its recommendation error evolves in both regimes, with action-corruption events lowering recommendations before recovery drift moves the error toward zero.Source: §4.2 · p. 7

4.3 Reinforcement learning, Hawkes-PPO and value comparison

• Hawkes-PPO improves on both fixed regimes by point estimates and approaches the oracle, while the pure-human and pure-AI regimes represent opposite risk, quality, and operating-cost trade-offs.Among eight RL variants, only Hawkes-PPO improves on the fixed-Human benchmark from a Human start; switching gains are algorithm-dependent under the stated budget.Source: §4.3 · p. 7

4.4 Switching between autonomous AI and AI-assisted Human

• During AI spells, state-risk events raise Hawkes intensity and quadratic variation and precede increased switching intensity toward Human mode; event-free Human spells increase the intensity of returning to AI as event history decays.Switches are not forced at the maximum stochastic intensity or at a fixed threshold.Source: §4.4 · pp. 7–8

4.5 How incident dynamics and costs shape supervision

• Higher excitation increases human involvement, while higher fixed Human cost reduces Human time and higher per-switch fees reduce switching in both initial regimes.At baseline excitation, higher Human cost is accompanied by increased effort during Human operation, more incidents and bursts, and lower terminal quality; crossed checks show exceptions to some effort and quality responses.Source: §4.5 · p. 8

5 Conclusion

• The framework models incidents that can trigger further incidents and amplify uncertainty, and formulates switching between full AI delegation and human–AI-assisted decisions to account for evolving risk and intervention costs.Application to deployed systems would require empirical calibration and validation of incident dynamics.Source: §5 · p. 8

A Markovian approximation and observable-policy stability

• The appendix extends an exponential-approximation framework to intensity-dependent diffusion and outlines convergence results, including information-loss bounds when only the AI recommendation is observed under additional assumptions.It distinguishes full observation of the AI error process from observing only the recommendation.Source: A Markovian approximation and… · pp. 13–14

A.1 Model and assumptions

• The model assumes a K-term exponential approximation of the Hawkes kernel and regularity conditions that support unique nonexplosive solutions; no subcriticality condition is required on the finite horizon.The approximation and original systems share the same initial state and regime, with empty incident history.Source: §A.1 · pp. 14–15

A.2 Filter representation and inherited Hawkes estimates

• Evaluating the approximating intensity on the original incident history with the original effort has L1(P ⊗ dt) error at most CϵK.The preceding result is stated to hold uniformly over the controls considered.Source: §A.2 · p. 16
• When the original and approximating systems use the same predictable control, the common-control intensity estimate follows uniformly over admissible controls.The proof uses the same switching and incident processes and a uniform resolvent bound.Source: §A.2 · p. 16

A.3 Full-information approximation

• As K →∞, the approximated full-information values converge to the primal full-information value under Assumptions A.1–A.4.The theorem bounds the value difference by ΔK, which tends to zero.Source: §A.3 · p. 17

A.4 Information loss under current-observation feedback

• The information-loss analysis assumes an AI-error estimator and a classical solution to the coupled HJB system, whose existence is not addressed.The estimator assumption depends on the supervisor’s capacity to monitor AI-error estimation.Source: §A.4 · p. 19
• For fixed G, the current-observation policy class has a value bound relative to full-information control under the stated estimator and verification assumptions.The bound concerns policies that do not observe the AI error directly; its size depends on the estimator’s variance accuracy.Source: §A.4 · pp. 18, 20, 21
• Variance-reduction techniques for improving the estimator are left for future research.The upper bound on the difference between full-observation and estimated-error values depends on estimator variance accuracy.Source: §A.4 · p. 21

A.5 Error decomposition for the learned policy

• The discrepancy from the full-information optimum separates kernel approximation, current-observation restriction, and learned-policy residual errors.The kernel approximation error is bounded by ΔK; the current-observation bound need not vanish as K increases.Source: §A.5 · p. 21
• The analysis does not certify optimality of the trained PPO policy because the learned-policy residual includes neural-class approximation and finite-training suboptimality.The residual is retained explicitly rather than bounded by a certified optimality result.Source: §A.5 · p. 21
• Sampled deployment, numerical simulation, and evaluation under the original kernel can introduce discrepancies separate from the three displayed decomposition terms.The decomposition is for the continuously evaluated feedback associated with the learned network.Source: §A.5 · p. 21

A.6 Markovianization procedure and Exponential-mixture fit

• Hawkes-PPO approximates the non-Markovian kernel with a mixture of exponential kernels and uses the same procedure to update the memory process.The appendix provides mathematical justification for the fitting procedure.Source: §A.6 · p. 22

• The appendix covers the numerical environment, AI recommender, observation-based RL methods, and full-information benchmarks.These components are specified as the shared numerical setup.Source: B Numerical Implementation and… · p. 22
• All reported outcomes use the same physical model of the AI black box.Source: B Numerical Implementation and… · p. 22
• The full-information value functions define oracle benchmarks and do not replace the observation-based control problem in the main text.Source: B Numerical Implementation and… · p. 22

B.1 Choice of parameters and evaluation protocol

• The simulator uses shared candidate streams across policies, and the full-information oracle additionally observes the AI error and knows the model coefficients.Supervisory decisions occur every 0.05 time units, with five microsteps per decision interval.Source: §B.1 · p. 22

B.2 Project-Only AI Recommender: black-box design

• The nominal AI recommender is computed before supervisory training and then frozen, while its optimization ignores Hawkes risk, recommendation errors, mitigation, and switching.Source: §B.2 · p. 23

B.3 Hawkes-PPO algorithm and foundations

• Hawkes-PPO uses public observations including event counts and Hawkes filters, but neither the nominal controller nor the AI error state is supplied to its actor or critic.The recommendation input is masked in Human mode.Source: §B.3 · p. 24

B.4 Full-Information Switching Oracle

• The oracle is a known-model, full-information numerical benchmark, and its simulated Monte Carlo returns need not match continuous HJB values.Its controls are deployed in the common fixed-grid simulator, so it is not a certified exact optimum.Source: §B.4 · p. 28

B.5 Optimal Fixed-Human Benchmark: pure human mode design

• The benchmark solves the control problem with switching and AI delegation disabled while optimizing Human action and effort.Source: §B.5 · p. 28
• The fixed-Human value function retains action-event jump effects because those events can excite future events even when no recommendation is implemented.Source: §B.5 · p. 28
• A gated DGM network approximates the fixed-Human value function, using the same action maximization, mitigation search, and mark quadrature as the switching oracle.Source: §B.5 · p. 28
• The approximate fixed-Human policy achieves an independent Human-start objective of −96.027, with a 90% interval of [−96.191, −95.864].This is the return of an approximate HJB-derived policy deployed in the common simulator, not a certified value bound.Source: §B.5 · p. 28

B.6 Additional RL Baselines

• The baselines include PPO, SAC, DDPG, and CT-DDPG, each implemented with and without the same 40 Hawkes filters.Filtered versions receive the filter state; unfiltered versions retain the same network dimensions with filter coordinates set to zero.Source: §B.6 · p. 28
• The compared actors use two width-128 hidden layers, while SAC and DDPG use action-value critics and CT-DDPG uses a value and an advantage-rate network.Source: §B.6 · p. 29
• The common fitting setup uses 1,024 parallel environments, split evenly between the two initial regimes, with exploration active during training.Source: §B.6 · p. 29
• Under a common environment-interaction budget, Hawkes-PPO achieves the highest evaluated objective among the RL methods from both initial regimes and approaches the oracle.The comparison concerns selected fitted policies; confidence intervals quantify evaluation uncertainty, not variability across training seeds.Source: §B.6 · p. 30, Table 5 · p. 30
• The alternative methods’ weaker performance may reflect optimization difficulties under the budget, and CT-DDPG’s selected checkpoints deteriorate, but neither result establishes nonconvergence or inherent PPO superiority.Source: §B.6 · p. 30

B.7 Comparative statics, supervision mechanisms, and numerical sensitivity

• The study varies excitation, Human fixed cost, and per-switch fee one factor at a time, then tests crossed excitation and cost settings against the shared baseline.Source: §B.7 · p. 30
• Reported intervals condition on selected fitted policies and exclude training-seed variability, checkpoint-selection uncertainty, and discretization error.They are also not adjusted for multiple comparisons.Source: §B.7 · p. 31
• Raising excitation from 0.5 to 1.5 reduces mean incidents and increases terminal quality for both initial regimes, but these outcomes combine a changed incident environment with a separately fitted policy’s response.For AI starts, mean incidents change from 5.595 to 5.127 and terminal quality from 1.168 to 1.881; for Human starts, incidents change from 5.581 to 4.918 and quality from 1.189 to 2.023.Source: §B.7 · p. 31
• At baseline excitation, raising Human fixed cost from 0.75 to 1.25 increases incidents and burst probabilities, while conditional effort rises and total Human effort exposure falls.Incident counts rise from 5.054 to 6.202 for AI starts and from 4.926 to 6.130 for Human starts; burst probabilities rise from 36.7% to 57.4% and from 33.5% to 56.5%, respectively.Source: §B.7 · p. 31
• Increasing the per-switch fee multiplier from 0.5 to 2 reduces mean switches for both starts, while incident and burst responses differ by starting regime and terminal quality rises for both.Mean switches fall from 1.344 to 0.907 for AI starts and from 1.924 to 0.659 for Human starts.Source: §B.7 · p. 31
• In crossed checks, higher excitation increases Human time and terminal quality and reduces incidents and burst probability, whereas higher Human cost reduces Human time and increases incidents and burst probability.These directions hold for both initial regimes at the crossed settings.Source: §B.7 · p. 32
• Effort and quality responses are conditional: higher Human cost changes conditional effort in opposite directions at low versus high excitation, and higher cost increases quality at high excitation but decreases it at low excitation.At high excitation, terminal quality increases by 0.133 for AI starts and 0.147 for Human starts despite higher incident burden.Source: §B.7 · p. 32
• Endpoint comparisons do not establish monotonicity, and switching responses depend on starting regime and the cost-risk combination.For example, with expensive supervision, AI-start incident counts across excitation multipliers 0.5, 1, and 1.5 are 5.606, 6.202, and 5.261.Source: §B.7 · p. 32