Source-linked AI summary
A Dynamic Games Approach to Proactive Defense Strategies against Advanced Persistent Threats in Cyber-Physical Systems
Linan Huang, Quanyan Zhu
TL;DR
APTs create a difficult cyber-physical security problem because their stealthy, adaptive, multi-stage behavior leaves defenders uncertain about attacker types and intentions. The paper develops a dynamic game with two-sided incomplete information, Bayesian learning, and an iterative PBNE algorithm, then evaluates proactive defense in the Tennessee Eastman process. The case study reports improved defensive outcomes from early intervention, learning, and deception, while identifying model specification and scalability as limitations.
Problem
APTs exploit cyber-physical systems through stealthy, adaptive, multi-stage attacks, while defenders face persistent uncertainty and difficult detection.
Method
The paper models attacker-defender interaction as a multi-stage dynamic game with two-sided incomplete information, Bayesian belief updates, PBNE analysis, and an iterative computation algorithm.
Results
A Tennessee Eastman case study shows that a sophisticated defender can receive up to 56% higher payoff, while preventing effective reconnaissance can reduce attacker utility by at most 38%.
Takeaways & Limitations
Proactive multi-stage defense, online learning, and defensive deception provide quantitative strategies for mitigating cyber-physical attack risks without reducing legitimate users’ payoffs.
Takeaways & Limitations
The approach requires identifying defenders’ and users’ utilities and feasible actions at each stage, with future work targeting N-person and partially observable settings.
Abstract
from arXiv · showhide
Advanced Persistent Threats (APTs) have recently emerged as a significant security challenge for a cyber-physical system due to their stealthy, dynamic and adaptive nature. Proactive dynamic defenses provide a strategic and holistic security mechanism to increase the costs of attacks and mitigate the risks. This work proposes a dynamic game framework to model a long-term interaction between a stealthy attacker and a proactive defender. The stealthy and deceptive behaviors are captured by the multi-stage game of incomplete information, where each player has his own private information unknown to the other. Both players act strategically according to their beliefs which are formed by the multi-stage observation and learning. The perfect Bayesian Nash equilibrium provides a useful prediction of both players' policies because no players benefit from unilateral deviations from the equilibrium. We propose an iterative algorithm to compute the perfect Bayesian Nash equilibrium and use the Tennessee Eastman process as a benchmark case study. Our numerical experiment corroborates the analytical results and provides further insights into the design of proactive defense-in-depth strategies.
1. Introduction
APTs threaten cyber-physical systems through stealthy, multi-stage attacks that exploit connectivity, heterogeneity, and system complexity. The paper models these interactions as a multi-stage game with incomplete information and uses proactive, cross-layer defenses, Bayesian learning, and deception to improve security.
- Motivation: APTs exploit expanded connectivity, heterogeneous components, and system complexity to launch stealthy attacks across cyber-physical systems.These conditions enlarge the attack surface and create vulnerabilities that make defense arduous and costly.
- APT structure: APT campaigns progress through reconnaissance, initial compromise, privilege escalation, lateral movement, and mission execution, potentially causing physical damage.Defense-in-depth maps these sequential phases to multiple defender-attacker interactions.
- Detection challenge: Timely and effective APT detection remains difficult because attackers are stealthy and deceptive, leaving defenders uncertain about user types throughout the attack stages.The paper notes that breaches may take months to detect and that Stuxnet-like attacks can remain hidden for years.
- Approach: The proposed framework extends Bayesian games to a multi-stage nonzero-sum game with two-sided incomplete information and Bayesian belief updates.Nested mathematical programs are combined with Bayesian updating to compute the Perfect Bayesian Nash Equilibrium.
- Case study: The case study frames proactive defense as a quantitative mechanism for hardening SCADA systems against attacks that move from phishing and privilege escalation to sensor tampering.The Tennessee Eastman process illustrates how cyber-stage actions can affect physical-system efficiency without triggering alarms.
- Findings: The reported results favor early intervention, defender learning, and defensive deception over waiting until the final attack stage.Early cyber-stage responses reduce attacker dominance; online learning exposes hidden information, while deception increases attacker uncertainty and learning costs.
2. Dynamic Game Modelling of APT Attacks
The paper models multi-stage APT attacks and proactive defense as sequential games with incomplete information. Players use behavioral strategies and Bayesian beliefs based on delayed observations, while Markov state aggregation reduces computational complexity.
- Players and types: The game has user and defender players with private types, including adversarial or legitimate users and defenders with different sophistication levels.Each player knows their own type but not the other player's type, and both update beliefs during the interaction.
- Sequential interaction: Each player observes the other player's actions with a one-stage delay, so current decisions use previously observed action histories rather than real-time observations.The history is updated after both players' actions at the preceding stage become observable.
- Sequential interaction: At each stage, both players choose actions from finite action sets, and the system state evolves through a known transition function.The state can represent reconnaissance outcomes, user location, privilege level, or sensor status.
- Beliefs and strategies: Players form beliefs about the other player's type from available information and update those beliefs using Bayesian rules as new observations arrive.The model considers history-based and state-based information structures.
- Beliefs and strategies: The Markov belief update aggregates action pairs that produce the same next state, reducing memory and computational complexity compared with full history tracking.The subsequent analysis adopts this Markov belief update.
3. PBNE and Dynamic Programming
The paper defines equilibrium behavior for a multi-stage Bayesian game through sequential rationality and belief consistency. Dynamic programming evaluates immediate and future utilities, while PBNE requires beliefs and strategies to remain mutually consistent.
- Equilibrium definition: Players choose strategies that maximize expected cumulative utility under beliefs about the other player's private type.The framework uses behavioral strategies and type-contingent beliefs at each stage.
- Equilibrium definition: An ε-PBNE requires sequential rationality and belief consistency, with PBNE obtained when ε=0.The belief sequence must satisfy the specified consistency condition in addition to equilibrium optimality.
- Equilibrium interpretation: PBNE refines DBNE by requiring players to account for how current actions affect the opponent's beliefs at future stages.This requirement captures the informational consequences of strategic actions over the game horizon.
- Equilibrium interpretation: Sequential rationality ensures that unilateral deviations do not benefit a player at any initial state, making equilibrium strategies predictions of multi-stage behavior.The stated property also gives DBNE strategies strong time consistency, including off-equilibrium states.
- Dynamic programming: The value function recursively combines immediate stage utility with expected utility from the future state generated by current actions.At the final stage, the future-value term vanishes, leaving only immediate utility.
4. Computational Algorithms
The computational framework reduces one-stage Bayesian equilibrium to constrained optimization, extends it recursively across stages, and uses these components to support PBNE computation. The resulting programs characterize equilibrium strategies and utility-to-go values under specified beliefs.
- One-stage equilibrium: A constrained optimization program computes the one-stage SBNE and its associated value functions.The program is formulated for the final-stage Bayesian game and supports the multi-stage recursion.
- One-stage equilibrium: Theorem 1 establishes that solutions of the constrained program are equivalent to SBNE strategies under the stated private types, beliefs, and state.The proof uses finiteness and discreteness of action and type spaces to guarantee mixed-strategy equilibrium existence.
- One-stage equilibrium: The optimization variables include type-contingent mixed strategies and scalar variables representing equilibrium value bounds.The strategy-variable dimensions depend on action-set and type-set sizes.
- Multi-stage recursion: For earlier stages, the method replaces immediate utility with utility-to-go and solves constrained programs backward from stage K to stage 0.Each stage's optimization incorporates the next-stage value function and yields DBNE strategies for a fixed belief sequence.
- Multi-stage recursion: The recursive constrained programs produce DBNE strategies and expected cumulative utilities for each stage under a given sequence of beliefs.Theorem 2 characterizes the resulting strategy pair and utility values as optimal solutions of the stage-specific programs.
16 Terminate 17 end
The numerical PBNE procedure alternates backward policy computation with forward Bayesian belief updates. It terminates when the computed strategies satisfy sequential rationality under the updated beliefs, or otherwise repeats the iteration.
- PBNE iteration: Algorithm 1 initializes beliefs at every stage and alternates backward strategy computation with forward belief updating.The backward pass computes SBNE at the final stage and DBNE strategies at earlier stages.
- Termination: When PBNE existence is not guaranteed, the procedure uses ε-PBNE solutions instead.The algorithm is explicitly described as a numerical method for finding ε-PBNE.
- PBNE iteration: The forward pass updates beliefs using the Bayesian rule based on the computed strategy sequence.Both players update beliefs at each stage before the next backward computation.
- Termination: The iteration terminates when the strategy pair satisfies the sequential-rationality condition under the updated beliefs.Otherwise, the algorithm repeats backward policy computation and forward belief updating.
5. Case Study
The case study models a multistage APT that moves from initial compromise through privilege escalation to sensor compromise, while representing defender and user types, actions, and expected utilities. The Tennessee Eastman process illustrates how the framework quantifies operational impacts and evaluates recovery after sensor access is removed.
- Attack Scenario: The Tennessee Eastman scenario models three stages: initial compromise, privilege escalation, and sensor compromise of the physical system.The cyber-state diagram links these transitions to a physical attack through SCADA compromise.
- Types and Utilities: The user and defender have binary type spaces, with adversarial versus legitimate users and sophisticated versus primitive defenders.Utility matrices represent expected payoffs for both players under their types, actions, and system states.
- Initial Stage: Phishing Emails: At the initial stage, users choose email targets and defenders use security training, while phishing outcomes depend on user type, receiver, and training.Adversarial users gain attack rewards from inadequately trained receivers, whereas contacting avatars yields a faked reward but an unfavorable future state.
- Intermediate Stage: Privilege Escalation: At the intermediate stage, users may escalate privileges or perform no operation, while defenders may restrict or permit escalation.Privilege escalation from a manager’s computer can reach higher levels than escalation from an employee’s computer, and restriction reduces the escalation degree.
- Final Stage: Sensor Compromise: The final stage represents four privilege levels, whose effects on sensor compromise are quantified through a new per-hour Tennessee Eastman utility metric.The metric is applied in the game matrix to compare normal and compromised operational utilities.
- Recovery Results: After access to XMEAS(40) is lost at the 6th hour, the system partially recovers in about 16 hours; after XMEAS(17) access is lost at the 36th hour, utility returns to normal in about 13 hours.These recovery trajectories are reported for the simulated sensor-compromise scenario.
6. Computation Results
The computation results examine equilibrium strategies, belief updates, state transitions, and defensive deception in static and multi-stage Bayesian games. They show that proactive defense, Bayesian learning, and uncertainty about player types can improve defensive outcomes and constrain attacker behavior.
- Experimental setup: The experiments compute equilibrium strategies and utilities for static and multi-stage Bayesian games using the proposed algorithms and a concrete case study.The implementation uses MATLAB, YALMIP, and external solvers to solve the optimization problems.
- Final-stage Bayesian game: As the perceived probability of an adversarial user increases, the defender invests more in cyber defense and the adversarial user becomes more likely to choose unencrypted commands.The defender’s expected utility decreases, while complete monitoring becomes more likely after a belief threshold.
- Defensive deception: When the defender is more likely sophisticated, selective monitoring becomes more likely, surveillance costs can decrease, and the attacker switches to a lower attacking probability after the 0.5 threshold.After the threshold, both defender types share the same payoff, while the attacker becomes more cautious.
- Defensive deception: Double-sided incomplete information lets the defender approach selective monitoring probability 1 and maintain a constant expected payoff after the user’s belief crosses a threshold.The result indicates that uncertainty about the defender’s type can reduce defense effort while mitigating adversarial deception.
- Sensitivity analysis: As final-stage penalty effectiveness increases, the attacker becomes more conservative, and the highest-privilege state is no longer necessarily most favorable to the attacker.When final-stage defense is ineffective, reaching the highest privilege level can substantially increase attacker payoff and defender damage.
- Multi-stage dynamics: Bayesian updating improves estimates of user types, while proactive intermediate-stage defense shifts state transitions toward lower-privilege states and reduces access to high-damage states.Belief updating is especially effective with inaccurate priors, and higher adversarial-user beliefs increase the probabilities of states x_2 = 0 and x_2 = 1.
7. Discussions and Conclusions
The framework combines belief-based dynamic games and iterative PBNE computation to design proactive multi-stage defenses against stealthy attacks. Tennessee Eastman experiments report improved defender payoffs and reduced attacker utility, while identifying modeling complexity and extensions to multi-player, partially observable settings.
- The nested algorithm alternates forward belief updates with backward policy computation and quickly converges to an ε-PBNE with consistent beliefs and policies.
- 56% higher payoff is achieved by a sophisticated defender than by a primitive defender in the Tennessee Eastman case study.
- Preventing effectual reconnaissance increases defender utility by at most 41% and reduces attacker utility by at most 38%.
- Adversarial deception increases attacker payoff by increasing the defender’s uncertainty about the user’s type.
- Defensive deception creates uncertainty for attackers, encouraging more conservative behavior and mitigating attack losses for the defender.
- The main modeling challenge is identifying defenders’ and users’ utilities and feasible actions at each stage, with future work targeting N-person and partially observable games.