A Review of Attacker–Defender Games and Cyber Security

: The focus of this review is the long and broad history of attacker–defender games as a foundation for the narrower and shorter history of cyber security. The purpose is to illustrate the role of game theory in cyber security and which areas have received attention and to indicate future research directions. The methodology uses the search terms game theory, attack, defense, and cyber security in Web of Science, augmented with the authors’ knowledge of the field. Games may involve multiple attackers and defenders over multiple periods. Defense involves security screening and inspection, the detection of invaders, jamming, secrecy, and deception. Incomplete information is reviewed due to its inevitable presence in cyber security. The findings pertain to players sharing information weighted against the security investment, influenced by social planning. Attackers stockpile zero-day cyber vulnerabilities. Defenders build deterrent resilient systems. Stochastic cyber security games play a role due to uncertainty and the need to build probabilistic models. Such games can be further developed. Cyber security games based on traffic and transportation are reviewed; they are influenced by the more extensive communication of GPS data. Such games should be extended to comprise air, land, and sea. Finally, cyber security education and board games are reviewed, which play a prominent role.


Introduction
This article reviews how game theory plays a role in attacker-defender games and cyber security.Cyber security has received increased attention in recent years due to the emergence of the Internet, over which text, voice, and, increasingly, money flow.Players such as firms, organizations, and governments hold, produce, and seek to protect assets and information.The same or other types of players may seek to steal, destroy, or compromise assets and information, either as competitors or with nefarious objectives.This review starts with an overview of attacker-defender games, multiple attackers and defenders, various defense methods, and incomplete information.Having established a foundation in these models, the review proceeds in more detail to cyber security, which typically involves defenders and attackers.Information sharing and security investment are assessed in various circumstances.Models of cyber security stockpiling, deterrence, and resilience are evaluated.Thereafter, reviews are conducted on stochastic games, transportation games, and security education and board games.
Introducing game theory, including attacker-defender games, to cyber security emphasizes how players have different preferences, beliefs, risk attitudes, and strategy sets, which may make the analysis more realistic.The defender seeks to enhance the cyber security of targets that it owns, controls, or operates, while the attacker seeks to compromise this security, e.g., by destroying or stealing targets or broadcasting or falsifying confidential information associated with the targets.
The reviewed articles were identified by using the search terms game theory, attack, defense, and cyber security in Web of Science, supplemented with the authors' knowledge of the field.The research judged to be most impactful was included.

This Article's Contribution beyond Earlier Reviews
Table 1 provides an overview of 12 earlier reviews.

Reference Topic Focus Points
Amin and Johansson [1] Dynamic games in cyber security Security and efficiency can conflict.Topics are asymmetric information, evolution of network security, vulnerability assessment, cyber-induced failures, incentives, and design of mechanisms to reduce risks.Six potential cyber game changers are that the cyber environment changes in terms of new computing paradigms and new territories for network complexity, new technology trends such as big data analytics and resilient self-adaption, and cybertechnology breakthroughs such as mixed-trust systems and active defenses.
Pala and Zhuang [10] Information sharing in cyber security Review of focus and methodology within cyber security information sharing involving firms, governments, citizens, and adversaries.The focus is on the actors involved, types of information shared, current legal baseline, information-sharing organizations/policies/architectures, benefits of sharing, and concerns/costs/barriers of sharing.Qualitative approaches discuss challenges and barriers to public/private collaboration pertaining to privacy and liability to ensure secure and effective sharing.Quantitative approaches balance cyber security investment and information sharing to ensure effective incentives.
Roy, et al. [11] Cyber security network games Applying game theory to network cyber security.Their classification taxonomy distinguishes cooperative and non-cooperative games.The latter can be static or dynamic.Static games can have complete or incomplete imperfect information.The latter can be Bayesian or non-Bayesian.Dynamic games can have the four combinations of complete/incomplete and perfect/imperfect information.
Sedjelmaci, et al. [12] Cyber security games for intelligent transportation systems Cyber security defense of intelligent transportation systems.Non-cooperative games are divided into interdiction games, mean field games, Stackelberg games, Bayesian games, and zero-sum games.Cooperative Stackelberg games are considered.Cost and security level of these games are assessed as low, medium, or high.
This review considers the more recent literature and has, to some extent, a broader focus than the earlier cyber security reviews.More specifically, as shown in Table 2, the review starts with a focus on attack and defense, acknowledging the centrality of intentional intelligent adversaries in cyber security, as opposed to other areas of risk analysis involving nature (weather, etc.) and technology (mechanical failure, etc.).Given this focus, the review proceeds with incomplete information.The defenders may not know who the cyber attackers are; their competences, preferences, and beliefs; and when, how, and who they may attack.The attackers may not know the value of the objects that the defenders protect, which objects are protected, and how they are protected.Since this information can be incomplete, a natural remedy in cyber security is information sharing, considered in Section 6.The remainder of the review delves into more specific areas of cyber security, i.e., stockpiling, deterrence and resilience, stochastic games, education, board games, traffic and transportation, and power systems.

One Player Defending or Attacking One Component in a System
One defender and one attacker may defend and attack entire systems or individual components within each system.One example of the latter is the analysis by Hausken [13] of probabilistic risk analysis and game theory.He shows that strategies by individual players at the component level may impact the risk at the system level.For example, consider a series system, e.g., a circular island where each citizen owns a wedge-shaped slice.To avoid flooding, each citizen may build a dike.Flooding may occur through the lowest dike, which is the weakest link.One Nash equilibrium, if dikes are expensive or flooding is unlikely, is that no citizens build dikes.A second Nash equilibrium is that all citizens build dikes.Next, consider a parallel system, e.g., multiple antimissile batteries, each controlled by one player, firing at an incoming missile against a city.It suffices that one antimissile battery can shoot down the incoming missile.One Nash equilibrium, if shooting down a missile is expensive, is that no player shoots down the missile.Additional Nash equilibria consist of one of the players shooting down the missile, which constitutes both a battle of the sexes game and a chicken game.The player that shoots down the missile earns lower utility than the other player(s).Next, consider a summation game, e.g., multiple companies that may or may not aid each other.If aiding is very expensive, no company will aid in a mutual defection game.If aiding is very expensive, no company will aid in a mutual defection game.If aiding is intermediately expensive, no company will aid in a prisoner's dilemma.If aiding is not expensive, all companies aid in a mutual cooperation game.Hausken [13] further analyzes combined series and parallel systems.

Multiple Attackers and/or Multiple Defenders
Although most of the literature studies a single attacker and a single defender, in reality, there could be multiple attackers interacting with multiple defenders.Ackerman, et al. [14] study the potential collaboration among multiple extremist groups from diverse milieus, despite significant ideological disparities, to align to a certain extent, enabling operational collaboration against Western societies.Xu and Zhuang [15] study a sequential one-defender-N-attacker game where N attackers are treated as independent agents.On the defender side, Zhuang, et al. [16] and Zhuang [17] study an interdependent security problem where multiple defenders are connected in a network but make security investment decisions under different discount rates.
Combining multiple attackers and multiple defenders, Shan and Zhuang [18] study the unique problem of subsidizing to disrupt a terrorism supply chain involving multiple governments and multiple terrorist groups.They first study two subgames: a proliferation game between terrorist groups and a subsidization game between governments.They then integrate these two subgames to study how the victim government can strategically use subsidies to incentivize the host government to disrupt the terrorism supply chain.

Multiple-Period Attacker-Defender Games
Researchers have also studied scenarios where an attacker and a defender interact over multiple periods.For example, Hausken and Zhuang [19] study a T-period game where both the attacker and the defender attack and defend, considering different scenarios, such as changing resources, the random determination of resources, and the impact of previous attacks on future resource allocation.Several interesting strategies are then studied in a multiple-period game setting, including terrorists who accumulate or stockpile resources [20][21][22] and terrorists who choose in which period to attack and could be deterred [23].Note that, within each period, most works study either a simultaneous-move or a sequential-move two-stage game (where the defender typically moves first).However, some works also study multiple-stage games, e.g., such that retaliation may occur in the third stage; see Shan and Zhuang [24].

Various Characteristics of Defense and Attack
Three common characteristics are security screening and inspection, detecting invaders, and defense through jamming.

Security Screening and Inspection
One specific attacker-defender game involves security screening and inspection in the context of airport security, visa approval, and cargo container inspections.The defender sets a probability of screening when facing an adaptive attacker who decides whether to attack or not.Wang and Zhuang [25] study the balance of congestion and security among three groups of players with incomplete information.These are a defender who sets the probability of screening, an attacker who decides whether to attack, and a group of normal applicants who decide whether to join the queue.Extending their work, studies have been performed on a two-stage [26] and N-stage [27] security screening problem with screening errors, a parallel-queue security screening problem with incomplete information [28], and impatient applicants [29].Haphuriwat, et al. [30] assess how to deter the smuggling of nuclear weapons in container freight through detection and retaliation.They find that unless the defender imposes high retaliation costs, 100% inspection is likely needed, and deterrence with partial inspection may be challenging.However, when the attacker can be credibly threatened with costly retaliation, partial inspection may be sufficient to deter nuclear smuggling attempts.Brown, et al. [31] develop a max-min project management critical path method considering how to interdict a nuclear weapons project.The prolifer-ator seeks to complete a batch of fission weapons quickly, while the interdictor seeks to delay indefinitely.They consider three uranium enrichment technologies that all involve cascade loading, causing fragility for the proliferator.This, in turn, enables the interdictor to intervene diplomatically, economically, and militarily.

Detecting Invaders
Detecting invaders is a challenge.Assessing detection at sea, Gerald, et al. [32] develop a defender-attacker model considering how to position patrol vessels optimally to detect an adversary in speedboat(s) seeking to evade detection (e.g., through elevated obstructions) while attacking.The vessels have a surface search radar, radios, and a machine gun.Alert defenders are almost guaranteed to detect attackers by optimal prepositioning, accounting for bottlenecks and the restricted navigational access channels to ports.They sometimes recommend positioning far from bottlenecks to better detect stealthy evading attackers.Assessing how to defend an oceanic bastion against submarine attacks, Brown, et al. [33] present a two-person zero-sum game for the application of ships, aircraft, etc., in defense.The attacker knows the ship locations but not where the other defense platforms are located.The attacker chooses a path towards the bastion, while the defender maximizes the detection probability.Considering detection more generally, Orojloo and Azgomi [34] develop a game for each of the two phases of intrusion and disruption by an attacker of a cyber-physical system.The attacker seeks to understand the system's failure conditions, control principles, and signal processing.The system evolves continuously between different states according to ordinary differential equations.They estimate the system's security according to metrics, e.g., availability and mean time to system shutdown, and exemplify with a chemical plant.

Defense through Jamming and Eavesdropping
Jamming is another common defense method.Focusing on a wireless mesh network that enables data, voice, and video communication, Nicholas and Alderson [35] develop a defender-attacker-defender model for its design, attack, and operation.The defender uses radio propagation over terrain downloaded from the Internet and minimizes the worst possible disruption that an adversary can inflict through jamming, i.e., electromagnetic interference.Jamming can be combined with eavesdropping.Xu, et al. [36] consider various games within mobile communication technologies and the commercial use of 5G, including eavesdropping and anti-eavesdropping and jamming and anti-jamming.They assess potential research directions.Xu and Baykal-Gursoy [37] consider a non-zero-sum game for the defense of a wireless communication network of channels through jamming an adversarial eavesdropper.When the eavesdropper attacks all of the channels, they solve a convex optimization problem.Otherwise, the eavesdropper selects channels probabilistically, which impacts the defender's defense.A unique Nash equilibrium is obtainable under certain conditions.A strategy iteration algorithm determines an equilibrium power allocation strategy that outperforms assuming that every channel is under attack.Finally, a stochastic approach is presented by Garnaev, et al. [38].They develop stochastic communication games where a user chooses optimally whether to transmit, which may lead the adversary to jam or delay the transmission, which may enable the detection of the adversary if it continues actively to jam instead of eavesdropping passively.The adversary may find eavesdropping less efficient if it cannot time-efficiently utilize the compiled information.They find that incorporating a detection time slot in the transmission may improve the communication reliability and secrecy.

Protecting Many Targets
The protection of many targets is analyzed by Bier [52].She considers how a defender allocates resources to protect many targets against an attacker with unknown preferences who chooses one target to attack and observes the defender's resource allocation.The increased defense of one target decreases the attack probability against this target.Some targets may optimally be left undefended.Higher vulnerability at one target may be optimal, even if lower vulnerability could be achieved at zero cost.The defender prefers centralized allocation.A larger number of targets to defend requires the number of valuable targets to be bounded for the defender to cost-effectively decrease the attack success probability.The optimal resource allocation can be nonmonotonic in how the attacker relatively values the outside option.The defender prefers its allocation to be public.Various extensions of this work have been presented.First, Bier, et al. [53] consider how to protect against an unknown attacker, assuming that the attack success probability depends on how the defense resources are allocated and that the attacker can be of as many unknown types as the number of assets attacked.Second, Hausken [54] assumes that the attacker's resources and target valuations are drawn probabilistically and that the attack success probability depends on how the defense and attack resources are allocated.More specifically, in a two-period game where the defender moves first and the attacker moves second, which asset is attacked depends on the attacker's type, the unit attack costs, the contest intensity, and the defense.An interior equilibrium for two equivalent assets exists for a low contest intensity.A corner equilibrium with no defense exists for a high contest intensity when the attacker is resourceful.The isoutility curves can be both upward-sloping (the defender prefers to invest less in defense) and downward-sloping (e.g., when one asset has a low value or high unit defense cost), which contrasts with the work of Bier, Oliveros and Samuelson [53], finding upward-sloping isodamage curves near the axes.In other words, the defender prefers to invest less, which increases the probability of successful attacks on both assets.Finally, Yolmeh and Baykal-Gürsoy [55] consider a simultaneous game with an unknown distribution of information about the target values and detection probabilities.The attacker maximizes the damage or infiltrates multiple targets that the defender defends.They determine the existence and uniqueness of a Nash equilibrium for two games and the shape of the Nash equilibrium for the third game.He and Zhuang [56] study the possibility of contracts or mutually beneficial arrangements between a government and a terrorist group using a sequential game framework.Equilibrium solutions are derived for models with complete and incomplete information, revealing that successful contracts can potentially deter attacks and increase the payoffs for both the government and certain types of terrorist groups, providing new insights into counterterrorism strategies.

Secrecy and Deception
Most of the literature assumes that the information disclosed in the attacker-defender game is truthful, e.g., the second mover is able to perfectly observe the first mover's decision and then respond accordingly.However, in reality, some information may be kept secret (Dighe et al., 2009) or even deceptive, leading the other players to potentially learn the truth [57].Zhuang and Bier [58] show that truthful disclosure is preferred in games of complete information.Zhuang and Bier [40] and Zhuang, Bier and Alagoz [42] use a Bayesian Nash equilibrium approach to study secrecy and deception, where the defender sends a signal that is different from what the defender actually does.Zhuang and Bier [41] summarize the potential reasons for secrecy and deception in homeland security resource allocation games.Hunt, et al. [59] study disclosure and secrecy using a signaling game in the context of technology adoption for airport security.

Threat Propagation, Denial of Service Attacks, and False Alarms
Threat propagation is analyzed by Liu, et al. [60].They observe that attackers in distributed cyber-physical systems tend to initiate attacks in the outer nodes and proceed towards the inner nodes, hoping not to be detected.Defining the weighted colored Petri net, the authors model threat propagation between nodes as a mixed-strategy Bayesian attackdefense incomplete information game.They determine the Bayesian Nash equilibrium, threat propagation matrix, and security state vector to determine the attack paths and losses.Denial of service attacks are investigated by Gupta, et al. [61].In a zero-sum game, they determine the saddle-point equilibrium.More generally, they develop an asymmetric information non-zero-sum game between an attacker and a cyber-physical system controller.Assuming resource-constrained players, an algorithm determines a subclass of Nash equilibria.Some of the research considers the role of false alarms.Han and Choi [62] present a dynamic game where a defender is penalized for false alarms in a cyber security intrusion detection system.They find that the demand and supply of cyber insurance can be low and that decreasing the operational risk decreases the cyber risks and increases the false alarm rate.They recommend a government intervention policy, which implies a socially optimal outcome.

Trust and Reputation
Trust and reputation often play a role in incomplete information games.Trust and reputation are linked since a reputable player more likely earns the trust of other players.Njilla, et al. [63] consider a cyberspace game where service providers seek to maintain a reputation that ensures economic gains, the users prefer to trust the service providers, and attackers seek to breach a provider's database and expose the users' private information.They recommend that service providers should invest in cyber security.Han and Choi [64] consider a reputation game where an attacker can pretend to be a normal user.The defender may have to announce that it has been attacked without knowing whether it has been attacked.They demonstrate a sequential equilibrium's existence and uniqueness in Markov strategies and propose empirically and theoretically how to calibrate the attack probability.

Information Sharing and Security Investment in Cyber Security
In risk analysis, information is often dispersed differently across players.This problematizes whether players have incentives to share information strategically.Gordon, et al. [65] assess the cost-side effects regarding how information sharing impacts information security.They illustrate how underinvestment in security may follow from a tradeoff between information security investment and free-riding.Gal-Or and Ghose [66] analyze the demand-side effects of economic incentives for information.They show that security investment and information sharing are strategic complements, while Hausken [67] shows that they are strategic substitutes.Hausken [67] finds that information sharing between firms is inversely U-shaped in an attack.He models information sharing across connected firms, finding that firms tend to underinvest and free-ride unless a single well-respected social planner moves first and coordinates sharing.He shows that individual optimization implies free-riding, which can be curtailed by a social planner.If the social planner moves simultaneously with the firms, it imposes unreasonably high sharing.If the social planner moves before the firms, it imposes reasonable sharing.
For a simultaneous move game, Hausken [68] finds that two hackers free-ride on each other's information sharing when attacking one firm.Each hacker's attack and information sharing are strategic complements, while one hacker's attack and the other hacker's information sharing are strategic substitutes.Hausken [69] analyzes information sharing between two attackers attacking one firm in a four-period game, firm-hacker 1firm-hacker 2. Hacker 1 ′ s information sharing increases both hackers' focus on reputation gains.Hacker 2 ′ s attack is deterred by hacker 1 ′ s focus on reputation gains.
Generalization to a similar four-period game with an attack on two firms is described by Hausken [70].Two hackers share information.Two firms share information.If hacker 2 is disadvantaged in some way, it receives less information from hacker 1. Mixed motives may exist between information sharing and reputation gains.Hacker 2 ′ s attack is deterred by the first hacker's reputation gain.Increased interdependence between firms causes more information sharing between hackers.The firms deter disadvantaged hackers.Increasing information-sharing effectiveness causes firms to substitute from defense to information sharing with each other.
Four four-period games where one firm defends proactively or retroactively against hacker 1, and thereafter against hacker 2, are analyzed by Hausken [71].Hacker 1 ′ s attack and information sharing are strategic substitutes.Various results are developed.For example, when the firm is proactive in period 1, hacker 1 ′ s information sharing decreases with hacker 2 ′ s attack cost.The firm's deterring effort in eight corner solutions is proportional to the deterred hacker's valuation and inversely proportional to the deterred hacker's unit effort cost.When hacker 1 exerts higher effort and shares more information, lower defense by the firm is sufficient to deter hacker 2. The results contrast the literature where the advantaged player commonly prefers to move first.
In the context of cyber security information sharing, He, et al. [72] provide a decisiontheoretic approach, discussing various information-sharing structures, strategic interactions between stakeholders, costs, benefits, and the mechanism of information sharing to enhance the understanding and provide a detailed cost-benefit analysis of this public-private partnership.Pala and Zhuang [10] examine the literature on cyber security information sharing and explore considerations of various stakeholders, including firms, governments, citizens, and adversaries.They highlight the prevalence of both qualitative and quantitative approaches, with quantitative approaches addressing challenges in public-private collaboration and proposing game-theoretic secure sharing mechanisms.Levitin, et al. [73] assess how a defender stores information to prevent an attacker from stealing or destroying it.The defender, either minimizing the probabilities of information detection and data theft or minimizing the cost, allocates information to multiple blocks and maximizes the number of copies of each block subject to resource constraints.
Tosh, et al. [74] present an evolutionary game where organizations share cyber security information to ameliorate cyber attacks against critical resources.They account for a dynamic cost adaptation scheme, a distributed learning heuristic.The economic benefits of information sharing are assessed, together with the consequences of not taking part in the game.
Many have sought to understand the role of cyber security investment in games of firms interconnected and interdependent in communication and supply chain networks.In many of these games, firms face the potential to suffer a cyber attack (or breach) directly or indirectly through shared network ties and must choose their levels of investment and information sharing.For example, Bandyopadhyay, et al. [75] present a model of firms interconnected in supply chain and communication networks, finding that the security investment depends on the nature of the dependence.They find that when firms are connected through communication networks alone, they are incentivized to underinvest and free-ride.In contrast, when firms are connected through both communication networks and supply chains (i.e., production, ownership, or financial ties), they are incentivized to increase their investment when tightly integrated and decrease their investment when they are loosely integrated.Nagurney, et al. [76] model retailers and customers connected in a supply chain, finding that increased interdependence can increase the vulnerability to an attack.
In a multiform model of investment, Nagurney and Shukla [77] find that information sharing leads to both financial and security benefits.Simon and Omar [78] present a model of a supply chain with multiple defenders (one at each node) and a single attacker.They model attackers as either non-strategic (random attacks) or strategic (adaptive adversary) and defenders as either uncoordinated or coordinated.They find that the security investment is suboptimal in the absence of coordination.Finally, Li and Xu [79] discuss overcoming the prisoners' dilemma and free-riding challenges in a supply chain game through coordination mechanisms including joint decision-making, risk competition, and information sharing.They find that joint decision-making and security risk compensation are preferable to stimulate firms' investments and decrease costs compared with security information sharing.The majority of the literature seeks to better understand defensive strategies; see, e.g., Alpcan and Basar [80], Acemoglu, et al. [81], and Kovenock and Roberson [82].A small amount of the literature seeks to understand the behavior of cyber attackers, including the stockpiling of zero-day cyber vulnerabilities.For example, Wang, et al. [83] develop a two-period game-theoretic model of zero-day attacks with stockpiling.In period 1, one player produces zero-day exploits for immediate deployment or stockpiling.In period 2, the same player repeats this procedure, supplemented with stockpiling from period 1.The other player defends in both periods.They show that the first player stockpiles when its unit effort cost of producing zero-day exploits is lower in period 1.It may even accept negative expected utility in period 1 if it is compensated in period 2. With a higher contest intensity in period 2, the players compete more fiercely with each other in both periods.Hausken and Welburn [84] consider a cyber war where two players produce zero-day cyber exploits allocated in a cyber attack and stockpiling and defend against attacks.Each player's utility is inversely U-shaped in each player's unit defense cost.Higher contest intensities cause higher effort until the players' resources are fully exploited and they receive zero expected utility.Lower Cobb-Douglas output elasticity for a player's stockpiling of zero days causes higher attack and expected utility, which eventually reaches a maximum; this is detrimental for the opposing player.Schramm, et al. [85] consider a zero-sum game where each player decides if and when to use a munition based on a cyber exploit discovered according to an independent random process.Each player's payoff increases if it postpones exercising the munition, which matures, but receives zero payoff if the opposing player also discovers the munition.They determine the optimal munition exercise strategies and quantify the value of cyber conflicts.

Cyber Deterrence
Others have used game theory, often leveraging the foundational game-theoretic work of Schelling [86], Dresher [87], and others, along with strategic discussions of cyber deterrence (e.g., Libicki [88], Nye [89], Crosston [90], Jensen [91], Clarke and Knake [92], Jasper [93]), to study the ability to defend against and deter cyber attacks.Edwards, et al. [94] introduce an attribution game in a defender-attacker setting where defenders are uncertain about the ability to attribute an attack to an attacker (the attribution problem).Baliga, et al. [95] advance the attribution game in a model with a single defender, multiple attackers, and uncertain attribution.Baliga, De Mesquita and Wolitzky [95] find that more frequent attacks from any given attackers increases their likelihood of successful attribution relative to others, which results in endogenous complimentary among attackers, where attackers are only as aggressive as the most aggressive attacker and not more.Moreover, Welburn,et al. [96] present a model between a defender and a cyber attacker with imperfect attribution, where the defender also can signal its capability to retaliate against an attack.They find the presence of equilibria where deterrence is achievable through signaling, while, in a counterintuitive case, defenders may be able to increase their rewards by luring weak attackers.

Cyber Resilience
Assessing cyber resilience in firms, organizations, and societies, Hausken [5] considers threat actors and non-threat actors nested inside each other.They possess competence, resources, technology, and tools.They choose strategies based on their preferences and beliefs, which influence and are influenced by cyber resilience.Cyber resilience relates to the Internet of Things, where artificial intelligence, machine learning, and robotics play increasing roles.Vulnerabilities may follow from possible excessive trust in computers and software, inadequate data handling, deficient technology, and too many attack surfaces.Cyber resilience also relates to cyber insurance due to cover limitations, preconditions or entry requirements for cyber contracts, data compilation, and the handling of incident responses.Zhu and Basar [97] discuss game-theoretic methods for the robustness, security, and resilience of cyber-physical control systems.They distinguish between robust, adaptive, and stochastic control, to address vulnerabilities within these frequently open networks.Backhaus, et al. [98] consider designs of attack-resilient smart grids and control systems, accounting game-theoretically for machine-mediated human-human interactions.They assess outcomes via simulations and consider how to develop tools to defend against a cyber-physical intruder.

Cyber Security Stackelberg Games
A few cyber security Stackelberg games have been identified.First, Zhang and Malacaria [99] apply mixed-integer conic programming to develop optimal cyber security controls against multi-stage attacks.They use preventive optimization, a learning mechanism, and online optimization, shown to be a Bayesian Stackelberg game solution and more efficient than, e.g., the Harsanyi transformation.Second, Shukla, et al. [100] develop a zero-sum, two-player Stackelberg game where a sufficiently resourceful defender protects a networked control system of nodes robustly against a budget-constrained cyber attacker.They solve this with backward induction and exemplify with electric power systems.Third, Shen and Feng [101] present a Stackelberg interdependent security game between individual self-interested non-malicious cyber-physical systems that are vulnerable due to their distributed and hierarchical nature.They determine pure strategy equilibria and the strategy gap between the individual and social optimum.

Cyber Security Games for Power Systems
Two cyber games have been identified for power systems.First, Gao and Shi [102] present a three-stage defender-attacker-defender game for cyber-physical power systems.They account for the operation risks and vulnerabilities of transmission lines.To mitigate attacks, they incorporate the time delay of system recovery and distributed denial of service and apply the particle swarm optimization approach and sequential quadratic programming.The approach is validated through two case studies.Second, Li, et al. [103] develop a graphical evolutionary game as competition between virus propagation and countermeasures to protect cyber nodes within power systems.Each node plays as a defender or an attacker according to its state.Probabilistic strategies and state transfers depend on a death-birth rule, which dynamically impacts the infection probability of each substation over time.

Stochastic Cyber Security Games
The relevance of incomplete information and uncertainty in cyber security suggests the relevance of stochastic analysis.Hence, as expected, several stochastic cyber security approaches have been proposed, as shown in Table 3.
Eight of the 12 games in Table 3 assume one defender and one attacker, covering phenomena such as intrusion detection, jamming and eavesdropping, and one epidemic model.The remaining four games model multiple players-typically multiple defendersand, in two models, also multiple attackers.Two models assume bounded rationality.Two of the models apply mean-field theory.Mean-field game theory intersects game theory, stochastic analysis, and control theory.Each player plays against a field of players, which can be realistic for many players, so that a representative player for the field can be identified.It is inspired by mean-field theory in physics, where individual particles, among many particles, impact the system negligibly.

Cyber Security Games on Traffic and Transportation
Table 4 shows cyber security games on traffic and transportation.Traffic and transportation occur in the air, on land, and at sea; may or may not involve humans; and may or may not involve transported goods.At present, the emergence of self-driving or driverless vehicles is prominent; they require the wireless communication of a plethora of data across multiple vehicles and data centers.Examples of data are the GPS coordinates, speed, direction, size, and types of other vehicles and non-vehicles, as well as the geography, weather, road conditions, laws and regulations, etc.Similar types of communication occur or may occur between units in the air or at sea and in air/sea/land traffic management.Such communication inevitably invokes cyber security, where both benevolent and non-benevolent players seek to obtain their objectives according to their beliefs.

Cyber Security Education and Board Games
The advances of computer technology have facilitated the simulation of cyber security phenomena.This may explain the emergence of games to facilitate learning, training, education, and awareness about cyber security.Some of these games are board games, tabletop games, card games, and experimental games; see Table 5.This review distinguishes itself from the earlier 12 reviews listed in Table 1 by incorporating more recent research and by starting with the longer and broader history of attacker-defender games, before proceeding with the shorter and narrower history of cyber security.The study of cyber security brings the need to review incomplete information and information sharing, followed by cyber stockpiling, deterrence, resilience, and stochastic analyses.Thereafter, two phenomena in which cyber security is essential are reviewed: traffic and transportation impacted, for example, by the communication of GPS data-e.g., between self-driving cars-and education and board games facilitated by the increasing use of computers and electronic devices.
Table A1 in Appendix A shows the players and phenomena considered in the reviewed articles.Fifty-seven of the 132 works involve one defender and one attacker, which is a common game-theoretic approach, while the others involve multiple players in various constellations.Future research may assess each phenomenon involving one defender and one attacker to determine whether more players can be introduced to obtain more realistic analyses.Similarly, some of the research with multiple players may remove or introduce players to realistically capture phenomena.Table 3 shows the players, assumptions, methods, and results of stochastic analyses of cyber security.Table 4 shows the game assumptions, methods, and results of cyber security games on traffic and transportation.Table 5 shows the game objectives, methods, and results for education and board games.These three tables provide a strong basis for the assessment of alternative phenomena, different constellations of players, different assumptions, and different games and methods, which may yield different results.Several of the models are in need of empirical validation to ensure that they meet societal needs.
Players in the literature commonly value targets subjectively along one dimension, which is challenging for targets of different types, which may differ in their economic, human, and symbolic value.Future research may value targets along multiple dimensions.To the extent that targets are not clearly distinguishable, they may be aggregated into distinguishable clusters, applying Simon's [134] principle of "near decomposability".The same applies for players that may have partly aligned preferences or beliefs, which may be aggregated into unitary players with preferences or beliefs approximating those of their constituent players.Future research should assess which players and targets are relevant to include in the analysis, which games they play, in which sequences the players move, whether the information is incomplete or uncertain, and whether the players are fully or boundedly rational, adjusted towards societal needs for an enhanced understanding.
Common methodologies in the reviewed literature, in addition to game-theoretic analyses implying, e.g., equilibria and min-max solutions, are simulations applying various algorithms, probability theory, and stochastic analysis.Future research may apply alternative or novel methodologies, e.g., machine learning, artificial intelligence, novel simulation methods, and intelligence gathering.The model parameters that commonly appear in game-theoretic analyses may be estimated by applying empirical data, which can be continuously improved through compilation by statistics bureaus and other actors.
Future research should generate results that are less dependent on specific assumptions, to ensure robust results that are valid across a variety of different circumstances, applicable for policymakers and decision-makers.One challenge in this regard is the balance to be struck between generalizability, simplicity, and precision, where, usually, only two of these criteria can simultaneously be satisfied.
Future research should broaden the cyber focus within traffic and transportation, accounting for air, land, and sea and distinguishing between the transport of humans and goods, etc.Such research should be extended to the transport of information and communication, which increasingly occurs wirelessly.This opens up new attack vectors.Cyber security within centralized and decentralized blockchain technology should also be researched.The 19 July 2024 CrowdStrike Information Technology outage highlights the need to analyze how individuals, firms, governments, and others depend on software and hardware from various contributors and potentially interceptors, which may have benign or less benign objectives and may possess unknown competences in rapidly changing environments.
Whereas this review has focused on traffic and transportation and cyber security education and board games, cyber security will, in the future, play an increasing role also in other domains, such as finance, banking, insurance, healthcare, emergency services, energy, utilities, water and power supplies, telecommunications, governments, public sector agencies, military, aviation, supply chains, logistics, manufacturing, education, retail, e-commerce, media, entertainment, smart cities, and critical infrastructure.Examples of its roles are fraud detection and prevention, regulatory compliance, security related to property rights, data storage, privacy, transparency, transactions, network operations, cloud operation, and software/hardware updates.
Several research gaps exist from a game-theoretic perspective in the cyber security literature, which are promising for future research.

1.
Multiple objectives: Utility functions should be developed focusing on the worstand best-case scenarios, minimizing the costs, maximizing the benefits or security, weighing human vs. economic vs. symbolic value, and weighing multiple objectives against each other.2.
Incomplete information: Games should account for players being uncertain about their surroundings and the future, including other players' preferences and beliefs.

3.
Mixed strategies: Games should focus on players choosing strategies probabilistically.

4.
Stochastic games: Randomness should be incorporated into the players' strategies and their surroundings.

5.
The time dimension: Repeated and dynamic games should be developed accounting for new events and information, where adversaries react to each other in various sequences.

6.
Complexity: Models should account for increasingly complex cyber security challenges, develop more efficient solution methods, utilize increasingly available supercomputers to solve large-scale problems, and question the available strategies, utility combinations, and the games that players play.7.
Empirical support: The models should be tested experimentally and in real-life settings to ensure their realism, validation, and practical implementation.8.
Behavioral game theory: Theory and empirics should be combined to ensure the increased realism of economic, political, and social interactions, accounting for bounded rationality and risk attitudes.9.
Learning: How players learn in a novel field such as cyber security should be analyzed, accounting for the adaptation, reinforcement, and adjustment of strategies, preferences, and beliefs.10.Cooperative games: How players form coalitions to share costs and benefits and obtain cyber security should be scrutinized.11.Interdisciplinarity: Game theory should be combined with other disciplines within the technological, natural, social, and human sciences to obtain more holistic insights.Examples of disciplines include Internet of Things security, 5G and nextgeneration network security, artificial intelligence, machine learning, quantum computing, cryptography, blockchain and distributed ledger technology, zero-trust architectures, privacy-enhancing technologies, cyber-physical systems, user education and awareness, psychological profiling, advanced threat intelligence, and frameworks for regulation, compliance, adaptation, resilience, and recovery.

Conclusions
This article reviews attacker-defender games, which have a longer and broader history than the more recent and narrower phenomenon of cyber security, which inevitably involves attack and defense.Hence, this review starts with a strong focus on attack and defense models, multiple targets, multiple attackers and defenders, multiple periods, and various characteristics of defense and attack.The literature commonly considers one player defending or attacking one component in a system, multiple-attacker and/or multipledefender games, and multiple-period attacker-defender games.Defense and attack games have various characteristics.For example, they may involve security screening and inspection, the detection of invaders, and jamming.
Whereas the majority of attacker-defender games in the literature assumes complete information, as a transition to cyber security, the prominent role of incomplete information is reviewed, involving multiple targets, secrecy and deception, threat propagation, and trust and reputation.Information about the players' characteristics may be drawn probabilistically.Thereafter, information sharing is considered, followed by cyber stockpiling, deterrence, and resilience.The joint operation of information sharing and security investment is reviewed.Players may prefer to receive information from others but not provide information, which suggests a free-rider dilemma, unless a social planner is introduced.Firms may experience cyber attacks or breaches directly or indirectly through shared network ties or dependencies through supply chains.Firms may tend to underinvest in security and free-ride unless otherwise incentivized.Cyber attackers may produce cyber munitions for present use or stockpile zero-day cyber vulnerabilities for future use.Cyber security deterrence and resilience are considered.
The presence of incomplete information in cyber security makes a review of stochastic analyses relevant, acknowledging the many uncertainties and probabilities of states, strategies, and outcomes that are involved.Most of these consider one defender and one attacker.They cover a variety of phenomena, including intrusion detection, jamming, and eavesdropping.
The review concludes with two topics having attracted substantial attention in recent years.One is cyber security in traffic and transportation, partly influenced by the communication of GPS data between moving units such as self-driving cars.The second is cyber security education and board games, tabletop games, card games, and experimental games, to enable people to face cyber threats in real life.Such games also involve learning, training, and awareness and are influenced by the ubiquitous presence and risk of portable electronic devices in human life.Strengths, weaknesses, opportunities, and future research are considered.Funding: J. Zhuang's effort was partially supported by the U.S. Department of Homeland Security under Grant Award Numbers 24STADA00002 and 22STESE00001-03-04.The views and conclusions contained in this document are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of the U.S. Department of Homeland Security.Alpcan and Basar [80] One

Table 1 .
Overview of 12 earlier reviews.

Table 3 .
Stochastic approaches to cyber security.

Table 4 .
Cyber security games on traffic and transportation.

Table 5 .
Education and board games.