On small perturbations of Markov cyber threat models

In this work, we consider Markov chain-based stochastic modeling of cyber threats acting on computer systems. In the framework of this approach, computer systems are considered as systems with failures and recoveries by analogy with technical system models in reliability theory. Under the assumption that the cyber threats are independent random events, we derive the explicit analytic formulas for the state probabilities of the corresponding Markov chain and the mean time to security failure (MTSF). Then we investigate the case of dependent cyber threats and derive the approximate expressions for the state probabilities and MTSF within the framework of the first-order perturbation theory. As an illustration of our results, we consider a few simple examples.


Introduction
The development and improvement of modern information security systems is a time-consuming and complex process. Many stages of this process are still based not on sound scientific principles, but the professional experience of experts. To a great extent, this is because each protected information system is a unique one, and together with the large variety of modern information technologies it is difficult to develop unified approaches to building cybersecurity systems. However, despite the existing methodological difficulties, the attention of experts to theoretical research in the field of information security is steadily growing, and the range of mathematical models used in this process is constantly expanding. In such a case, probability-theoretic models of security, and in particular, Markov process models are of increased interest [1,2,3,4].
The paper [5] proposes a model of cyber threats formulated in terms of discrete-time Markov chains. In this model, a computer system exposed to cyber threats is described as a system with failures and recoveries, similar to models of technical systems in reliability theory. In the articles [6,7], the authors studied this model in more detail, indicating the possibility of its application to evaluating the effectiveness and optimization of the applied information security tools.
This paper continues the research by generalizing the original model for cyber threats which are dependent random events at different instants of time. In particular, considering the mutual generation of some cyber threats by other rare random events, we develop a relevant perturbation theory of the corresponding Markov chains. Based on this perturbation theory, we obtain approximate analytic formulas for the probabilities of Markov chain states and for the mean time to security failure. The paper concludes with several examples illustrating the obtained results.

Description of the original cyber threat model
In this section, we recall the main provisions of the original Markov model of cyber threats and some related results, following the works [5,6,7].
Let us consider a computer system (hereinafter simply a system) that is affected by n independent cyber threats. We accept the following assumptions.
1. Cyber threats affect the system only at discrete time instants: t = 1, 2, . . . . 2. Only one cyber threat can act at a given instant t.
3. If one of the cyber threats is active at some instant t, then the system tries to reflect it at the next instant t + 1.
According to the assumptions made, we can suppose that at any given instant the system is in one of the states s 0 , s 1 , . . . , s n , s f . In the state s 0 , called the security state, no cyber threats take effect. In the case when the i-th cyber threat is implemented, the system performs a transition from the state s 0 to the state s i where i = 1, 2, . . . , n. If the system is in the state s i at a given instant t, then there are two alternatives at the next instant t + 1: • the i-th cyber threat is eliminated with probability r i , and the system returns to the state s 0 ; • the i-th cyber threat is not eliminated with probabilityr i ≡ 1 − r i , and the system goes to the final state s f , which means a security failure.
It is clear that the probability of each state at an arbitrary instant depends only on the state reached at the previous instant of time. This means that the sequence of possible system states is a simple Markov chain. Its diagram is shown in Figure 1.  The dynamics of the system is described in terms of functions p i (t), which are the probabilities of states s i at a given instant t. These probabilities can be calculated by the formula where π ji is the probability of the transition from state s j to s i . The set of quantities π ji forms a transition matrix Π which in our case takes the form Here we have introduced the notation q 0 ≡ 1 − n i=1 q i . It is also natural to assume that the system is in the security state at the initial instant t = 0: The equality (1) together with the initial conditions (3) unambiguously determines the probabilities of the system states at a given instant t.
From (1) and (2) we can obtain the following linear homogeneous second-order recurrence relation for the probability p 0 (t): It is known from the general theory of linear recurrence relations that the explicit expression for p 0 (t), in this case, is in the form where λ 1 and λ 2 are the roots of the characteristic polynomial f (λ) = λ 2 − q 0 λ − i q i r i , and the constants c 1 and c 2 are defined using the initial conditions p 0 (0) = 1 and p 0 (1) = q 0 . Thus we obtain where w 2 = q 2 0 +4 n i=1 q i r i . The probabilities of the rest states are calculated by the probability p 0 (t): The most important characteristic of the system that reflects the effectiveness of defense mechanisms is the time to security failure, that is, the number of T transitions in the corresponding Markov chain before the the final state s f is first reached. It is clear that T is a discrete random variable that takes an infinite series of values: T = 2, 3, . . . . The probability distribution P (T ) of this random variable can be found by the formula (4): The normalization condition is checked directly: ∞ T =2 P (T ) = 1. For practical applications of this model, it is important to have specific numerical characteristics of a random variable T : its expected value and variance. In particular, the expected value τ ≡ M[T ] which means the mean time to security failure (MTSF), is defined as ITNT It is not difficult to see that the obtained formula is quite consistent with the expected results in some special cases. For example, if all q i are equal to zero or all r i are equal to one, MTSF becomes infinite. These limiting situations correspond to the case of total threat absence or the case of absolute protection, respectively.
We note that the resulting expression for τ can be used to evaluate the sufficiency of the used defense mechanisms in the context of information security management. In practice, for example, we can impose a condition τ ≥ τ cr , which means that MTSF should not be less than a certain critical value τ cr . Violation of this requirement means insufficient protection of the system and signals the need for additional means of cybersecurity.

Model perturbations: the case of dependent cyber threats
In real computer systems, cyber threats are rarely independent; often the appearance of one threat can generate a series of other cyber threats. For example, the threat of unauthorized access may lead to the threat of data leakage, and the threat of "denial of service" may lead to the threat of unauthorized use of computing resources. Based on the above, when modeling cyber threats, it is necessary to take into account the fact that for neighboring instants t and t + 1 they are dependent random events.
In this paper, we investigate a family of small perturbations of the Markov model described above, allowing for transitions between states s i , where i = 1, 2, . . . , n. To do this, we consider the Markov chain defined by the transition matrix Herer i = 1 − r i − n j=1 ij , and it is natural to require that 0 ≤ n j=1 ij ≤ 1 − r i for all i. The value ij makes sense of the transition probability from state s i to state s j , i.e. ij is the probability of the j-th cyber threat at the instant t, provided that the i-th cyber threat occurred at the previous instant t − 1. Just as above, we assume that the system is in the security state at the initial instant t = 0, that is, the equalities (3) hold.
The dynamics of the specified Markov process, as in the case of the original model, is defined by the recurrent relations (1). Its numerical analysis is not particularly complicated, but it is generally difficult to obtain exact analytic results here. We investigate, in this context, a particular situation when transitions between states s i , where i = 1, . . . , n, are rare random events. Mathematically, the specified requirement can be written as ij 1 for all i, j = 1, . . . , n.
In this case, we can obtain approximate analytic results, keeping to the first approximation in the coefficients ij .

Formulas for probabilities of states in the first approximation
Within this approximation, using relations (1) we obtain the following linear recurrence relation n j=1 q i r j ij when ij = 0 has three roots λ 1,2 = (q 0 ± w)/2 and, λ 3 = 0, which can be considered as zero approximations to the corresponding roots for arbitrary ij . In particular, in the first approximation, these roots have the form Thus, the formula expressing the general term of the recurrent sequence (9), up to the first-order terms in ij , has the form where the constants c 1 , c 2 , and c 3 are defined from the initial conditions p 0 (0) = 1, p 0 (1) = q 0 , and p 0 (2) = q 2 0 + n i=1 q i r i , and in the specified approximation are written as Hence, for the probability of the security state in the first approximation in ij we obtain the following expression: We recall that q 0 = 1 − n i=1 q i , and w 2 = q 2 0 + 4 n i=1 q i r i . The resulting formula (10) allows us to calculate the probabilities of the rest states of the system in the same approximation: It is obvious that if ij = 0, the obtained formulas go into expressions (4) and (5).

Calculation of MTSF in the first approximation
Now we obtain an expression for the mean time τ to security failure in the considered approximation.
If the system reaches the final state s f for the first time at the instant t = T , it means that it was in one of the states s 1 , s 2 , . . . , s n at the instant t = T − 1 (see the transition matrix (8)). Thus, for the probability distribution of the random variable T , we obtain The mean time to security failure τ is defined as the expected value of the random variable T . Substituting expressions (10) and (12) in the formula τ = ∞ T =2 T P (T ) and performing summation up to the first order terms in ij , we have It is not too difficult to see that with ij = 0 the resulting formula coincides with formula (7). It is obvious that the transition matrix for the given Markov chain takes the form

Some examples
The dependence of the probability of the security state on t, in this case, can be found exactly: On the other hand, according to formula (10) in the first approximation in , we obtain  where w 2 = q 2 0 + 4qr. To compare the approximate solution with the exact one, in Figure 3 graphs of the functions p i (t) and p * i (t) are shown for different sets of values q, r, and . The mean time to security failure in this example can also be found exactly. We give the exact and approximate expressions obtained by the formula (13): The corresponding remainder term ∆τ ≡ τ − τ * has the form From here, it is not difficult to get an estimate for , at which the error in calculating MTSF will not exceed the specified value σ > 0: In this example, the mean time to security failure τ can also be calculated exactly (the corresponding expression is cumbersome to write it out here). The approximate formula for this quantity according to (13) takes the form τ ≈ τ * ≡ 1 + q 1 (1 + 3 ) + q 2 (1 + 4 ) q 1 (1 − r 1 ) + q 2 (1 − r 2 ) + (1 + q 1 + q 2 )(q 1 r 1 + 2q 1 r 2 + 3q 2 r 1 + q 2 r 2 ) q 1 (1 − r 1 ) + q 2 (1 − r 2 ) . Table 1 shows the values τ and τ * for various values of the parameter at q 1 = 0.3, q 2 = 0.4, r 1 = 0.6, r 2 = 0.7, and the values of the corresponding relative error δτ ≡ |τ − τ * |/τ . The table illustrates that our perturbation theory gives a good estimate for τ at = 0.01, 0.02, 0.03. On the contrary, when = 0.04 and 0.05 the relative error becomes greater than 15%; in this case, the perturbation theory in the first approximation gives a rough estimate and it is necessary to either involve higher-order terms, or use other approximate methods to find τ .

Conclusions
In the study, we have examined a stochastic modelling approach to describing computer systems affected by cyber threats. The computer systems are considered as systems with failures and recoveries by analogy with models of reliability theory. In the case of independent cyber threats, we derive explicit formulas for the probabilities of system states and for the so-called mean time to security failure (MTSF), the time which the system spends to arrive the failure security state. Then we have considered the case when some cyber threats can give rise to other threats. Assuming that such events are rare, we developed the corresponding perturbation theory, which provides the evaluations for the state probabilities and the mean time to security failure in the first-order a pproximation. F inally, we g ave s ome e xamples t hat i llustrate o ur results.
The future investigations will focus on developing the perturbation theory of a higher order approximation. We also intend to examine the convergence problem of this perturbation theory.

Acknowledgements
The reported study was funded by RFBR, project number 19-37-90122.