PDF-based tuning of stochastic optimal controller design for cyber-physical systems with uncertain delay dynamics

: Uncertain dynamics in communication network, including random delays and packet losses make it difficult to guarantee stability of cyber-physical systems (CPSs). Many existing works consider the uncertainties of network channel with strong assumptions that network delay bounds and its distribution are known a priori and time-invariant. However, these assumptions could be invalidated in realistic CPSs by malicious attacks, system hardware faults, topology changes etc. A probability density function (PDF)-based tuning of stochastic optimal control (PTSOC) is proposed to manage the unknown dynamics in the embedded network. The update law of the proposed controller is derived and updated based on the PDF estimation of network delays that explicitly consider delays and its time-varying distribution. The results illustrate that the proposed PTSOC has a better performance in terms of the overshoot, convergence time, and cost when compared with the conventional stochastic optimal control.


Introduction
Modern industrial systems, such as smart grid, healthcare and automotive control systems, are implemented as distributed event-triggered control systems wherein the control loops are connected by a real-time communication network with limited resources. Such systems are referred to cyber-physical systems (CPSs) which offer many advantages: the ease of maintenance and installation, flexibility, and low cost. However, limited resources and constraints of the embedded communication network create challenges for a control system stability. For example, changing topology or background traffic result in varied delays, packet losses, and quantisation over time. In addition, cyber-attacks including denial-of-service, spoofing, and eavesdropping can degrade the system performance. For example, delays exceeding the range assumed for the controller due to jamming cyber-attack lead to the control signal arriving too late for the actuator to take appropriate actions. Consequently, system outputs deviate from the desired trajectory and the entire CPS becomes unstable. Therefore, an optimal networked resilience controller design for CPSs is needed. It has to mitigate the negative effects of the network uncertainties and dynamics on the CPS performance.
Many existing works on cyber security [1][2][3][4] proposed network schemes to defend against the cyber-attacks without modification to the controller design. In contrast, other researchers focused on fault tolerant controller design to improve resilience against physical component faults [5][6][7][8][9][10]. In these works, the network dynamics and uncertainties are often oversimplified under strong assumptions: (i) the delays are bounded within a specific range and (ii) the distribution of delays and package losses are known and time-invariant. However, in realistic CPSs, the delay can easily exceed such restrictive bounds and lead to unstable system. The unexpected variation in delay leads to the malfunction or overreaction on the controller and actuator, and eventually instability of the entire CPSs. Therefore, relaxing these assumptions should be addressed to improve the resilience of CPSs.
In this paper, a PDF-based tuning of stochastic optimal controller (PTSOC) is developed to address the degraded system performance induced by network uncertainties including attacks, transmission faults, and channel dynamics. This relaxes the earlier mentioned strong assumptions (i) and (ii) made in existing works. A stochastic system model, which includes stochastic parameters that represents network dynamics, is employed. A kernel density estimation (KDE)-based online PDF identifier is proposed to capture the variation of network dynamics. The probabilities of delays provided by the PDF identifier are used to tune the control law. In addition, the system stability is mathematically analysed using Lyapunov approach. Overall, the proposed approach improves the robustness, optimises cost of regulation, prevents the physical components from unrepairable damages, and keeps the CPS working within the desired operating condition. This paper is organised as follows: In Section 2, the related works and motivation are briefly discussed. The proposed controller design in terms of PDF identifier and stability analysis are presented in Section 4. Section 6 illustrates the effectiveness of the proposed controller through simulations in MATLAB. Section 7 gives the conclusion.

Motivation example
The embedded cyberspace imposes restrictions on the exchange of information in CPSs, such a limited channel capacity and traffic congestions. Malicious cyber-attacks further restrict the information delivery. The delay and packet loss caused by the above restrictions are stochastic with unknown bounds and difficult to predict. Importantly, they have a potentially negative effect on the performance and stability of CPSs. For example, inappropriate control actions caused by increasing delay lead to a big overshoot as well as more actuation cost to manage such overshoot.
A following example shows that the network dynamics significantly affect the performance of CPS and the existing control approaches may fail to guarantee its stability.
This scenario emulates a route hijacking by an attacker to eavesdrop control information (data integrity). Such a route hijacking increases delay and delay variation on the longer path. A network is simulated using network simulator 2 with a random topology of 11 nodes. Ad hoc on-demand distance vector routing scheme is adopted. The route through the topology is altered during the simulation such that the packet delays vary for the controller loop, as shown in Fig. 1a. Note that similar network performance could be a result of topology or traffic pattern changes.
The results show the disturbance in CPS introduced by the network dynamics. With these delays, a proportional-integralderivative controller (PID) controller is simulated for a simple 2I4O (two input four output) system [11]. Fig. 1b shows PID make the system states converge when delay bound is low before 6 s. Then, the sudden changes of delay at k = 6s, 16s, and 20s make the system states vibrate. Consequently, the CPS becomes unstable due to such dynamics of delay.
The attack changes the delay distribution. This stochastic disturbance increases the probability that the system becomes uncontrollable and unstable. Therefore, it is necessary to include such dynamics both in system model and controller design.

Related works
In this section, first, some existing works on control schemes for CPSs are introduced. Then, some methodologies for capturing network dynamics are discussed to illustrate that KDE is suitable to be used in the proposed control scheme. At last, the main contributions of our work are highlighted and demonstrated.

Control design for CPS
The following literature review discusses the existing approaches that address the uncertain dynamics of network, especially long-time delays and packet losses, either in network or physical system.
Many researchers developed network protocols and tools [1][2][3][4]to keep delays and packet losses within traditional controller constraints. The controller design of physical systems is not modified in these works. Such approaches can maintain system stability if the network configuration is simple and fixed. Designing more complicated networks and systems under cyber-attacks becomes challenging if not impossible due to restrictive constraints dictated by the physical controller. Xie [1] proposed a channel estimation approach to calculate the packet success rate, the worst-case packet delay and average energy consumption based on acknowledgement information. However, the worst-case delay has a upper bound known a priori while the system model is linear time-invariant and not stochastic. Lee et al. [2] proposed a quality-of-service-based remote control scheme for CPSs via the Profibus token passing protocol. They used the time delay data provided by transmitting the real-time messages to approximate the delays for high-and low-priority messages. They assumed a strong condition that the network delay has time-invariant lower and upper bounds. The control schemes in above literature are valid if the delays are bounded within the acceptable range. However, if the configuration of the embedded network becomes complicated, dynamic, and stochastic, the bound constraints cannot be met. For instant, in intelligent transportation systems, moving vehicles exchange their information with other vehicles, transportation management system, and users. Such a complicated and dynamic network results in delays and packet losses exceeding their bounds because of interference from the environment and network topology changes. Thus the controller cannot guarantee the stability of CPSs. Overall, a more robust approach should consider network dynamics in controller design to relax the constraints.
Simultaneously, other researchers had designed several controllers to address stochastic network dynamics under strong constrains. Gao et al. [5,6] modelled a CPS as a sampled-data system and solved a set of linear matrix inequalities to derive the feedback gain of a memory-less controller. The dynamics in the cyberspace are simplified as known bounded delays. Similarly, Hao et al. [7], Tian et al. [8] and Liu [9] proposed networked controllers by using Lyapunov stability analysis with a priori knowledge of the delay bounds. Moreover, Tiberi [10] proposed a self-triggered sampling for achieving substantial reduction of communication traffic. The system stability can be guaranteed under certain assumptions: (i) the measurements are sent to the central node within a bounded time delay and (ii) the system states have to converge initially. All these works considered the network delay issues in various perspective. However, they commonly assumed the network delay had a known bound and the distribution of delay was fixed. Such schemes would fail in realistic CPSs where delay bound is unknown before hand. For instance, new communication nodes generating new traffics will increase the upper bound which cannot be known ahead of time. Furthermore, dynamics of the delay distribution make the entire system unstable.
Xu et al. [11] made a significant progress when they included the random parameters representing network dynamics in system model. All of system matrices and control input calculation concurrently change as network delays and packet losses randomly change. This model is the prototype we used in the proposed PTSOC design.
The conventional discrete time model was described as [12] x where .., d are the system matrices. T s is sampling interval. A is the system matrix of the continuous-time model. Xu [11] derived the stochastic model expressed as . . .

00
·· are the time-varying system matrices [11]; A s and B k i are the system matrices calculated by (1). u k is the control input; g k i and g k−i are binary random variables representing the package reception status (if the package is received, g = 1, otherwise, g = 0). I m is an identity matrix.
Then, the stochastic optimal control (SOC) law can be obtained by optimised the following cost function.
where Q z = diag{Q, R/d, ...}, and R z = R/d are symmetric positive semi-definite and symmetric positive definite, respectively. z m is the state variables vector, and u m is the control inputs vector.
. Although the above model included the terms of network dynamics, the SOC only indirectly considers such dynamics through the stochastic model. It also has a strong assumption that the delay distribution is fixed. System performance becomes suboptimal when the above assumptions are invalidated by changes in bounds or distribution as observed in simulations for cases B and C in Section 6.

Capturing network dynamics
To relax the bounds and distribution constraints, the system model and the control law should be tuned based on current probabilities of delays. Therefore, a PDF estimation is needed.
For estimation of delay probabilities, several PDF identification methods exist in literature, such as histogram, KDE, and maximum likelihood estimation [13]. The advantage of KDE is more accurate estimation with fewer samples than other methods. Offline KDE has been used in various applications, including computer graphics [14], image processing [12,[15][16][17], and industrial process [18,19]. He et al. [18] introduced a novel KDE-based framework for non-linear metric learning, Kernel density metric learning. This method has been successfully applied in face recognition. Elgammal [17] investigated the use of fast Gauss transform for efficient computation of KDE techniques for computer vision applications. Chen [19] proposed a KDE-based method to estimate the spatial intensity of false alarms for multi-target tracking system. Additionally, KDE is suitable [13] for PDF identification in CPS because it can handle different types of distributions, such as mixture distribution and Poisson distribution. In contrast, the other approaches only work for normal distributions. In Section 4.2, an online KDE estimator is introduced. It can estimate unknown PDF with a good accuracy and adapt to the dynamic changes of delay distribution. Then, the provided PDF information is used to tune the PTSOC control law. Hence, the controller adapts to the given network dynamics.
In this paper, we proposed PTSOC to relax the bounds and distribution constraints in the existing works. The PDF variation of network delays can be captured by the KDE-based PDF identifier. Also, the PDF information is used to tuning the optimal control law. Thus, the resilience of the entire CPS is improved. Next, the details of the proposed PTSOC are presented.

Proposed PTSOC design
In this section, the overview of the proposed control scheme is given. The PTSOC scheme with an online PDF estimator is introduced. The online KDE-based PDF identifier is introduced in Section 4.2. Then, PTSOC is derived from the proposed cost function 4 in Section 4.3.

Overview
PTSOC takes into account uncertain network dynamics by applying online KDE to capture PDF variation of delays and tuning its control law based on the PDF information. The overall architecture of the proposed control system is shown in Fig. 2. The proposed PTSOC includes three main steps that are continuously repeated: (a) Data collection of delays: n delays ([d k−n+1 , ..., d k ]) in the sliding window are used to do the PDF estimation at time k (PDF k ). When new delay is measured, the data in the sliding window is updated. (b) PDF estimation: The PDF of these n delays is obtained. The probability for each delay interval is calculated. (c) PTSOC law calculation: For each delay interval, the cost function (5) and the optimal control law (8) are derived from (3). The cost function of PTSOC defined by (4) is a probability weighted sum of the cost functions for all delay intervals. Similarly, the PTSOC law derived in Section. 4.3 is the summation of the weighted SOC laws of delay intervals. These weights are probabilities from the delay PDF. The final cost function is where d int represents the delay interval that we take 0.1 s in the simulation section.
n is the total number of delay cases; k represents sampling interval; P i is probability of delay within d int i to d int (i + 1) provided by the KDE-based PDF identifier; x is states vector; u i is control inputs vector; Q i and R i are weighted constants of states and control inputs, respectively.

Remark 1:
The cost function of PTSOC is more generalised than that of SOC. SOC only considers the worst delay case that is d int , d k , d int (i + 1), P i = 1, and P j = 0, j = i. In such a case, the PTSOC cost function (7) becomes SOC (3).
As the probabilities of each possible delay changes, PTSOC continuously tracks the network dynamics with a PDF identifier and updates its parameters based on PDF information of delay to adapt to the given system situation.

PDF identifier
To capture the dynamics of network delays, an online KDE-based PDF identifier is proposed to iteratively estimate the distribution. T h ed a t au s e dt od oi d e n t i fication is updated every sampling interval for a window of n last packet delays. The main steps of online PDF identification are shown in Table 1. n delays are loaded in the sliding window first. Then, KDE algorithm is applied to estimate the probability of each delay. The overall PDF of this finite data set is obtained with selecting a normal kernel smoother here. Finally, the estimated PDF is used to tune control law and the new delay is loaded in the sliding window. The PDF is updated in an online manner to capture the variation of delay distribution.

Remark 2:
With a large window size, KDE provides an accurate estimation of PDF with neglectable bias [20]. In this paper, we estimate PDF of n samples in the sliding window. Such that the estimation error is guaranteed to converge to a small value denoted as d KDE . The effect of d KDE on the regulation error convergence is shown in Theorem 3.

Optimal controller design with consideration of dynamics of delay distribution
In this section, PDF identification is employed to develop a novel SOC for CPSs while considering stochastic dynamics of network. First, we derived time-varying system matrices that explicitly include probability information of delays. Each element of the matrices is derived. Then, PTSOC is proposed based on the time-driven sensing and event-driven actuating controller framework. The maximum number of control input that affects system matrices is d.
Only the latest control input is allowed to act on the controlled plant when several control inputs are received at the same time [11].
In the stochastic model (2), all of the candidate models (dynamic matrices A zki and B zki ) and their corresponding probabilities depend on the g k i values and their probabilities. g k 0 corresponds to control input with the delay less than sampling period (T s ). g k i (k = 1, 2, d) is determined by g k−1 i−1 . Therefore, g k i depends on g k−1 0 and g k− i . The probability P g k i can be determined by P d k .T s . Equation (5) denotes the probability for each g i . (see (5)) where All of the probabilities in (5) can be obtained from the PDF identifier. The data in the sliding window maps the PDF and provide the probabilities of delays in different ranges ). The probability of delays can be denoted as P(g k 0 = i) and P(g k i = i). Then, it is easy to obtain the probabilities of each possible B k i based on the above information. Finally, a set of candidate stochastic models A zki and B zki are defined. The corresponding control law (candidate control law) for each candidate model (6) is obtained by optimising its corresponding cost function (7). Each candidate cost function has its optimal parameters Q zi and R zi for the corresponding control law derivation. The final proposed control law (8) is a weighted sum of these candidate control laws, with weights that are equal to the corresponding probabilities from the PDF identifier. (see (6)) . . . (a) Choosing a kernel function K centred on t with a bandwidth h; (b) Each observation t i receives a specific weight proportional to the scaled distance from the observation t i to t, which is u = (t − t i )/h; (c) At a given t, the estimate is found by vertically summing up over the k shapes.
This can be synthesised as The general formula for KDE will be given by where the dependence of the estimate on the kernel function K (.) is denoted as f k . 2. Updating the new data for time k + 1 in the sliding window and go back to Step 1; where K ik is the optimal gain when d int i , d k , d int (i + 1) at k, J i is the corresponding cost function. Finally, the control law of the proposed PTSOC is derived as where K k is the optimal gain and u k is the control input. S zik ≥ 0i s the solution to the ARE. n d = d upper /d int , d upper is the maximum delay in the sliding window; P ki is the probability of d int i , d k , d int (i + 1). PTSOC, which considers PDF of delays, is able to optimise the trade-off between system state error and cost of regulation. Its control law is more accurate for the given network situation than that of SOC because the weights of each candidate control law are continuously updated to reflect the actual PDF of delays. In contrast, SOC considers only one of delay cases (7) in optimisation.

Stability analysis
Three theorems and their corresponding proofs are presented to demonstrate stability of the proposed PTSOC. Lyapunov-based stability analysis is used. Theorems 1 and 2 demonstrate asymptotic convergence of regulation error and estimation error of the control gain. Theorem 3 shows uniformly ultimately bounded (UUB) stability of the regulation error when the irremovable bias of PDF estimation exists.
In Theorem 1, the control law tuned by probability information guarantees the asymptotic convergence for regulation error with an assumption that delay PDFs are accurately estimated without bias. Theorem 2 relaxes the assumption in Theorem 1. It shows the control gain estimation asymptotically converges even if PDF estimation has an error provided it asymptotically converges to zero. Theorem 3 considers the irremovable bias of PDF estimation as a bounded disturbance. However, an UUB stability is guaranteed.
Theorem 1: (Asymptotic stability of regulation error with a perfect PDF estimation). Given the initial conditions as the system state z 0 and system matrices A z0 , and B z0 , let u 0 (z k ) be an initial admissible control policy for the CPS (1). Let the control update law be given by (8) with properly selected Q zm and R zm . Let the PDF estimation be a no error estimation. Then, there exists a constant K max satisfying K max ≤ (1 − a)/b such that the regulation error of system states converges to zero asymptotically in the mean.
Proof: Consider the following positive definite Lyapunov function candidate: V zk = z T k z k . z k is the state vector of k. The system matrices are time-varying and stochastic; therefore, we consider DV z km = V z k+1 m − V z k m for each possible system matrices (A zkm and B zkm ). m represents the number of the candidate systems. If the maximum value of DV z k m is negative definite, the convergence of system states is proved.
With Cauchy-Schwarz inequality, where A zj and B zj are the system matrices of j. K j is the control gain of j.
.., n d is the maximum control gain of k. n d is the number of system matrices case.
We define a = max a 1 , a 2 , ..., a n d and b = max b 1 , b 2 , ..., b n d . Q zm and R zm are selected properly [21]. Since V z k m is positive definite and DV z k m is negative definite provided K max is selected as above. Therefore, the regulation error converges to zero asymptotically. □ Next, the assumption that PDF estimation has no bias is relaxed with Theorem 2. The control gain estimation error still has an asymptotic convergence.
Theorem 2 (Control gain estimation error convergence): As the delay data keeps updating PDF identifier and ( n j=1 P (k+1)j − n j=1 P kj ) , 0 is satisfied, then the estimation error for control gain K k asymptotically converges to zero.
Proof: First, we define the estimation error of control gain K as K k = K k − K k = n j=1 P kj K j − n j=1 P kj K j . P ij is the actual probability at k. Then, Lyapunov function candidate is

see equation at the bottom of next page)
Since V K k is the positive definite and DV K k is the negative definite which provide K k = K k − K k = n j=1 P kj K j − n j=1 P kj K j . Therefore, the estimation error of control gain asymptotically converges to zero. □ Remark 3: Monotonically decreased estimation error for each delay range ( P (k+1)j − P kj , 0) is not a necessary condition. The convergence only requires the estimation error of the entire PDF (( n j=1 P (k+1)j − n j=1 P kj ) , 0) monotonically decreasing over time. The maximum error occurs when the first sample of the new distribution comes in the sliding window. Then the accuracy of PDF estimation improves as the sliding window includes more and more new samples from the new distribution after the PDF change occurs. Therefore, ( n j=1 P (k+1)j − n j=1 P kj ) , 0 holds. Theorem 2 considers an ideal case where the PDF estimation error converges to zero. In realistic case, there is an irremovable bias due to the finite sliding window size. Theorem 3 shows the UUB convergence of the regulation error in such a case.
Theorem 3 (UUB stability of the regulation error): Given the initial conditions as the system state z 0 and system matrices A z0 , and B z0 , let u 0 (z k ) be an initial admissible control policy for the CPS (1). Let the control update law be given by (8) and if the disturbance induced by the irremovable bias of PDF estimation has a bound d KDE and K min , 1/b min such that the regulation error of system states has an uniformly ultimate bounded convergence in the mean.
Proof: Consider the following positive definite Lyapunov function candidate: where Since V z k is the positive definite and D V K k is the negative definite which provide the system states z k ≥ ( d M + A max )/(1 − b min K min ) and K min , 1/b min . Therefore, UUB stability of the regulation error is proved. □

Simulations
In this section, the proposed PTSOC is evaluated and shown to be better than SOC [11]. The metrics of regulation performance are overshoot and convergence time when the delays of network randomly changes. Three scenarios A, B, and C are simulated. In Case A, the delays satisfy the bound constrains [11]. It illustrates that PTSOC can obtain as good performance as that of SOC. Case B illustrates that the system performance are significantly  improved by employing PTSOC, even though the bound changes over time due to network changes or cyber-attacks. In realistic CPSs, topology variation and cyber-attacks bring more dynamics and uncertainties to the embedded network and entire CPS. Therefore, the bounds and constrains of delays made in ideal case are relaxed in Case C. This illustrates that PTSOC has a better performance in terms of overshoot, convergence time, and cost of regulation than that of the conventional SOC when the delay bound and PDF are time-varying. Simulated benchmark example: The simulations employ the continuous-time model of a batch reactor system [11,13,14] The parameters of this CPS are selected as (i) the sampling time T s = 100 ms and (ii) d = 2, while the delays are not bounded. The continuous-time model is converted to a discrete time model with stochastic parameters representing network dynamics [11].

Case A: bounded random delays with a fixed PDF
PTSOC is evaluated for an ideal case with delays that satisfy the a priori set bound constrains of SOC [11]. The delays follow a normal distribution d ≏ N (0.15, 0.05 2 ). The upper bound is 0.2 s. Overshoot, convergence time, and cost of regulation are the primary metrics to evaluate SOC and PTSOC. The tests are repeated 50 times for the statistical validation.
In Figs. 3, 4 and Table 2, the performance of SOC and PTSOC are compared. Both of these controllers make the errors converge. PTSOC presents an opposite improvement with high percentages but closed values. This illustrates that SOC is optimal when the bounds and distribution of delays are satisfied the assumed constraints. Similarly, a longer convergence time of PTSOC is induced because the sliding window needs several iterations to converge the PDF estimation. Importantly, a significantly reduced cost by 92.1% is conspicuous. It demonstrates that PTSOC tuned by PDF of delays can optimise the trade-off between regulation error and cost. Overall, SOC and PTSOC have good and comparable performance when the delays are less than 0.2 s.

Remark 4:
Tuning the Q and R of PTSOC cost function can improve its performance in terms of overshoot and convergence time at the expense of a higher actuation cost. However, increasing the cost can lead to inefficient actuation and operating the actuators beyond its preferred range which leads to excessive wear and damages to actuators and the system.

Case B: increased random delays with a fixed PDF
In this case, the delays increase beyond the initially set bounds of SOC while maintaining a fixed PDF distribution d ≏ (0.3, 0.1 2 ). Such a case aims to demonstrate PTSOC still guarantee a good performance with delays with an unknown bound, while SOC performance degrades in terms of overshoot, convergence time, and the cost of regulation.
The performance of SOC and PTSOC are presented in Figs. 5 and 6. PTSOC has a significant improved performance when compared with SOC, as shown in Table 3. Its overshoot is reduced at least by 36.4%. The convergence time is reduced by 7.1%. Moreover, the cost of regulation is reduced by 94.9% thus the control inputs implemented by the actuators are within their preferred ranges. Such that the physical components are prevented from excessive wear and damages. Overall, SOC no longer guarantees the stability if the delay is over its upper bound. In contrast, PTSOC uses PDF information to tune the control law such that the performance and stability of the entire CPS is guaranteed.

Remark 5:
In this case, the initial conditions are same as that in Case A. The initial control law of SOC is pre-tuned for the smaller delay bound. Consequently, SOC control law is selected inappropriately for the case with longer delays. Hence, for the first second, the system states have large deviations and overshoots as shown in Fig. 5. Over time, SOC updates the system model and allows it to converge. In contrast, PTSOC can tune its control laws based on  both dynamic model changes and the drift of PDF of delays. Such that PTSOC quicker adapts to the uncertainties on system model and network delay.

Case C: high variation delay with time-varying PDF
Next, the distribution of delays is allowed to change over time. First, it follows a normal distribution with 0.45 s mean value. Then, the mean value jumps to 0.2 s at randomly selected time within [40 s, 50 s]. The simulations are repeated to obtain statistically valid comparisons. The results firmly indicate that tracking of PDF changes is critical in both system modelling and optimal controller design. Fig. 7 shows one specific case when the distribution changed at time 47 s. The parameters Q z = 0.15 × I 8×8 and R z = I are set for the SOC. The simulations show that the controller can make the regulation error converge. However, the errors increase right after the change in PDF of delays. Additionally, the input command to actuator increases. Such large inputs often exceed the preferred operating range for actuators thus increasing wear and tear or saturating actuator response.
In Fig. 8, the performance of PTSOC is presented. PTSOC convergence is sped up by using the updated PDF information. Hence, a shorter convergence time and a lower cost of regulation (Table 4) are observed.
Overall, SOC makes the error converge with a significant overshoot and a high actuation cost. Instead, the overshoot and cost of PTSOC are significantly reduced by several orders of magnitude. Table 4 shows the average value for 50 simulations. As the weights of each control input are updated continuously in terms of cost function, PTSOC has a better performance on overshoot (reduced by 80%) and cost of regulation (reduced by 96.1%). A cost function with the optimal parameters (Q and R) selection not only guarantees that algebraic Riccati equation has a finite solution but also reduces the control input cost. The convergence time of PTSOC is improved by 17.3%. Overall, PTSOC significantly improves the system performance in terms of overshoot, convergence time, and cost for every simulated scenario. Therefore, PTSOC indeed strengthens the resilience of the entire CPS.
Remark 6: In this case, the delay PDF is kept constant for at least 40 s to allow both PTSOC and SOC to achieve initial convergence. With such 'pre-training' process, the physical system model (1) is accurately tuned. When the change of PDF occurs, only the network dynamic terms in the model (2) are updated. Thus, both SOC and PTSOC perform better on overshoots than that for Case B (Table 3).   In this work, PTSOC is proposed to address uncertainties of the embedded cyberspace, particularly delays with an unknown bound and time-varying distribution. The simulation results show that PTSOC has a reduced overshoot (by 80%), convergence time (by 17.3%), and cost of regulation (by 96%) over that of the SOC. Continuous update of the probability weights speeds up convergence, optimises control input selection, and make control law adapt to delays with unknown bound and time-varying distribution. It facilitates the trade-off between system states and cost of regulation. Additionally, the constrains of known bound of delays and fixed distribution in existing works are relaxed.
Overall, reliability of CPSs in terms of resilience is improved.