Risk Scenario Generation Based on Importance Measure Analysis

Abstract

A risk scenario is a combination of risk events that may result in system failure. Risk scenario analysis is an important part of system risk assessment and avoidance. In engineering activity-based systems, important risk scenarios are related to important events. Critical activities, meanwhile, mean risk events that may result in system failure. This article proposes these definitions of risk event and risk scenario based on the characteristics of risk in engineering activity-based systems. Under the proposed definitions, a risk scenario framework generated based on importance measure analysis is given, in which critical activities analysis, risk event identification, and risk scenario generation are the three main parts. Important risk events are identified according to activities’ uncertain importance measure and important risk scenarios are generated on the basis of a system’s critical activities analysis. In the risk scenario generation process based on importance analysis, the importance degrees of network activities are ranked to identify the subject of risk events, so that risk scenarios can be combined and generated by risk events and the importance of scenarios is analyzed. Critical activities are analyzed by Taguchi tolerance design, mathematical analysis, and Monte Carlo simulation methods. Then the degrees of uncertain importance measure of activities are solved by the three methods and these results are compared. The comparison results in the example show that the proposed method of uncertain importance measure is very effective for distinguishing the importance level of activities in systems. The calculation and simulation results also verify that the risk events composed of critical activities can generate risk scenarios.

1. Introduction

Risk analysis and assessment are an important part of risk management, which plays an important role in production, supply chain, energy use, and other fields related to social sustainability and economics [1,2,3]. When assessing system risks, one of the primary tasks is to identify the possible risk scenarios. The identification and generation of risk scenarios in system risk assessment are not only important steps but the basis of risk avoidance as well [4,5]. System risk analyzers and managers can determine the circumstances that will cause risk consequences, and figure out how to avoid risk scenarios.

A scenario has different meanings and uses in different fields. For example, a scenario is typically used for system requirements acquisition, system design, and evaluation or testing in the field of computers and industrial design, described as predictable interactions between different types of users (roles) and systems, including objectives, expectations, motivations, actions, and response information [6,7]. In the field of reliability and risk assessment, a risk scenario is described as the combinations of events that could lead to product failure (fault) in order to analyze the failure (fault or accident) mode and probability of products (or systems) [8]. One view is that a scenario is the set of events that occurs in a certain order [9]; another view is that a scenario refers to a group of stochastic events or some specific combination of events [10]. In the field of project management and risk analysis, a risk scenario is considered as an idea or description of every possible future plan or situation, resulting from rigorous reasoning [11,12]. Although the understanding of scenarios varies between different fields, there is indeed a common feature among all definitions in set theory terms—that is, the scenario is composed of many system trajectories. Those system trajectories, which have some common features, are combined into a same scenario precisely because of these features. The scenario is surely the common description of those system trajectories [8,13,14].

Advance analysis based on the definition of risk scenarios is of great importance in the reliability and risk assessment field as well as in project management and risk analysis, where it is used for risk scenario generation, occurrence probability, and importance degree assessment. In the field of reliability and risk assessment, there are a lot of PRA (Probabilistic Risk Assessment) methods to help with identifying system risk scenarios [15,16], such as MLD (Master Logic Diagram), FMEA (Failure Modes and Effects Analysis), HAZOP (Hazard and Operations Analysis), ET (Event Tree), ESD (Event Sequence Diagram), FT (Fault Tree), AFD (Anticipatory Failure Determination), and so on, which can be unified into a framework called “TSS (Theory of Scenario Structuring)” [13]. In the field of project management and risk analysis, the SA (Scenario Analysis) method has been attracting more and more attention as an important risk identification method [17]. The SA method is presented to conceive a variety of possible future plans according to detailed and rigorous future reasoning on the basis of various key assumptions on economic, industrial, and technological evolution [18]. The greatest advantage is that decision-makers can predict future tendencies and avoid a common mistake: under- or overestimating the coming change and its impacts. Some industries have gradually adopted scenario-based analysis methods on risk prediction; for instance, Shell, the first industry to apply the SA method to great success, has a number of large projects underway at present [19].

In the field of reliability and risk assessment, although there are already many ways to identify and generate scenarios and these methods have been widely used (for example, in some PRA methods quantitative probability assessment is built on the basis of system risk scenarios, problems such as depending on personnel quality and heavy identifying works exist in risk scenario identification. Therefore, a growing number of studies on risk scenario generation and simulation have appeared in recent years [8,20], particularly those based on the IM (Importance Measure) method to identify important risk scenarios. As one of the important aspects on PRA, IM analysis, filtering out less important risk scenarios and then allowing risk analysts to assess the risk scenarios that can cause serious consequences, has significant guiding value for improving and optimizing system design, maintenance development, test strategy, etc. In order to determine the importance of risk scenarios, a variety of importance indexes measuring system components have come into being in recent studies. The most commonly used indexes are Birnbaum measure, FCI (Failure Critical Index), the Fussell‒Vesely measure, RAW (Risk Achievement Worth), RRW (Risk Reduction Worth), etc. [10]. In addition, there are derivatives of a parameter in system risk, such as the likelihood ratio gradient [21]. Moreover, research on the UIM (Uncertain Importance Measure) method of parameters combined with the PRA method in reliability models [22,23,24,25] is a priority in the field of reliability and safety analysis. Meanwhile, other UIM indexes are researched from different prospectives in reliability engineering, such as time-independent, cost-based, etc. [26,27,28,29,30].

Similarly, many qualitative methods are proposed for system risk identification in the field of project management and risk analysis, such as Check List, Specialists Method, Flow Chart, Analogy Comparison, experiential learning, etc. In addition, the SA method is used more and more widely, but how to identify the most important risks that will have a significant impact on systems, especially important risk scenario identification, has also been a focus for researchers. Generally, events that are produced by critical activities in system networks are critical risk events. To find the critical risk events, many importance measure indexes are proposed to measure critical activities in recent studies, for example the index PCI (Path Criticality Index) and ACI (Activity Criticality Index) for computing critical path and critical activities probability [31], the index SI (Significance Index) for computing the significance of each activity and activities’ critical measurement [32,33], and so on. Furthermore, in terms of engineering system risk analysis and sensitivity analysis, the number of studies on activities’ uncertain importance measure have increased for measuring the importance of critical activities and inspecting the impact on whole-system parameter uncertainty [31,34,35,36].

Engineering projects have a failure or fault mode—that is, if the project quality does not meet the user’s requirements, the project schedule is seriously overdue and the project cost is over budget [37,38]. Similarly, as an essential part of the whole system, activities of different importance have different influence on the whole project’s system risk. Meanwhile, for maintenance and logistic systems in the reliability and risk assessment field, the schedule and cost also need to be taken into account when assessing the importance of activities in the system [39,40]. From this perspective, importance measurement of activities and importance evaluation of parts in the reliability field are as important when analyzing system risk. Dividing risk categories effectively, identifying those risks and mutual relations, and finding critical risks are the main tasks in engineering system risk management, and provide a scientific basis for focusing on the disposal of risks. Risk scenarios are significant for the management and control of engineering system risk. Therefore, identifying and generating important risk scenarios in advance has become a scenario-based critical risk analysis problem.

This article belongs to two crossover fields: the project management field and the reliability and system risk assessment field. Based on the second view of risk scenarios [8,10], it is considered a combination of events that could lead to system failure, which usually includes time, cost, and quality dimensions. This article is mainly about identifying risk events from a system’s critical activities and combining the important events to generate risk scenarios by computing the importance measure of risk events. With reference to the UMI index in the field of reliability and risk assessment [29,39], we proposed a set of important measurement indexes to identify and generate risk scenarios. Meanwhile, three critical activities analysis methods, Taguchi tolerance design, mathematical analysis and Monte Carlo simulation methods, are applied to GERT (Graphical Evaluation and Review Technique) networks, used to model activity-based systems such as engineering project systems or maintenance and logistic systems [40], to determine activities’ uncertain importance measure by focusing on mathematical analysis and Monte Carlo simulation. A specific case is proposed to compare the three methods’ advantages and disadvantages for any guide of system risk scenario analysis and risk assessment. The results show that the importance measure analysis approach can be used to generate risk scenarios and assess system risk.

2. Risk Scenarios Generation

2.1. Definition of a Risk Scenario

In system risk analysis, possible causes of risk are often identified and assessed, and those risk events that may lead to system failure constitute risk scenarios. Usually the basic unit to describe a project system is an activity. The definitions of a risk event and a risk scenario are as follows.

Definition 1.

Risk Event. A risk event is an event that may cause system failure, described as the completion of a system activity in schedule, cost or quality. This can be expressed as:

$$E=\left(A_i;eType;eContent\right),i=1,2,\cdots ,n.$$

(1)

A risk event includes several attributes: event subject $A_i$, event type $eType$, and event content $eContent$. $A_i$ is the subject of the risk event, namely the system activity, $eType$ is the type of risk event; according to system risk classification, a risk event in an engineering system can be a schedule risk (time), a cost risk (cost), or a quality risk (quality) [38]. $eContent$ is the content of the risk event; the specific circumstances of stochastic risk events are the content of the event. For example, risk event $e_i=\left(A_i;time;t_i>t_0\right)$ means that the completion time $t_i$ of activity $A_i$ is over the threshold $t_0$.

Definition 2.

Risk Scenario. A risk scenario is a series of stochastic risk events that may cause system failure. This can be expressed as:

$$S=\left\{e_i|1\le i\le n\right\},$$

(2)

where $e_i$ represents a stochastic event related to risk. Since the stochastic events in the whole system are often uncountable, we pay more attention to those stochastic events that have happened. Hence, in this article, events that have not happened are no longer included in scenario sets—that is, if $e_i$ appears in set $S$, it means that $e_i$ has already happened in $S$, while if $e_i$ does not appear in set $S$, it means that $e_i$ does not happen. According to Equation (2), all non-empty subsets of $S$ are risk scenarios of n-risk events.

In this paper, we consider a risk scenario as a group of stochastic events or some specific combination of events [8,10]. To better explain the inherent relationship of risk event $E$, risk scenario $S$, and system failure, the notion of a “Scenario Box” is proposed as shown in Figure 1. A scenario box represents the stochastic risk events involved in a scenario and the consequences of the risk events. A risk scenario box consists of three dimensions, quality, time, and cost, and all stochastic risk events included in the scenario are around the scenario box. Under the combination of these stochastic risk events (i.e., under the effect of scenario $S$), when the risk of the whole system is enough beyond the defined threshold range of the scenario box, whether in time, cost, or quality, it is considered a system failure. This concept is considered the criterion to decide if the system fails in Monte Carlo simulation process of risk scenario in the following.

2.2. Risk Scenario Importance Measure

As a sensitivity analysis, IM (Importance Measure) is used to measure the importance of the reliability and risk of a component on the whole system [23]. The significance of a risk scenario is that its occurrence may cause system failure, so the identification of important risk scenarios is the top priority in system risk analysis. From the definition of risk scenario in this article, important risk scenarios and critical risk events are closely related. At the same time, risk events have event subject, which is activity, so critical activities mean that important risk events may happen. Therefore, in order to analyze system risk based on risk scenarios, important risk scenarios need to be identified and critical activities need to be found from important risk events. This article proposes definitions and formulas for activities’ uncertain importance measure, events’ importance measure and scenario importance measure, which are needed in risk scenario generation. Analysis and simulation of risk scenarios are based on these definitions.

(1) UIM (Uncertain Importance Measure) of Activity

Critical activities, which are necessary to meet system requirements, have a significant impact on the system’s total duration. In a stochastic network, we can start from critical activity analysis and then measure the importance of activities to identify critical activities and important risk events. There are two ways to assess activity importance: computing the activity criticality index and analyzing the uncertain importance measure. The former is used to identify an activity’s expected time, which will heavily influence the expected system completion time, while the latter can be used to identify activities that are worth more attention in terms of reducing the fluctuation of the uncertainty (namely variability) in the system completion time, T.

UIM (Uncertain Importance Measure) is generally used for sensitivity analysis of a model, in order to find the input parameters that have the greatest impact on the uncertainty of the whole model output [22]. In terms of system activities, we can measure the uncertain importance to identify those activities that have the greatest impact on the uncertainty of a system’s overall parameters (i.e., time, cost, or quality) from the uncertainty of the parameters (i.e., time, cost or quality) of a single activity [29]. Generally, those activities that have high uncertain importance are often critical in a system. Uncertain importance measure of activity, the basis of the entire risk scenario analysis, will be discussed in Section 3 to identify risk events and to give importance measure results of the risk scenario; finally, a risk scenario will be generated.

(2) ECI (Event Critical Index) of Event

In the system, when an activity has been identified as critical, it means that correlative stochastic risk events are critical risk events that make up the risk scenarios we are interested in. A risk event includes three attributes: event subject, event type, and event content (Equation 1). A risk event that makes critical activity its event subject should be identified first. In addition, determining whether an activity is critical from the time, cost, and quality dimensions is feasible. Thus, if an activity is determined to be critical in the time dimension, the type of risk event that should successfully identify the activity as the event subject is time. Likewise, if an activity is determined to be critical in the quality dimension, the type of risk event that should successfully identify the activity as the event subject cannot be time or cost.

Unlike other important measurement indexes proposed from the time or cost perspective [27,29], the importance of events in risk scenarios should be considered in three dimensions—cost, time, and quality. In order to measure the importance of events, an ECI (Event Critical Index) is proposed against the importance degree of system failure by risk events in this paper. The definition is as follows.

Definition 3.

Event Critical Index. An event critical index is the ratio of the number of system failures to the occurrences of risk events, which can be expressed as follows:

$$EC{I}_i=\frac{N\left(F|e_i\right)}{N\left(e_i\right)}.$$

(3)

(3) SCI (Scenario Critical Index) of Scenario

In order to determine the risk scenario, risk events and their importance need to be determined. According to the results, risk event sets that have the greatest effect on system failure are generated. Therefore, the generation of risk scenarios and the importance ranking can provide a foundation for follow-up risk assessment. Risk scenarios reflect the uncertainty of a system and the importance need to be analyzed when the probability of occurrence is assessed.

For measuring scenario importance, an SCI (Scenario Critical Index) is proposed to measure the importance degree on system failure by risk scenarios. The definition is as follows.

Definition 4.

Scenario Critical Index. A scenario critical index means the ratio of the number of system failures to the occurrences of risk scenarios, which can be expressed as follows:

$$SC{I}_i=\frac{N\left(F|S_i\right)}{N\left(S_i\right)}=\frac{N\left(F|e_{i1},e_{i2},\cdots ,e_{ik}\right)}{N\left(e_{i1},e_{i2},\cdots ,e_{ik}\right)}.$$

(4)

2.3. Framework of Risk Scenario Generation

The importance of critical activities needs to be analyzed and risk events need to be identified before the importance analysis of risk scenarios can take place. Risk event identification is a critical step; its results not only help us determine which events constitute risk scenarios, but also show the sorting results of risk events that have the greatest impact on system failure. The prerequisite of risk event identification is to identify system-critical activities according to the results of an activity importance measure analysis; events associated with critical activities are typically important risk events. After successfully identifying the events that have the highest impact on system failure, we need to analyze the importance of risk scenarios. The results form the basis of risk assessment and avoidance through ranking the importance of risk scenarios that may cause system failure. Based on the risk scenario analysis ideas above, a framework of risk scenario generation is proposed based on importance measure (Figure 2), in which the three basic steps of risk scenario generation are critical activity analysis, risk events identification and importance measure, and risk scenario importance analysis and generation.

The process of generating and analyzing system risk scenarios includes critical activity analysis, risk event identification, risk scenario generation, and other steps in this article. Critical activity analysis is the cornerstone of scenario generation method and can be carried out through simulation or mathematical analysis; when all risk events of interest are identified, the correlative scenarios can be obtained through risk events combination. In a system, some activities have a significant impact on the total duration and are an important part of on-time system completion; these are called critical activities. Managers tend to be more concerned about activities that can decide the entire system process or even the success or failure of the system. Therefore, it is important for managers to determine which activities are critical. Currently, there are two ways to assess activity importance: computing activity criticality index and analyzing uncertain importance measure. The former is used to identify an activity’s expected time, which will significantly influence the expected system completion time; the latter can be used to identify activities that are worth more attention in terms of reducing the fluctuation of the uncertainty (variability) in the system completion time. This article begins with a critical activity uncertain importance measure index and gives a study of risk scenario generation based on critical activity analysis.

3. Risk Scenario Generation Based on UIM of Critical Activity

3.1. The UIM Index of Critical Activity

As a general uncertainty analysis method, sensitivity analysis (SA) provides ideas for analyzing logistic support process uncertainty. Uncertainty analysis aims to figure out the influence of input variables on the final output uncertainty of the model throughout sensitivity analysis. The assessment and presentation of the influence of uncertainty have been widely recognized as important parts of analysis for complex systems [41]. At the simplest level, such an uncertainty analysis process can be viewed as the study of functions in the form of

Y = f[X] = f[x₁, x₂, …],

(5)

where Y is the dependent output, whose uncertainty is influenced by fluctuation of its independent inputs; function f represents the common model or models of Y under study; and X = [x₁, x₂, …] are k-independent inputs. Figure 3 shows that the goal of the uncertainty analysis is to determine the influence of input (X) uncertainty on the entire output (Y) distribution.

In an activity-based system, the uncertainty of overall system parameters (i.e., time, cost, or quality) caused by the uncertainty of parameters (i.e., time, cost or quality) of a single activity is called the “uncertain importance measure.” A critical activity is a predominant influence on the entire system, and is mainly divided into two types [31,39]: (1) Variance‒Variance, the variance of entire system parameters (i.e., whole time or whole cost) as influenced by the variance of activity parameters (i.e., time or cost); (2) Mean‒Variance, the variance of entire-system parameters (i.e., whole time or whole cost) as influenced by the change in activity parameters’ (i.e., time or cost) means. This article belongs to the Variance‒Variance type, studying the influence of the variance of activity parameters on the variance of the system parameters.

In CPM (Critical Path Method) and PERT (Program Evaluation and Review Technique) diagram, the analysis method of critical path and critical activity has been common and mature [42]. Common criticality indexes include PCI (Path Critical Index), ACI (Activity Critical Index), SI (Significance Index), CRI (Criticality Index), etc. [24,35]. When analyzing critical activities in a GERT network, we can start with the UIM of mean and variance of activities, while variance uncertainty is the emphasis of study in this article. As system parameters include time, cost, and quality, uncertain importance includes time-uncertain importance, cost-uncertain importance and quality-uncertain importance. In order to facilitate research, this article only researches time-uncertain importance, namely activity duration uncertain importance. This article chooses the definition of PERT network activity critical index of Cho & Yum [34] as a reference, giving the meaning of UIM of activity, that is, UIM is the uncertainty of total system duration caused by the uncertainty of duration of a single activity, which measures the degree of influence on the entire system by activity duration as a valid indicator. UIM is defined in this article as follows.

Definition 5.

UIM (Uncertain Importance Measure). The UIM of activities means the degree of the network activity time fluctuation compared to the total system time $T$, which can be expressed as follows:

$$UIM\left(i\right)=\frac{thevariabilityofTcausedbytimevariabilityfluctuationof{A}_i}{thevariabilityofT}.$$

(6)

Uncertainty in the above formula means the variation of activity time and $T$, while Importance measure means the influence on the degree of total time variation by the degree of a single activity’s time variation. For computing UIM of system activities, critical activity analysis is proposed with Taguchi tolerance design, mathematical analysis, and Monte Carlo simulation methods. These three methods have their own advantages, disadvantages, and applicability.

3.2. UIM Calculation of Critical Activity

3.2.1. Taguchi Tolerance Design Based on PERT

Taguchi tolerance design methods can be used in UIM computing processes [8,34]. PERT networks are classified into two types: Type A with one longer path (the critical path) than the others, and Type B without such a path. The difference between the two types is whether the activities on the dominantly longer path (namely critical path) have linear effects on T. Let ${\mu }_i$ and ${\sigma }_i{}^2$ be the mean and the variance of activity $i$, respectively. If $i$ is assumed to have a linear effect on the variability of $T$, it may have two test levels, ${\mu }_i-{\sigma }_i$ and ${\mu }_i+{\sigma }_i$, respectively, for the low and high levels. If $i$ is assumed to have a curvature effect on the variability of $T$, then three test levels, ${\mu }_i-{\sigma }_i\sqrt{\raisebox{1ex}{$2$}\!\left/ \!\raisebox{-1ex}{$3$}\right.},{\mu }_i$ and ${\mu }_i+{\sigma }_i\sqrt{\raisebox{1ex}{$2$}\!\left/ \!\raisebox{-1ex}{$3$}\right.}$, are included for the low, center, and high levels, respectively. To get the UIM results of network activities, Taguchi tolerance design usually proceeds as follows.

Stage 1: make all activities a set of $\mathsf{\Omega }$ and determine two test levels of each activity in $\mathsf{\Omega }$. Assign each of the activities to the chosen suitable Taguchi orthogonal array (OA) column according to the level. Calculate the system completion time using CPM (Critical Path Method) at each run of the selected design and perform ANOVA on the system completion times. Calculate the contribution ratio (the variation of $T$ attributable to the main effect of activity $i$) ${\rho }_i$, which is taken as an estimate of ${\mathrm{UIMA}}_i$.

Stage 2: let ${\mathsf{\Omega }}^{*}$ be the set of activities remaining after screening out those whose ${\mathrm{UIMA}}_i$ were negligible in magnitude (<1%, say) in Stage 1. Determine three test levels of each activity in ${\mathsf{\Omega }}^{*}$, which is partitioned into two sets according to ${\rho }_i$ in Stage 1, and select or construct an appropriate three-level OA. Then repeat the steps of Stage 1 and calculate ${\rho }_i$ and ${\rho }_{ij}$ (the variation of $T$ attributable to t the interaction effect of activity $i$ and $j$), respectively. Estimates of ${\mathrm{UIMA}}_i$ and ${\mathrm{UIMA}}_{ij}$ are given by ${\rho }_i$ and ${\rho }_i+{\rho }_j+{\rho }_{ij}$, respectively.

The drawback of tolerance design in the UIM method is that this method only holds for general PERT networks. For complex stochastic networks, such as GERT networks, the complexity of computation is very high. Since the standardized GERT work has analytical results, this article attempts to focus on mathematical analysis to replace Taguchi tolerance design for solving activity UIM indexes in general stochastic networks.

3.2.2. Mathematical Computation Based on GERT

Based on the above definition, UIM aims to measure the activity time fluctuation so that $T$ will change. Variation is proposed in this article to measure the activity time fluctuation and the change of $T$. Therefore, the UIM index of an activity $A_i$ can be redefined as follows:

$$UIM\left(i\right)=\frac{variancechangesofTcausedbyvariancechangesof{A}_i}{variancechangesofT},$$

(7)

where $A_i$ and $T$ are the activity $i$ and the system total time. According to Equation (7), we can get the computing definition of UIM as follows:

$$UIM\left(i\right)=\frac{\Delta V\left(T_i\right)}{V\left(T\right)},$$

(8)

where $\Delta V\left(T_i\right)$ and $V\left(T\right)$ are the variation value of variance of $T$ caused by the time fluctuation (variance changes) of a specific activity $A_i$. Based on the above definition, UIM results can be obtained under an analyzable standard GERT network. Several important steps are usually involved, as follows.

Step 1. Analyze the type of GERT network and compute the transfer function expression;

Step 2. According to the calculation of different network structures, compute the corresponding $n$-order moments about the origin (the one-order moments about the origin is the expectation value) when the variances of $A_i$ in a GERT network are 0 and a given value, and get $E\left(T_{i_0}\right)$, $E\left(T_{i_1}\right)$, $E\left(T_{i_0}^2\right)$ and $E\left(T_{i_1}^2\right)$, where $T_{i_0}$ and $T_{i_1}$ are the different $T$ when the variances of $A_i$ are 0 and a given value;

Step 3. Compute the corresponding variances of $T$ when the variances of $A_i$ are 0 and a given value according to Step 2, and get the values as follows:

$$V\left(T_{i_0}\right)=E\left(T_{i_0}^2\right)-E^2\left(T_{i_0}\right)$$

(9)

$$V\left(T_{i_1}\right)=E\left(T_{i_1}^2\right)-E^2\left(T_{i_1}\right).$$

(10)

Step 4. Compute the variation of $A_i$, namely the variance $\Delta V\left(T_i\right)$, according to Step 3 and get the values as follows:

$$\Delta V\left(T_i\right)=\left|V\left(T_{i_1}\right)-V\left(T_{i_0}\right)\right|.$$

(11)

Step 5. Compute the variation of $T$, namely the total variance of GERT network $V\left(T\right)$, similar to in Steps 2 and 3, and get the values as follows:

$$V\left(T\right)=E\left(T^2\right)-E^2\left(T\right).$$

(12)

Step 6. Put the results of $\Delta V\left(T_i\right)$ and $V\left(T\right)$ into Equation (8) and compute the UIM results of $A_i$.

The mathematical method certainly has more general applicability than the Taguchi tolerance design, but it is essential that stochastic networks be converted into a standard GERT network model, which is the worst drawback. In addition, the Mason formula and moment generating function are computed in an analytical algorithm, which is a cumbersome step. To solve these problems, the Monte Carlo simulation method is used to analyze the critical activities in GERT networks.

3.2.3. Simulation Method Based on Monte Carlo

The simulation framework of a system GERT network is shown as Figure 4. Current completion (including the achievement of nodes, activities completion, etc.) needs to be monitored as the program is undergoing continuous simulation. Information on activities (activity time, probability, precedence nodes and successor nodes, etc.) can be batch-imported to generate the corresponding GERT models.

Figure 5 shows the whole simulation process. It includes three main parts in Monte Carlo simulation analysis of system risk scenarios.

(1)

Random sample. Using the known parameter distributions (time distribution, cost distribution, etc.) for direct sampling. Generally, the parameter distributions involved are normal triangular distributions, so all sampling is done at once in this article, with sampling results to compute the relevant parameters of the entire system.

(2)

Determine the rate of risk scenarios and system failure according to the criteria. When judging whether a risk scenario has happened or not, risk events must be assessed first; whether the system fails or not depends on parameter sampling in Step 1: whether the system fails or not can be determined after the end of each sampling.

(3)

Importance measure analysis according to the statistical data of risk scenarios. This article focuses on two types of data, the occurring numbers ($N\left(S\right)$) or occurring probability ($P\left(S\right)$) of risk scenarios, and the failure numbers ($N(F|S)$) or failure probability ($P(F|S)$) of system when some specific scenarios happened.

In the second part, the description of risk events and the judgment criteria of system failure need be given based on specific situations of different systems, as the criteria of different systems vary widely. Collection of this information is important when system risk assessment is in progress. The flow of Monte Carlo simulation program of system risk scenarios is shown in the next figure.

3.3. Generation Steps of Risk Scenarios

In order to find the UIM values of all system activities, risk scenarios can be generated. Concrete steps are as follows:

Step 1. Rank the UIM values of activities, and got the results: $A_{\overline{1}}>A_{\overline{2}}>\cdots >A_{\overline{i}}>\cdots >A_{\overline{n}}\left(1<i<n\right)$, where $\overline{i}$ means the ranking numbers of UIM values;

Step 2. Analyze the system activities that have high UIM values; those that have a significant impact on the variance of system time will be chosen, such as activities as $A_{\overline{1}},A_{\overline{2}},\cdots ,A_{\overline{i}}\left(1<i<n\right)$, while low-impact activities are ignored;

Step 3. Make activities with high UIM values the subject of risk events. Rank the importance of risk events and identify risk scenarios;

Step 4. Combine the events in certain rules, and get important risk scenario set $S=\left\{e_m|1\le m\le i\right\}$, where all non-empty subsets can be considered risk scenarios.

The risk scenario generation method according to system critical activities in this article is simple and straightforward. According to the ranking results of activity importance, we can understand intuitively which activities have the greatest effect on the whole system and eliminate the low-impact activities to greatly increase the efficiency; moreover, if the identified events are a total of $N$, the number of scenarios may reach $2^N-1$ in theory. Once the level of $N$ cannot be reduced, $2^N-1$ will be a very large number. Therefore, some rules can be adopted to downsize scenarios in the stochastic combination of risk scenarios. For instance, one way is to add in the preference of decision-makers to filter out the risk scenarios; another is to classify the importance of activities and risk events into type A, type B, or type C. Ranking the importance of risk events first would be an effective way to judge the importance of risk scenarios, since important scenarios always include important events.

When risk events are independent of each other, the more important events there are in one scenario, the larger probability it will be a more important scenario. There are two plans to rank risk events: one is ranking with ECI results but requires additional Monte Carlo simulation; the other is ranking the preliminary events according to the results of critical activities ranking. The latter not only takes advantage of the information on critical activities, but saves time and resources as well. Therefore, this article directly ranks the UIM values.

4. Examples

4.1. Overview

A specific example in this article is shown to analyze the critical activities and generate risk scenarios. By ranking the UIM values of the critical activities of the system in the example, the critical activities and events are identified and risk scenarios are generated based on importance measure analysis, so as to illustrate the logical rationality of UIM indexes proposed in this article and verify the risk scenarios generated method based on importance measure analysis. Such activity-based systems are usually embodied in maintenance and logistic systems or engineering project systems, which are usually composed of specific activities [24,34,39]. A PERT network with six nodes and nine activities is shown in Figure 6, which is abstracted from an engineering project system. The durations of activities 1, 2, 4, 5, 6, 8, and 9 obey normal distribution with a mean of 3 and a variance of 1 (months), while the other two activities obey normal distribution with a mean of 6 and a variance of 2 (months).

4.2. Analysis of Critical Activity

First, transform the PERT model to a standard GERT network (Figure 7). Then compute the probability of every path in a standard GERT network (in

Table 1. Probability of each path.

Path	Probability	Path	Probability
$W_a$	$P_a=0.97803$	$W_e$	$P_e=0.00548$
$W_b$	$P_b=0.00547$	$W_f$	$P_f=0.00003$
$W_c$	$P_c=0.00548$	$W_g$	$P_g=0.00003$
$W_d$	$P_d=0.00546$	$W_h$	$P_h=0.00003$

) so that the transfer function of the entire GERT network can be obtained, as shown in Equation (13).

$$\begin{gathered} W_E=P_a{W}_a+P_b{W}_b+P_c{W}_c+P_d{W}_d+P_e{W}_e+P_f{W}_f+P_g{W}_g+P_h{W}_h \\ =P_a{e}^{21s+7s^2}+\left(P_b+P_c+P_d+P_e\right)e^{15s+5s^2}+\left(P_f+P_g+P_h\right)e^{9s+3s^2} \end{gathered}$$

(13)

As stated above, the UIM values of each activity could be computed and compared with the tolerance design results. Meanwhile, this article uses simulation to analyze the case. Using a pre-established GERT network model based on Monte Carlo simulation, this article calculates the mean and variance of the total system time under normal circumstances (i.e., the activity duration parameter is not changed), and the distribution of the total system time can be obtained as shown in Figure 8. The mean of the total time is 21.06 and the variance is 11.54.

Next, the effect on the system time of one activity’s variance fluctuation can be compared and the mean and variance of system total time can be simulated after the variances of each activity changed. The variance change of $A_3$ and $A_7$ affecting the total time is shown in Figure 9 and Figure 10. Obviously, the variance in $A_3$ and $A_7$ does not change the mean of total time much, but the variance grows significantly larger. In order to better compare the variance importance of all activities, this article computes the effect on the variance of system total time of the variance of each activity: UIM index. After statistical analysis, the results are as shown in

Table 2. Contrast of impact on system total time of the variance of each activity.

Activity Name	Variance of Activity Time Is 0		Variance of Activity Time Is Specific Value
	Mean	Variance	Change of Mean	Change of Variance
$A_1$	21.165	9.94	−0.50%	16.10%
$A_2$	21.024	11.53	0.17%	0.09%
$A_3$	21.18	6.90	−0.57%	67.25%
$A_4$	21.09	10.54	−0.14%	9.49%
$A_5$	21.21	10.10	−0.71%	14.26%
$A_6$	21.11	11.54	−0.24%	0.00%
$A_7$	20.84	7.70	1.06%	49.87%
$A_8$	20.67	11.31	1.89%	2.03%
$A_9$	21.08	9.96	−0.09%	15.86%

.

From Table 2, it is known that the influence of the variances of the nine activities on the mean of the total time is not significant. The variances of $A_3$ and $A_7$ have the greatest impact on the variance of system total time, reaching 67.25% and 49.87%, respectively, with $A_1$, $A_5$, and $A_9$ following behind. Only 9.49% of the impact is from $A_4$, while $A_2$, $A_6$, and $A_8$ can be neglected.

A comparison of UIM indexes and results from the three methods is given in

Table 3. Activities UIM results comparison of three methods.

Activity	Monte Carlo Simulation		Mathematical Analysis		Taguchi Tolerance Design
	UIM Values (%)	Rank of UIM	UIM Values (%)	Rank of UIM	UIM Values (%)	Rank of UIM
1	13.86	3	13.5382	2	14.29	2
2	0.09	7	0.0749	3	0	3
3	40.21	1	26.9273	1	28.57	1
4	8.67	5	0.0749	3	0	3
5	12.48	4	13.464	2	14.29	2
6	0.00	7	0.0749	3	0	3
7	33.28	2	26.9273	1	28.57	1
8	1.99	6	0.0749	3	0	3
9	13.69	3	13.5382	2	14.29	2

, in which Monte Carlo simulation obtains the most detailed analysis results. The importance values calculated by some activities are very close in mathematical analysis methods and Taguchi tolerance design, so activities are only divided into three types: The most important activities are $A_3$ and $A_7$, with $A_1$, $A_5$, and $A_9$ following behind, along with $A_2$, $A_4$, $A_6$, and $A_8$. In Taguchi tolerance design, $A_2$, $A_4$, $A_6$, and $A_8$ have almost no importance at all (a UIM value of zero), while in mathematical analysis, although the UIM values of $A_2$, $A_4$, $A_6$, and $A_8$ are not zero; this method does not make a further division of the activities of the three different importance measures. Monte Carlo simulation results show that $A_3$ is more important than $A_7$ (only from the perspective of UIM values comparison), while $A_4$ is more important than $A_2$ and $A_6$.

From the results of UIM index, the mathematical method is also the preferred method to assess the criticality of project activities, and with respect to the Taguchi tolerance design method, the mathematical method is not only applicable to PERT networks but more suitable for any stochastic networks that can be converted to standard GERT networks. Thus, compared with Taguchi tolerance design, the mathematical method is more common and easy to transplant. However, after comparing it with the simulation method, Monte Carlo simulation calculates the results more accurately. One reason is the specialty of the case: mathematical analysis makes further calculations after transforming the PERT networks into standard GERT networks. The process involves path probability problems that cannot be solved; the probabilities of paths are obtained by simulation, so it is not a strictly mathematical process. The other reason is that the simulation method is a stochastic sampling method; because of its limited number of simulations, the data convergence is not strong, and this will create differences in the UIM values.

4.3. Identification of Risk Events

According to the ranking results of UIM achieved by Monte Carlo simulation, $A_3$, $A_7$, $A_1$, $A_5$, and $A_9$ are chosen as activities of interest. The mean of the system is 21.06 and the variance is 11.54.

Table 4. Failure criterion of the system.

Criterion Type	Expectation	Failure Determination
time	21	T > 25.2

shows the failure criterion of this system case, with only the consideration of risk on time dimension; the failure criterion does not take cost and quality into account. Based on the results of critical activity analysis, $A_3$, $A_7$, $A_1$, $A_5$, and $A_9$ are the subject body of risk event and the type of event is time, as shown in

Table 5. Identification results of risk events based on critical activity analysis.

Risk Events	Subject	Type	Content
$e_1$	$A_3$	time	t > 7.2
$e_2$	$A_7$	time	t > 7.2
$e_3$	$A_1$	time	t > 4.2
$e_4$	$A_5$	time	t > 4.2
$e_5$	$A_9$	time	t > 4.2

.

In order to ensure the seriousness of risk events and their ECI values, the Monte Carlo simulation method is used to obtain the ECI indexes and rank the importance of each set of events, as shown in

Table 6. ECI index statistics of risk events.

Risk Events	Subject	ECI (%)	UIM of Subject (%)
$e_1$	$A_3$	27.3	32.23
$e_2$	$A_7$	27.2	32.23
$e_3$	$A_1$	15.6	12.07
$e_4$	$A_5$	17.9	12.03
$e_5$	$A_9$	17.1	12.07

. Among them, $e_1$ and $e_2$ have the highest ECI values of about 27.3% (error between 27.2% and 27.3% is ignored); their events activities, $A_3$ and $A_7$, have the highest UIM values. Meanwhile, the UIM values of $A_1$, $A_5$, and $A_9$ are about 12% smaller than $A_3$ and $A_7$; the ECI values of involved events $e_3$, $e_4$, and $e_5$ are also smaller than $e_1$ and $e_2$ and they are close numerically (the difference can be ignored).

From Table 6 the conclusion can be drawn that if an activity is determined as a critical activity in the time dimension (or in the cost or quality dimension), the importance measure of risk events (which makes this activity the subject and time the event type) is relatively high, as well as the UIM values of this activity. In other words, when we identify risk events through critical activity analysis, it is not necessary to simulate and compute ECI indexes with Monte Carlo simulation for judging the importance of risk events, because the ranks of risk events and critical activities relate to each other closely (the more critical the activities are, the higher importance the involved risk events have).

4.4. Generation of Risk Scenarios

After critical activity analysis of GERT network, important risk events can be identified from the UIM of project activities. Based on different combinations of rules, risk scenarios are generated to have a greater impact on the entire system failure risk. Since the systems in this article are small and the quantities of identified events are not high, this article takes a stochastic combined approach to generate a combination of risk scenarios, as shown in

Table 7. Importance analysis of risk scenarios combinations generated by risk events.

Risk Scenario S	SCI (%)	Sum (ECI)	Risk Scenario S	SCI (%)	Sum (ECI)
$S_1=\left\{e_1,e_2,e_3,e_4,e_5\right\}$	100	105.1	$S_{14}=\left\{e_1,e_3,e_5\right\}$	58.6	60
$S_2=\left\{e_1,e_2,e_4,e_5\right\}$	91.8	89.5	$S_{15}=\left\{e_2,e_3,e_5\right\}$	57.2	59.9
$S_3=\left\{e_1,e_2,e_3,e_4\right\}$	92.2	88	$S_{16}=\left\{e_1,e_2\right\}$	58.2	54.5
$S_4=\left\{e_1,e_2,e_3,e_5\right\}$	92.2	87.2	$S_{17}=\left\{e_3,e_4,e_5\right\}$	60.6	50.6
$S_5=\left\{e_1,e_3,e_4,e_5\right\}$	73.2	77.9	$S_{18}=\left\{e_1,e_4\right\}$	35.4	45.2
$S_6=\left\{e_2,e_3,e_4,e_5\right\}$	75.2	77.8	$S_{19}=\left\{e_2,e_4\right\}$	31.6	45.1
$S_7=\left\{e_1,e_2,e_4\right\}$	79.2	72.4	$S_{20}=\left\{e_1,e_5\right\}$	36	44.4
$S_8=\left\{e_1,e_2,e_5\right\}$	77.2	71.6	$S_{21}=\left\{e_2,e_5\right\}$	37.4	44.3
$S_9=\left\{e_1,e_2,e_3\right\}$	75.8	70.1	$S_{22}=\left\{e_2,e_3\right\}$	42	42.8
$S_{10}=\left\{e_1,e_4,e_5\right\}$	53.6	62.3	$S_{23}=\left\{e_1,e_3\right\}$	37.2	42.9
$S_{11}=\left\{e_2,e_4,e_5\right\}$	55.4	62.2	$S_{24}=\left\{e_4,e_5\right\}$	32.2	35
$S_{12}=\left\{e_1,e_3,e_4\right\}$	54.8	60.8	$S_{25}=\left\{e_3,e_4\right\}$	34.4	33.5
$S_{13}=\left\{e_2,e_3,e_4\right\}$	55.6	60.7	$S_{26}=\left\{e_3,e_5\right\}$	37.6	32.7

. In practical applications, other methods can be used to add risk preference to generate the corresponding risk scenarios.

From Table 7, the SCI generated by stochastic combinations of risk events is related to its ECI. Table 7 shows SCI values of 26 risk scenarios and the sum of their ECI values, which are analyzed after Monte Carlo simulation (which does not contain the scenarios combined by a single event). This shows that when the sums of ECI are more than 60%, there is a more than 50% probability that would the project system will fail. The higher sum of ECI indicates that according to the risk scenarios SCI is higher. For example, risk scenario $S_4$ is combined with risk events $e_1,e_2,e_3\mathrm{and}{e}_5$, which are subjected to activities $A_3,A_7,A_1\mathrm{and}{A}_9$. That means that if activities $A_3,A_7,A_1\mathrm{and}{A}_9$ of the project fail, the whole-project system failure probability is 92.2%. So, when project managers are assessing project risk, the risk scenario is critical. In particular, if activities $A_3,A_7,A_1,A_5\mathrm{and}{A}_9$ of the project all fail, corresponding to events $e_1,e_2,e_3,e_4\mathrm{and}{e}_5$, respectively, scenario $S_1$ will occur, and the probability of occurrence of project failure caused by it is 100%. Basically, risk scenarios that contain more critical risk events (namely, higher ECI values of events) or more risk events will have higher SCI indexes.

Based on the above analysis, we can conclude that risk scenarios combined with critical risk events (which refers to the higher ECI index here) are also critical risk scenarios. Therefore, after the “critical activity analysis‒risk event identification‒risk scenario generation” process of risk scenario generation, risk scenarios can be ranked in importance. The ranking results will let project managers know which risk scenarios to focus on.

5. Conclusions

Risk assessment is an important issue affecting the sustainability of economics and management. In this article, we propose a set of importance measurement indexes for activity, event, and risk scenarios in activity-based systems. Then we analyze the importance of critical activities by three different methods. Through computing the UIM indexes of activities, a ranking is obtained so that risk scenarios can be generated for project engineering. Finally, we analyze the SCI and ECI indexes of different risk scenarios and get the critical risk scenarios combinations generated by critical risk events.

After simulation and analysis, we conclude that the three critical activity analysis methods have their advantages and disadvantages. In the Taguchi tolerance design method, the steps are variable and the formulas are complex, so this method is mainly applicable to simple PERT networks with fewer factors (activities). In the mathematical analysis method GERT networks need to be standardized; for a simple standard GERT stochastic network this method is undoubtedly the most effective, but if the activity-based system is really complex, the GERT networks may not be transformed into standard networks directly. Monte Carlo simulation is a common method that, with the help of a computer program, can greatly reduce the human workload and improve the accuracy of the analysis results.

In brief, from the simulation results of the example system in this paper, it is proven that the proposed risk scenario generation method based on importance measurement analysis is effective compared with other methods. It includes three importance measurement indexes (for activity, event, and scenario respectively) during risk scenario simulation processes. Due to the constraints of the generation framework of risk scenarios, the method is much more suitable for activity-based systems, such as maintenance and logistics systems or engineering project systems. When implementing the present method to analyze system risk, a time-related system model should be built before the activity, event, and scenario analysis to describe the system risks. Sometimes it is difficult to abstract various types of systems to a time-related model. Meanwhile, the current importance measurement indexes, UIM, ECI, and SCI, all work on just one dimension of system failure (time, cost, or quality), which indicated by the variance of system parameters. In future work, the covariance could be considered to represent a multi-parameter importance measurement.