The step complexity measure for emergency operating procedures — comparing with simulation data

Abstract

In complex systems, such as nuclear power plants (NPPs) or airplane control systems, human errors play a major role in many accidents. Therefore, to prevent occurrences of accidents or to ensure system safety, extensive effort has been made to identify significant factors that cause human errors. According to related studies, written manuals or operating procedures are revealed as one of the most important factors, and complexity or understandability of a procedure is pointed out as one of the major reasons that make procedure-related human errors.

Many qualitative checklists are suggested to evaluate emergency operating procedures (EOPs) of NPPs. However, since qualitative evaluations using checklists have some drawbacks, a quantitative measure that can quantify the complexity of EOPs is imperative to compensate for them.

In order to quantify the complexity of EOPs, Park et al. suggested the step complexity (SC) measure to quantify the complexity of a step included in EOPs. In this paper, to ensure the appropriateness of the SC measure, SC scores are compared with averaged step performance time data obtained from emergency training records. The total number of available records is 36, and training scenarios are the loss of coolant accident and the excess steam dump event. The number of scenario is 18 each. From these emergency training records, step performance time data for 39 steps are retrieved, and they are compared with estimated SC scores of them. In addition, several questions that are needed to clarify the appropriateness of the SC measure are also discussed.

As a result, it was observed that estimated SC scores and step performance time data have a statistically meaningful correlation. Thus, it can be concluded that the SC measure can quantify the complexity of steps included in EOPs.

Introduction

In complex systems, such as nuclear power plants (NPPs) or airplane control systems, human errors play a major role in many accidents. For example, it was reported that about 70% of aviation accidents [1], and approximately 28% of accidents in process industries are caused by human errors [2]. In addition, it was pointed out that from 50 to 70% of the risk at nuclear facilities was because of human errors [3]. Therefore, to prevent an occurrence of accidents or to ensure system safety, extensive effort has been spent to identify significant factors that can affect/cause human errors. According to related studies, written manuals or operating procedures are revealed as one of the most important factors in aviation and manufacturing industries [4], [5]. In the case of NPPs, the importance of procedures is more salient than other industries because not only over 50% of human errors were due to procedures [6] but also about 18% of accidents were caused by the failure of following procedures [5]. In addition, complexity or understandability of a procedure is pointed out as one of the major reasons that make these kinds of procedure-related errors [4], [5], [6], [7]. Therefore, the importance of these procedures has been accentuated to ensure the safety of NPPs.

Recently, requirements such as “emergency operating procedures (EOPs) should be prepared so that the operators can easily and clearly understand the contents of them,” or “EOPs will be designed so that operators’ tasks can be accomplished within an acceptable workload or task performance time” are emphasized during their development and/or improvement [8], [9]. However, relevant evaluation frameworks to decide whether EOPs can satisfy these requirements are not suggested sufficiently.

Traditionally, a lot of guidelines or checklists have been used to investigate the suitableness of EOPs [10], [11], [12], [13]. Although the evaluation of EOPs using them has many advantages such as ease of use, it is insufficient because of two reasons. Firstly, evaluation results obtained from checklists would be subjective because they can be changed by experience or knowledge level of inspection personnel. Secondly, it is ambiguous whether EOPs are designed so that the operators can easily understand the contents of their tasks (i.e. what to be done), because many of them focused on the format of EOPs (such as sentence structure and vocabulary that used for task descriptions). For example, in the case of a steam generator tube rupture (SGTR) event, the operators should enter a SGTR procedure and perform many steps to accomplish tasks such as isolating ruptured steam generator and depressurizing reactor coolant system (RCS), within limited time period [14], [15]. In this situation, let us assume that these steps satisfy all kinds of evaluation guidelines or checklists. Nevertheless, it is still uncertain that whether the operators can clearly and easily understand the contents of a procedure or not, because results from the checklists cannot provide objective and quantitative estimations for the complexity or understandability of EOPs.

In order to compensate for these drawbacks, Park et al. suggested the step complexity (SC) measure to quantify the complexity of a step included in EOPs [16]. The SC measure consists of three sub-measures and the overall complexity for a step is defined by Euclidean norm of three sub-measures. In addition, the appropriateness of the SC measure was investigated by two ways: comparing estimated SC scores with subjective workload evaluation results and with averaged step performance time data obtained from protocol analyses of emergency operation training records. As a result, the SC measure seems to be appropriate because the estimated SC scores are reasonably agreed with both subjective workload evaluation results and averaged step performance time data. However, as the number of training records used to retrieve step performance time data is relatively small, the appropriateness of the SC measure cannot be sufficiently validated.

In this paper, to ensure the appropriateness of the SC measure, SC scores are compared with averaged step performance time data obtained from additional emergency training records. The total number of available records is 36, and training scenarios are the loss of coolant accident (LOCA) and the excess steam dump event (ESDE). The number of scenario is 18 each. From these emergency training records, step performance time data for 39 steps are retrieved, and they are compared with the estimated SC scores of them. In addition, several questions that are needed to clarify the appropriateness of the SC measure are also discussed.

The remaining paper is organized as follows. In Section 2, the background information related to the development of the SC measure is described. The results obtained from additional comparisons between estimated SC scores and averaged step performance time data are given in Section 3. Several questions that are needed to ensure the appropriateness of the SC measure are discussed in Section 4. Finally, conclusions of this study are drawn in Section 5.