The Experimental Study on Verifying the Rationality and Weakness of THERP with an Example of Truck Refrigerant Filling Process

With an example of truck refrigerant filling process, an experimental method by E-prime was used to verify the analysis of human error probability in the refrigerant filling process by The Technique of Human Error Rate Prediction (THERP). The result of the experiment is basically in coincidence with the result of THERP. The weakness of THERP is that it calculates the human error, ignoring the cognitive process of the operation behavior.


Introduction
There are many definitions of human error. From the view of engineering, the definition given by Swain is human behavior or action which exceeds the accepted standard or permissible scope of the normal work of the system [1]. From the perspective of psychology, the definition given by Reason refers to human behavior deviated from the intended plan or procedure sequence, or human intention or action, not getting the desired result or reaching the intended target [2].
Human reliability refers to the probability that human successfully completes the task at any required stage of the system operation within the specified minimum time limit [3]. The Technique of Human Error Rate Prediction (THERP) [4] is a widely used human reliability analysis method. THERP is a very powerful tool for analyzing the reliability of operators who perform testing or repairing tasks in existed procedures [5]. Kirwan's study indicated that the core problem of THERP is whether human error database and performance shaping factors (PSFs) can give the right result [6]. Kirwan and Richard conducted an experimental study with 30 British participants to measure the probability of human error and the result indicated that THERP was effective [7]. In another study, Kirwan explained the practical value of THERP from the perspective of reducing the probability of human error [8]. Huang et al. [9] analyzed the application of THERP method in Chinese nuclear power plant with quantified method and explained the feasibility of THERP method [9]. Tong et al. [10] studied the class factors of the application of THERP in nuclear power plants and gave some suggestions of improvement [10].
The research on human reliability analysis mainly focuses on nuclear power, aerospace or petrochemical fields [3,4,9,10]. In these areas, the rate of human error is very low, but it often leads to very serious production accidents. In truck manufacturing industry, there is few study on the application of the human reliability analysis. The present study applied experiment method to study one production process of truck manufacturing industry to verify the rationality and disadvantages of THERP.
The refrigerant filling process is an important process in the process of truck interior decoration in a certain enterprise, which influences the quality of the air conditioning of truck. According to the on-site investigation, the main operations of the refrigerant filling process includes connecting high-low pressure line of air conditioner with high-low pressure pipeline of filling machine, identifying type of air conditioner, choosing the filling quantity according to the type of air conditioner and pressing the button to fill. The refrigerant filling process relies on the main four operations to guarantee the completion of filling. Any errors in the process will result in the failure of the filling or quality problems, or even delay in truck production. The study focuses on the human error in the truck refrigerant filling process.

Analysis of the truck refrigerant filling process with THERP
THERP method makes task decomposition and analysis with human reliability analysis event tree, quantitatively calculates in accordance with human error data obtained from field experience and expert judgment, and amends with the performance shaping factors (PSFs), correlation and recovery factors. THERP method was used to analyze the human error of the truck interior refrigerant filling process. Figure 1 shows the HRA event tree diagram of the refrigerant filling process. There are two considered conditions including task series and parallel connection. In task series connection, the probability of error is as follows: In task parallel connection, the probability of error is as follows: According to the data value range provided by the THERP manual [1], combining the enterprise field investigation and expert scoring, the K values of the PSFs were obtained as follows: operator experience K 1 =0, psychological pressure K 2 =0, human machine interface K 3 =0.44. Through quantitative calculation, the human error probability of each operation in the process of refrigerant filling was obtained as follows: Operation 1 (connecting pipelines) P(F 1 )=4.32 × 10 -3 Operation 2 (identifying type of air conditioner) P(F 2 )=7.77 × 10 -4 Operation 3 (choosing the filling quantity) P(F 3 )=1.39 × 10 -2 Operation 4 (pressing the button to fill) P(F 4 )=1.08 × 10 -3

Experimental study
In the truck refrigerant filling process, human cognitive process is the most important. First, operator needs to identify different color pipelines belong to high pressure pipeline or low pressure before connecting pipelines. Second, operator must pay attention to the air conditioner nameplate and identify the air conditioner brand or manufacturer. Third, operator must store the information of the matching relation between air conditioning type and truck type in the brain, then transfer the stored information and respond to choose the correct filling quantity. Fourth, operator needs to identify the opening or closing button before pressing. The error probability of cognition process is the core factor in the whole filling process. In order to simulate the four cognition processes, one experiment is designed below.

Methods Participants
Forty-eight voluntary participants (25 men and 23 women) were recruited at Xi'an Jiaotong University for this experiment. They were informed of the objectives and details of the study and gave informed consent before the experiment commenced. They are all in good health and have normal vision or corrective vision.

Instrument and materials
One Lenovo notebook computer with good button function was used. E-prime experimental design software was applied. The experimental materials were displayed by E-prime, mainly including pictures of black color pipeline and coffee color pipeline, pictures of nameplates of six different types of air conditioners and pictures of buttons of "on" and "off".

Experimental task
Four experimental tasks were assigned to the participants. Task 1 was to identify pipelines with black and coffee color. Task 2 was to identify the type of air conditioner according to the nameplate. Task 3 was to choose the corresponding number according to the air conditioner nameplate. Task 4 was to identify the button of "open" or "off". For each operation, there was a cognitive task that corresponded to a picture. During the experiment, a picture was randomly sampled from the corresponding picture library and displayed to the participant.

Procedure
Before formal test, participants were familiar with the computer and the experimental materials. They learned to operate with buttons based on the printed experiment instruction. Then they began to perform the four experimental tasks in sequence of task 1, task 2, task 3 and task 4. Each participant needed to do this for twenty times. The operation errors of all the participants were automatically recorded.

Results
The average error rate of participants was calculated. Table 1 shows the average error rate and correct rate of participants. The result indicated that task 3 had the highest error probability, followed by task 1, task 2 and task 4.
The data were analyzed using Analysis of Variance (ANOVA) with SPSS19. The result indicated that task 3 was significant (p<0.05) higher than task 1. Task 3 was also significant higher than task 2 (p<0.05) and task 4 (p<0.01). Task 1 was significant (p<0.05) higher than task 4. There was no significant difference between task 1 and task 2, task 2 and task 4.

Discussion
From the experiment data, the human error probability in descending order was task 3, task 1, task 2 and task 4. Using THERP method, the descending order of the human error probability was also task 3, task 1, task 2 and task 4. Choosing the filling quantity and connecting pipelines had higher human error probability. Identifying type of air conditioner and pressing button to fill had lower human error probability. The difference of the results of two methods is that the values are much different. For example, the average error rate of task 1 in the experiment was 0.077, but the error rate of task 1 calculated by THERP was 0.00432.
In experiment, the error rate of task 1 was significant higher than that of task 4, but the difference in the result of THERP is not obvious. In the result of THERP, the error probability of task 1 and task 2 are very low, close to 0. THERP method has some weakness, such as, THERP method is mainly used in simple operations for the process and action analysis, without considering the cognitive process during operation. Especially for operations involving simple cognitive processes, the THERP method tends to ignore the cognitive process. In task 1, which is a more complex and complete information processing process, participant paid attention to color from one picture, stored and analyzed the vision sense of color, then compared with long term memory encoding [11], made decisions to press button in the end.
In task 2 and task 4, the long memory encoding only corresponded to text and button, such as "K" meant "open", "T" meant "Ted" (one type of air conditioner), and so on. The corresponding relationship between these words and letters are Chinese characters and the first Chinese phonetic alphabet of the characters. All participants remembered the information of the corresponding relation having more than ten years of repeated encoding, and better effect in long term memory. The correspondence in task 1 was a new correspondence, and there was no other relation between texts and letters. Obviously, the cognitive process of task 1 was more complex than task 2 and task 4. This illustrated that why the error rate of task 1 in the experiment was higher than task 2 and task 4, and why task 2 and task 4 differed slightly in error rate.
Overall, the experimental results are basically consistent with the THERP method, and there are some differences in absolute value. This verifies the rationality of the THERP method.

Conclusion
In the present study, the method of E-prime experiment was used and compared with the THERP method to analyze the human error probability in truck interior refrigerant filling process. The results are basically similar, which verifies the rationality of the THERP method. This is the same as the results of Kirwan and Richard [7]. The present study indicated that the weakness of THERP is that the cognitive process of behavior is ignored when calculating human errors.