Efficiency of VR-Based Safety Training for Construction Equipment: Hazard Recognition in Heavy Machinery Operations

Abstract

Machinery operations on construction sites result in many serious injuries and fatalities. Practical training in a virtual environment is the key to improving the safety performance of machinery operators on construction sites. However, there is limited research focusing on factors responsible for the efficiency of virtual training in increasing hazard identification ability among novice trainees. This study analyzes the efficiency of virtual safety training with head-mounted VR displays against flat screen displays among novice operators. A cohort of tower crane operation trainees was subjected to multiple simulations in a virtual towards this aim. During the simulations, feedback was collected using a joystick to record the accuracy of hazard identification while a post-simulation questionnaire was used to collect responses regarding factors responsible for effective virtual training. Questionnaire responses were analyzed using interval type-2 fuzzy analytical hierarchical process to interpret the effect of display types on training efficiency while joystick response times were statistically analyzed to understand the effect of display types on the accuracy of identification across different types of safety hazards. It was observed that VR headsets increase the efficiency of virtual safety training by providing greater immersion, realism and depth perception while increasing the accuracy of hazard identification for critical hazards such as electric cables.

1. Introduction

The construction industry accounts for more than 15% of all work-related fatal accidents in developed economies [1]. With the increased use of heavy machinery for construction operations, the key factor for avoiding incidents and managing risks effectively is to develop construction workers’ ability to identify hazards. The highest number of fatalities recorded on construction sites are related to workers being struck by moving machinery and vehicles [2,3]. Even in 2020, about 17% of fatalities on construction sites in Australia were caused due to workers being hit by moving vehicles, moving objects, or being trapped by moving machinery [1]. Earthmoving machinery and cranes are also responsible for many fatal accidents on construction sites [4,5]. Most of these accidents have been due to operator negligence or lack of training [6].

Several studies have analyzed the root causes of accidents and provided recommendations to avoid them. One of the most frequently cited recommendations for avoiding such accidents is providing site-specific safety training to heavy machinery operators and construction workers to increase their hazard perception ability while familiarizing them with the operation site [6,7]. Another important causality of accidents that needs to be addressed is the lack of structured training for novice heavy-machinery operators such as Tower crane operators. Lack of experience combined with a lack of site-specific training among novice operators can lead to accidents on site [8]. To solve this, many studies have recommended structured virtual training for novice construction operators [9]. However, only a few studies in the domain of heavy-machinery operations have employed head-mounted displays (HMDs) to utilize depth-perception offered by VR (Virtual Reality) HMDs to enhance safety during operations such as [10,11,12]. The importance of depth perception in training crane operators to perform critical tasks has also been highlighted by Juan et al. in their study where they developed stereoscopic and kinesthetic vision for training crane operators in a virtual environment [13]. This study aims to analyze the difference in training effectiveness of crane operations using head-mounted VR displays and flat screens by comparing the performance of novice operators in the virtual environment. Therefore, this paper uses an analytical approach using multicriteria decision-making (MCDM) to interpret participants’ feedback and reactions. The received feedback is analyzed using interval type-2 fuzzy analytical hierarchical process to rank the most critical factors to hazard perception. A sensitivity analysis is then performed to develop an in-depth insight into the impact of shortlisted factors on the overall training outcomes.

The rest of the paper is organized as follows: Section 2 presents the literature review of safety training and the analysis of MCDM problems in this space. Section 3 details the methodology used in this study. Section 4 provides details regarding results obtained from the analysis, and Section 5 discuss the results and draw conclusions for this study, respectively.

2. Literature Review

Heavy equipment is increasingly used in construction, increasing productivity and efficiency [14]. However, the downside of this advancement is an increase in accident rates caused by heavy equipment [15]. Previous research shows that around 15% of site accidents are “Struck by” heavy equipment [16]. Struck by moving objects was the leading cause for all accidents behind “Falls” and “Collapse of object” for most of these years. The use of construction equipment is one of the leading causes of fatalities on construction and infrastructure projects—specifically tower cranes, which account for up to 8% of all fatalities [4]. Previous research identified human error and falling from height as the two most common causes of crane accidents [5].

Furthermore, collapsing crane structures have been found to be another frequent cause of crane accidents [17]. In another study, worker negligence or human error was confirmed as the primary cause of tower crane accidents [18]. Considering the findings of these studies, it can be concluded that safety training can increase awareness of crane operators related to construction safety and hazardous situations, potentially reducing site accidents and injuries.

2.1. Safety Training for Heavy Construction Machinery Operations

The medium of the training is an essential factor for accident prevention in construction and infrastructure projects [19]. The effectiveness of VR training over conventional training has been measured by analyzing immediate effectiveness, short-term effectiveness, and information retention ability among construction workers [20]. A similar study used both VR and conventional displays to compare knowledge retention, self-efficacy, engagement, and self-reported presence in safety training [21]. The findings of these studies have been implemented for the skill enhancement of various heavy machinery operators. The applicability of VR systems for training crane operators was effectively demonstrated by Patrao et al. [22]. They developed two different simulators for stationary and mobile cranes, which effectively trained crane operators in an immersive environment. In another safety study focused on crane operations, the effectiveness of VR systems in training workers for the dismantlement of tower cranes was demonstrated by Li et al. [9]. The multiuser virtual safety training system was able to demonstrate better memory retention and site identification among workers trained using the virtual training.

Recent studies have also demonstrated an extension of research for skill enhancement, real-time data visualization, and instruction for optimized performance. The prediction of individual learning performance to indicate support requirements for training and learning experiences was studied by Arashpour et al. [23]. The use of 4D-BIM to create real-time augmented reality (AR) visualizations for crane operations, thereby optimizing safety and time of operation, was demonstrated by Lin et al. [24]. Similarly, the development of digital twins for the remote monitoring of tower cranes and the provision of information on an AR display for safety enhancement was demonstrated by He et al. [25]. In another study, real-time information on construction progress, lift path, and hazards present in the construction environment was relayed to tower crane operators using an intelligent crane operations assistance system [26].

Enhancing safety during crane operations has been the recent focus of studies that use VR and gamification of the virtual environment for operator training or lift path analysis. The use of VR to visualize lift paths and gamify the lifting process to detect a collision, thereby enhancing safety and lift planning, was demonstrated by Zhang et al. [27]. Virtual reality-based training for the assembly and dismantling of tower cranes was developed by Li et al., which focuses on training construction workers to recognize and respond to hazards during these operations [9]. Similarly, the use of VR for interactive tower crane layout planning and operator training has been investigated by Zhang et al. [28]. This study was an advancement to their earlier study for lift planning and operator training. They demonstrated intractability in multicriteria analysis using VR, thereby utilizing the human-in-the-loop approach to arrive at the optimum solution for the safety and efficiency of lifts. In a similar study, the effectiveness of VR training in optimizing lift planning and safety during lift operations was demonstrated by Kahyani et al. [29].

2.2. Multi-Criteria Decision Making for Assessing Effectiveness of VR Safety Trainings

The ability of a machine operator to recognize hazards, understand the risks involved, and react timely according to perceived hazards and risks is key to maintaining safety on a construction site. An important aspect of recognizing hazards stems from the ability to understand the distance to an object—i.e., depth perception. The superiority of VR in imparting hazard-perception knowledge when compared to that imparted by the two-dimensional representation of information has been demonstrated by Periman et al. [30]. The effectiveness of VR in memory retention and understanding of tasks has been demonstrated by Lee et al. [31], while cognitive load in VR immersive environment has been analyzed using the fuzzy Analytical Hierarchical Process (f-AHP) and Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) by Wang et al. [32]. However, none of these studies have evaluated the ability of equipment operators for depth perception and hazard identification as a result of VR-based safety training.

The analytical Hierarchical Process (AHP) is a process for ranking alternatives and criteria to reach an optimum solution to MCDM problems. AHP and fuzzy logic have been used to rank alternatives and analyze important criteria in the VR and training domains. Fuzzy type-1 AHP has been used to analyze the performance of VR and conventional displays in overhead crane operation training [11]. Moreover, this method has been used to analyze the effectiveness of VR-based training systems for mining environments [12]. Fuzzy AHP has also been used to solve MCDM problems by studying the effects of stereoscopic and kinesthetic vision on depth perception during safety training simulations [13].

Because of the inherent uncertainty of type-1 fuzzy sets, fuzzy type-2 sets have been proposed as effective MCDM problem-solvers. Site selection and prioritization of alternatives for renewable energy generation are among the complex problems that have been addressed using fuzzy type-2 AHP [33,34]. Fuzzy AHP has also been used to analyze the efficacy of VR to study peoples’ behavior during simulated fire evacuations [35]. Selection problems such as tunnel Boring Machine (TBM) selection [36] and multicriteria evaluation of construction equipment have been some of the applications of fuzzy AHP methods [37]. Fuzzy AHP has also been combined with TOPSIS to assess construction sites’ safety risks and relevant controls [38,39].

With the proven robustness of the fuzzy AHP analysis in the raking of multiple criteria for decision making [40,41], this paper employs interval type-2 fuzzy sets to convert qualitative inputs into quantitative fuzzy sets allowing evaluation of outputs according to the degree of their membership. The Interval Type-2 Fuzzy Analytical Hierarchy Process (IT2F-AHP) is used to obtain the quantitative relative weights of factors contributing to depth perception in hazard identification. These factors are then analyzed as an MCDM problem to rank the better of the two display types. This paper adopts a user-centric approach to assess depth perception and hazard identification among participants subjected to VR training with immersive and semi-immersive displays.

3. Methodology

This study focuses on analyzing the hazard recognition of crane operators on construction sites that employ tower cranes for transporting bulky precast or prefabricated modules during construction. Safety incidents simulated in the virtual environment include (1) Electrocution as a result of the crane hook or load colliding with electric lines, (2) Collisions due to hook or module colliding with the building structure or stationary vehicles, (3) Collisions with moving vehicles or workers on-site leading to “Struck-by” incidents, and (4) Collision with other tower cranes on site.

Unity3D [42] was used for creating the virtual environment due to its ability to simulate real-world physics. Tower crane simulations were based on a “Pick-and-place” activity wherein the crane operator needs to pick up a load or module from a supply point to a demand point. All the simulations were developed from a crane operator’s view in a tower crane cabin and had multiple incidents modeled during each simulation (Figure 1). A total of four scenarios were developed representing real-world pick-and-place activity with multiple hazards and safety incidents. Further, a script was developed that recorded reaction time when a participant provided feedback using a joystick.

Users were shown four different simulations based on the four scenarios developed. To ensure consistency during the experiment, two simulations (randomly selected from the four scenarios) were shown on a semi-immersive screen and the other two simulations were on an immersive display. To register instances when users identified hazards during each of the simulations, they were instructed to move the joystick lever in either of the four directions. The joystick used for the experiment was a four-axis Arduino-based joystick whose form was similar to a jib-operation joystick found in most hammer-head tower-crane cabins. Reaction times from joystick inputs were recorded for each of the participants and were analyzed against the modeled time of each incident to understand the ease of hazard recognition for different hazard types that were modeled in the virtual environment. The procedure for experimenting is represented in Figure 2.

The stepwise procedure for conducting this study is listed below:

• Participants are given a presentation on “pick and place” tower crane operations and potential hazards on construction sites.
• Participants are familiarized with the VR environment and registration of inputs using the joystick in the VR environment.
• Once the participants are familiar with the VR settings, they are shown two simulations on VR headset and two simulations on a semi-immersive flat-screen display. Their inputs to hazard identification are recorded in a database for further analysis.
• After the experiment, they are asked to provide feedback regarding various aspects of the simulations via a questionnaire survey.

The study participants were asked about their previous experiences with VR. It was seen that only 10% of participants had used VR previously for training, and 20% of the participants had experience using VR for gaming only. Before subjecting the participants to VR simulations, they were given a short 5 min discourse on these hazards frequently observed during tower crane operations.

Once familiar with the virtual environment and operation of the joystick, participants were shown the four simulations and their feedback on the joysticks was recorded (Figure 3). At the end of all four simulations, participants were asked to respond to a questionnaire evaluation of hazard recognition. Feedback from participants and reaction times form the basis of analysis to answer the following questions:

• How much difference does a VR headset make to training efficiency for construction machinery operator safety training?
• What type of hazards are better perceived using VR?
• Which aspects of construction machinery operator safety training can be improved using VR?

3.1. Participants

A cohort of 30 participants was selected from a course imparting knowledge on tower crane operations as novice operators for the purpose of this study. These participants were asked about their previous experiences with VR. It was seen that out of 30 participants, only three participants had used VR previously for training and seven of the participants had experience using VR for gaming applications; 70% of the participants had not used VR before.

The selected participants did not have experience operating tower cranes and can be considered a fair representation of the target area of improvement of this study—i.e., novice construction machinery operators.

3.2. Determining the Set of Evaluation Factors

The evaluation factors for the efficiency of virtual training were divided into four significant factors related to performance, effectiveness, presence, and device based on an existing body of knowledge as elaborated in this section combined with expert interviews. Each factor contains two or more sub-factors to evaluate hazard perception in virtual environments. These evaluation factors are presented in Figure 4.

3.2.1. Performance Factors

Performance factors focus on the core task of spotting hazards and reacting to them. Performance factors comprise the following sub-factors.

• Ease of operations: This criterion measures the ease of conducting the given task in VR, including task complexity and problem-solving ability [43].
• Enjoyability: This criterion relates to the ease of learning and the sense of enjoyment and satisfaction while performing the task in VR [44,45].
• Level of Engagement: The amount of learning absorbed from virtual training depends on the correlation to the actual task [46].
• Response Accuracy: Accuracy of response to hazard recognition task that can be measured in terms of time, i.e., how quickly a participant can recognize and react to a hazard during the operation [12].

3.2.2. Effectiveness Factors

Effectiveness factors are related to the utility and usefulness of the device while spotting hazardous conditions in the virtual environment.

• Objective usability: Previous studies have employed a comprehensive fuzzy evaluation of safety training systems to measure the effectiveness of the training in terms of cognitive load and decomposition of the task [12]. These terms have been captured by “objective usability” under effectiveness factors.
• Perceived usefulness: An important sub-factor that constitutes training efficiency is its “perceived usefulness”. This sub-factor focuses on the difference in learning and recognition when using virtual safety training for the enhancement of depth perception.
• Ease of recognition: Participants’ response to “ease of recognition” was also evaluated under effectiveness evaluation. The aim is to understand the effectiveness of visuals for recognizing hazards when using VR displays and flat screens [47].

3.2.3. Presence Factors

The feeling of presence in a VR depends upon involvement, interface quality, sensory fidelity, and immersion relayed by the technology [48]. Since this experiment relies on reaction to perceived hazards in a simulated environment, the only two applicable sub-factors are “realism” and “immersion”. Some of the notable publications that have analyzed presence factors as part of the study of VR-based safety training are [11,12,49].

• Immersion: The level of involvement and the feeling of “being there”. This sub-factor measures the effect of immersion on the hazard perception of participants.
• Realism: This sub-factor aims to understand the effect of visual fidelity on depth perception and hazard recognition.

3.2.4. Device Factors

Along with studying the effect of depth perception in recognizing and reacting to hazardous situations, the level of comfort during training is captured. To compare training concerning comfort, device factors consisting of discomfort and ease of interaction were considered. The weight of the device, sickness feeling, and the freedom of interaction was evaluated by seeking participants’ responses to these criteria. These factors have been studied in multiple articles with respect to the comfort of use [11,12,50,51]

• Weight: Feedback from participants in understanding if the weight of the VR device caused any discomfort during the training
• Sickness: Investigating if the VR headset caused any nausea or disorientation during the training.
• Interaction: This factor aims to understand if training using VR and flat screen offered various levels of interactivity.

3.3. Interval Type-2 Fuzzy AHP Process

The analytical Hierarchical Process (AHP) was developed as a practical decision-making process by Saaty [52] for solving complex problems using qualitative and quantitative criteria. AHP has been widely used in various research fields to solve MCDM problems. Using AHP to solve MCDM problems involves creating a hierarchy of criteria and conducting decision-makers pairwise comparisons of criteria.

Since linguistic variables are widely used to compare and rank criteria and alternatives, using linguistic variables can lead to vagueness in decision-making. Fuzzy sets were introduced to cater to vagueness in responses to multi-factor problems in 1965 by Zadeh et al. [53]. Most fuzzy AHP methods utilize type-1 fuzzy sets to represent ambiguity in linguistic variables. However, since the linguistic terms used can have different meanings for different people (i.e., linguistic terms can be relative), an additional degree of freedom can allow for modeling uncertainty and verbal ambiguity. Thus, the concept of type-2 fuzzy sets was introduced by Zadeh [54] as an extension to type-1 fuzzy sets to represent uncertainty by introducing a third dimension in the membership function of fuzzy sets. Type-2 fuzzy sets allow the incorporation of linguistic uncertainty in the concept of fuzzy sets by allowing for an additional degree of freedom with the third dimension for determining the membership function of the fuzzy set. However, due to the complexity of calculations with type-2 fuzzy sets, a special case of generalized type-2 fuzzy sets, named interval type-2 fuzzy (IT2F) sets, has seen greater application due to the simplicity of computations [55]. This paper employs IT2F numbers to deal with vagueness while performing AHP. The advantage of using this method is that it can convert qualitative inputs into quantitative fuzzy sets allowing the evaluation of outputs according to the degree of their membership [12].

The process of using IT2F sets to perform AHP evaluation can be complex due to various methods for the defuzzification of IT2F sets. A popular method for using IT2F sets with AHP has been developed and described by Kahraman et al. [56]. It is popular due to the simplicity of the defuzzification process. The linguistic variables and the equivalent IT2F numbers used in this study were selected based on the scale suggested by Kahraman et al. [56], given in

Table 1. Definition of the linguistic variables and the equivalent IT2F scales.

Linguistic Variables	Trapezoidal IT2F Scales
Absolutely Strong (AS)	((7, 8, 9, 9; 1, 1), (7.2, 8.2, 8.8, 9; 0.8, 0.8))
Very Strong (VS)	((5, 6, 8, 9; 1, 1), (5.2, 6.2, 7.8, 8.8; 0.8, 0.8))
Fairly Strong (FS)	((3, 4, 6, 7; 1, 1), (3.2, 4.2, 5.8, 6.8; 0.8, 0.8))
Slightly Strong (SS)	((1, 2, 4, 5; 1, 1), (1.2, 2.2, 3.8, 4.8; 0.8, 0.8))
Exactly Equal (E)	((1, 1, 1, 1; 1, 1), (1, 1, 1, 1; 1, 1))

, and graphically represented in Figure 5.

Although other scales for the representation of linguistic terms using IT2F sets exist [57,58], this scale is more suitable as it allows participants to rate two options as equal at the lowest end of the scale. As this study aims to differentiate between two alternatives, this scale is deemed suitable. The IT2F set $\widetilde{\tilde{\mathrm{A}}}$ can be written as follows:

$$\widetilde{\tilde{\mathrm{A}}}=\left\{\left({\mathrm{l}}_U,{\mathrm{m}}_{1U},{\mathrm{m}}_{2U},{\mathrm{u}}_U;{\alpha }_U,{\beta }_U\right),\left({\mathrm{l}}_L,{\mathrm{m}}_{1L},{\mathrm{m}}_{2L},{\mathrm{u}}_L;{\alpha }_L,{\beta }_L\right)\right\}$$

(1)

where ${\mathrm{u}}_U$ is the largest possible value of the upper membership function; ${\mathrm{l}}_U$ is the least possible value of the upper membership function; ${\mathrm{m}}_{1U}$ and ${\mathrm{m}}_{2U}$ are the second and third parameters of the upper membership function, respectively; ${\mathrm{u}}_L$ is the largest possible value of the lower membership function; ${\mathrm{l}}_L$ is the least possible value of the lower membership function; ${\mathrm{m}}_{1L}$ and ${\mathrm{m}}_{2L}$ are the second and third parameters of the lower membership function, respectively; ${\alpha }_U$ and ${\beta }_U$ are the membership degrees of ${\mathrm{m}}_{1U}$ and ${\mathrm{m}}_{2U}$, respectively; ${\alpha }_L$ and ${\beta }_L$ are the membership degrees of ${\mathrm{m}}_{1L}$ and ${\mathrm{m}}_{2L}$, respectively.

Different steps taken in this study for solving the MCDM problem in this study are illustrated in Figure 6 and described thoroughly in this section.

Step-1—Construct IT2F pairwise comparison matrices: The pairwise comparison matrices of all hierarchy levels, including the main factors, subfactors, and alternatives levels, are constructed from participant responses using linguistic terms.

Step-2—Consistency check: In order to evaluate the consistency of a comparison matrix with IT2F sets as its elements, defuzzification of the matrix should be performed. Defuzzification of a type-2 fuzzy set is a two-step process where IT2F set is converted to a type-1 fuzzy set using a reduction process in the first step, and then a defuzzification method for ordinary type-1 fuzzy sets is used to find the equivalence of the IT2F set. Reduction of an IT2F set to a type-1 fuzzy set using the centroid of the type-2 fuzzy set was proposed by Karnik and Mandel [59], while the use of pessimistic, optimistic, realistic, and weighted average indices for using different points of view while performing the reduction was proposed by Niewiadomski et al. [60]. These methods, along with methods for ranking the type-2 fuzzy sets [61,62] were compared by Kahraman et al. [56] and two new ranking methods, based on the center of area (CoA), for the better ranking of type-2 fuzzy sets according to the shape of the interval type-2 fuzzy set were proposed. The equation for defuzzification of trapezoidal type-2 fuzzy sets (DTraT) is described below.

$$\mathrm{DTraT}=\frac{\frac{\left(u_U-l_U\right)+\left({\alpha }_U·m_{1U}-l_U\right)+\left({\beta }_U·m_{2U}-l_U\right)}{4}+l_U+\frac{\left(u_L-l_L\right)+\left({\alpha }_L·m_{1L}-l_L\right)+\left({\beta }_L·m_{2L}-l_L\right)}{4}+l_L}{2}$$

(2)

Elements of the IT2F set used in the above equation are described in Equation (1).

Step 3—Aggregate pairwise comparison matrices: In order to create an aggregated pairwise comparison matrix at each level, response matrices from each participant are aggregated. Aggregation for each element is conducted by taking the geometric mean for the respective element. The following equation is used to obtain the element $r{̃}_{ij}$ by aggregating n pairwise comparison matrices:

$$r{̃}_{ij}={\left[{\tilde{\tilde{a}}}_{ij,1}\otimes {\tilde{\tilde{a}}}_{ij,2}\otimes \dots \otimes {\tilde{\tilde{a}}}_{ij,n}\right]}^{\frac{1}{n}}$$

(3)

where $r{̃}_{ij}$ is the aggregated IT2F number related to the pairwise comparison of the elements i and j; ${\tilde{\tilde{a}}}_{ij,k}$ is the IT2F number given by the kth participant. The pairwise comparison matrices for level-1 and level-2 factors are tabulated in

Table 2. Aggregated IT2F comparison matrix Level 1.

	Performance	Effectiveness	Presence	Device
P	((1, 1, 1, 1; 1, 1), (1, 1, 1, 1; 1, 1))	((0.66, 0.75, 0.98, 1.15; 1, 1), (0.68, 0.77, 0.94, 1.11; 0.8, 0.8))	((0.56, 0.67, 0.91, 1.09; 1, 1), (0.58, 0.69, 0.88, 1.04; 0.8, 0.8))	((0.84, 1.19, 1.89, 2.39; 1, 1), (0.92, 1.24, 1.81, 2.27; 0.8, 0.8))
E	((0.86, 1.04, 1.34, 1.51; 1, 1), (0.9, 1.06, 1.3; 0.8, 0.8))	((1, 1, 1, 1; 1, 1), (1, 1, 1, 1; 1, 1))	((0.46, 0.56, 0.86, 1.18; 1, 1), (0.49, 0.58, 0.81, 1.09; 0.8, 0.8))	((0.94, 1.2, 1.7, 2.03; 1, 1), (1, 1.24, 1.63, 1.95; 0.8, 0.8))
Pr	((0.92, 1.11, 1.49, 1.77; 1, 1), (0.96, 1.13, 1.44; 0.8, 0.8))	((0.84, 1.18, 1.78, 2.14; 1, 1), (0.92, 1.22, 1.72, 2.06; 0.8, 0.8))	((1, 1, 1, 1; 1, 1), (1, 1, 1, 1; 1, 1))	((2.39, 3.07, 4.26, 4.82; 1, 1), (2.53, 3.19, 4.15, 4.71; 0.8, 0.8))
D	((0.42, 0.53, 0.85, 1.18; 1, 1), (0.44, 0.55, 0.8; 0.8, 0.8))	((0.49, 0.6, 0.84, 1.06; 1, 1), (0.51, 0.61, 0.8, 1; 0.8, 0.8))	((0.2, 0.24, 0.33, 0.42; 1, 1), (0.21, 0.24, 0.31, 0.39; 0.8, 0.8))	((1, 1, 1, 1; 1, 1), (1, 1, 1, 1; 1, 1))

Note: P: Performance, E: Effectiveness, Pr: Presence, and D: Device.

,

Table 3. Pairwise IT2F comparison matrix—Performance factors.

	EO	EN	LE	RA
EO	((1, 1, 1, 1; 1, 1), (1, 1, 1, 1; 1, 1))	((1.37, 1.82, 2.74, 3.44; 1, 1), (1.46, 1.9, 2.61, 3.25; 0.8, 0.8))	((0.44, 0.61, 0.95, 1.25; 1, 1), (0.47, 0.63, 0.9, 1.16; 0.8, 0.8))	((0.37, 0.42, 0.54, 0.6; 1, 1), (0.38, 0.43, 0.51, 0.59; 0.8, 0.8))
EN	((0.29, 0.37, 0.56, 0.73; 1, 1), (0.31, 0.38, 0.52; 0.8, 0.8))	((1, 1, 1, 1; 1, 1), (1, 1, 1, 1; 1, 1))	((0.13, 0.15, 0.21, 0.27; 1, 1), (0.13, 0.15, 0.2, 0.25; 0.8, 0.8))	((0.33, 0.41, 0.61, 0.78; 1, 1), (0.35, 0.42, 0.57, 0.73; 0.8, 0.8))
LE	((0.8, 1.06, 1.68, 2.27; 1, 1), (0.85, 1.1, 1.58; 0.8, 0.8))	((3.71, 4.84, 6.81, 7.67; 1, 1), (3.95, 5.06, 6.61, 7.5; 0.8, 0.8))	((1, 1, 1, 1; 1, 1), (1, 1, 1, 1; 1, 1))	((0.99, 1.39, 2.09, 2.47; 1, 1), (1.07, 1.45, 2, 2.38; 0.8, 0.8))
RA	((1.65, 1.9, 2.39, 2.71; 1, 1), (1.69, 1.94, 2.3; 0.8, 0.8))	((1.27, 1.67, 2.46, 2.97; 1, 1), (1.36, 1.74, 2.35, 2.85; 0.8, 0.8))	((0.4, 0.48, 0.73, 1; 1, 1), (0.42, 0.5, 0.69, 0.93; 0.8, 0.8))	((1, 1, 1, 1; 1, 1), (1, 1, 1, 1; 1, 1))

Note: EO: Ease of Operation, EN: Enjoyability, LE: Level of Engagement, and RA: Response Accuracy.

,

Table 4. Pairwise IT2F comparison matrix—Effectiveness factors.

	OU	PU	ER
OU	((1, 1, 1, 1; 1, 1), (1, 1, 1, 1; 1, 1))	((0.49, 0.61, 0.91, 1.18; 1, 1), (0.52, 0.63, 0.87, 1.11; 0.8, 0.8))	((0.18, 0.21, 0.28, 0.35; 1, 1), (0.18, 0.21, 0.27, 0.34; 0.8, 0.8))
PU	((0.84, 1.11, 1.64, 2.03; 1, 1), (0.9, 1.15, 1.58; 0.8, 0.8))	((1, 1, 1, 1; 1, 1), (1, 1, 1, 1; 1, 1))	((0.19, 0.23, 0.32, 0.39; 1, 1), (0.2, 0.23, 0.3, 0.37; 0.8, 0.8))
ER	((2.81, 3.58, 4.91, 5.52; 1, 1), (2.96, 3.72, 4.75; 0.8, 0.8))	((2.54, 3.19, 4.44, 5.2; 1, 1), (2.68, 3.32, 4.29, 5.02; 0.8, 0.8))	((1, 1, 1, 1; 1, 1), (1, 1, 1, 1; 1, 1))

Note: OU: Objective Usability, PU: Perceived Usefulness, and ER: Ease of Recognition.

,

Table 5. Pairwise IT2F comparison matrix- Presence factors.

	IM	RL
IM	((1, 1, 1, 1; 1, 1), (1, 1, 1, 1; 1, 1))	((0.48, 0.73, 1.37, 2.07; 1, 1), (0.54, 0.77, 1.28, 1.87; 0.8, 0.8))
RL	((0.48, 0.73, 1.37, 2.07; 1, 1), (0.54, 0.77, 1.28; 0.8, 0.8))	((1, 1, 1, 1; 1, 1), (1, 1, 1, 1; 1, 1))

where IM: Immersion and RL: Realism.

and

Table 6. Pairwise IT2F comparison matrix- Device factors.

	W	SK	IT
W	((1, 1, 1, 1; 1, 1), (1, 1, 1, 1; 1, 1))	((0.25, 0.31, 0.52, 0.85; 1, 1), (0.26, 0.31, 0.48, 0.75; 0.8, 0.8))	((0.92, 1.28, 1.97, 2.37; 1, 1), (0.99, 1.34, 1.89, 2.28; 0.8, 0.8))
SK	((1.17, 1.92, 3.3, 3.97; 1, 1), (1.33, 2.06, 3.15; 0.8, 0.8))	((1, 1, 1, 1; 1, 1), (1, 1, 1, 1; 1, 1))	((1.12, 1.6, 2.75, 3.81; 1, 1), (1.22, 1.69, 2.59, 3.52; 0.8, 0.8))
IT	((0.42, 0.51, 0.79, 1.08; 1, 1), (0.44, 0.52, 0.75; 0.8, 0.8))	((0.26, 0.37, 0.63, 0.9; 1, 1), (0.28, 0.38, 0.59, 0.82; 0.8, 0.8))	((1, 1, 1, 1; 1, 1), (1, 1, 1, 1; 1, 1))

Note: W: Weight, SK: Sickness, IT: Interaction.

.

Step-4—Fuzzy geometric mean: In order to calculate the weights of criteria and alternatives, the geometric mean of each row of defuzzified matrix is calculated using Equation (2). This step is represented by the following equation:

$${\tilde{\tilde{r}}}_i={\tilde{\tilde{r}}}_{i1}+{\tilde{\tilde{r}}}_{i2}+\cdots +{\tilde{\tilde{r}}}_{iD}$$

(4)

Step-5—Calculate fuzzy weights of factors, subfactors, and criteria: Using the geometric means, the priority weight of the ith element (${\tilde{\tilde{w}}}_i$) can then be calculated using the following equations, in which D is the dimension of the pairwise comparison matrix.

$${\tilde{\tilde{w}}}_i={\tilde{\tilde{r}}}_i\otimes {\left[{\tilde{\tilde{r}}}_1\oplus {\tilde{\tilde{r}}}_2\dots \oplus {\tilde{\tilde{r}}}_i\dots \oplus {\tilde{\tilde{r}}}_D\right]}^{-1}$$

(5)

Step-6—Defuzzify and Normalize weights: Fuzzy weights for criteria and alternatives calculated in the last step are defuzzified using Equation (2). These weights are normalized to arrive at local weights for all the subfactors, and then further global weights are calculated according to the defined hierarchy of factors as detailed in Step-7 and Step-8. The fuzzy, local, and global weights of evaluation factors and subfactors are presented in

Table 7. Local weights of Evaluation factors.

Factors	Fuzzy Weights	Crisp Weights	Normalized Weights
Performance Factors	((0.14, 0.18, 0.32, 0.44; 1, 1), (0.14, 0.2, 0.3, 0.39; 0.8, 0.8))	0.2513	0.2621
Efficiency Factors	((0.13, 0.18, 0.31, 0.43; 1, 1), (0.15, 0.2, 0.3, 0.39; 0.8, 0.8))	0.2488	0.2595
Presence Factors	((0.12, 0.17, 0.29, 0.39; 1, 1), (0.13, 0.17, 0.27, 0.36; 0.8, 0.8))	0.2265	0.2362
Device Factors	((0.12, 0.17, 0.29, 0.4; 1, 1), (0.14, 0.18, 0.28, 0.37; 0.8, 0.8))	0.2323	0.2422

and

Table 8. Weights of subfactors.

	Fuzzy Weights	Crisp Weights	Normalized Local Weights
Weights of subfactors for Performance Factors
EO	((0.11, 0.16, 0.28, 0.39; 1, 1), (0.12, 0.17, 0.26, 0.37; 0.8, 0.8))	0.2218	0.2534
EN	((0.12, 0.16, 0.27, 0.36; 1, 1), (0.13, 0.17, 0.25, 0.33; 0.8, 0.8))	0.2133	0.2437
LE	((0.09, 0.15, 0.27, 0.38; 1, 1), (0.11, 0.15, 0.26, 0.37; 0.8, 0.8))	0.2123	0.2426
RA	((0.12, 0.17, 0.29, 0.39; 1, 1), (0.14, 0.18, 0.26, 0.36; 0.8, 0.8))	0.2278	0.2603
Weights of subfactors for Efficiency Factors
OU	((0.18, 0.23, 0.35, 0.44; 1, 1), (0.18, 0.24, 0.33, 0.42; 0.8, 0.8))	0.2820	0.3316
PU	((0.16, 0.21, 0.35, 0.46; 1, 1), (0.17, 0.22, 0.33, 0.43; 0.8, 0.8))	0.2775	0.3263
ER	((0.17, 0.23, 0.36, 0.49; 1, 1), (0.18, 0.23, 0.33, 0.45; 0.8, 0.8))	0.2910	0.3422
Weights of subfactors for Presence Factors
IM	((0.23, 0.36, 0.68, 0.98; 1, 1), (0.26, 0.39, 0.63, 0.89; 0.8, 0.8))	0.5270	0.5000
RL	((0.23, 0.36, 0.68, 0.98; 1, 1), (0.26, 0.39, 0.63, 0.89; 0.8, 0.8))	0.5270	0.5000
Weights of subfactors for Device Factors
W	((0.15, 0.21, 0.39, 0.58; 1, 1), (0.16, 0.23, 0.37, 0.53; 0.8, 0.8))	0.3125	0.3125
SK	((0.12, 0.2, 0.46, 0.74; 1, 1), (0.14, 0.23, 0.42, 0.67; 0.8, 0.8))	0.3563	0.3563
IT	((0.16, 0.23, 0.42, 0.59; 1, 1), (0.18, 0.25, 0.4, 0.55; 0.8, 0.8))	0.3313	0.3313

, respectively.

Step-7—Determine Priority: Based on local weightages, the priorities for main evaluation factors and subfactors are determined. These priorities allow the understanding of the most important evaluation factor and subfactors that correlate depth perception and hazard identification. These priorities are also referred to as local weightages.

Step-8—Rank alternatives: Based on the local weightages, the overall ranking—global weightages, of subfactors, and alternatives are determined. To arrive at the global weightage of a subfactor, the local weight of the subfactor is multiplied by the local weight of the evaluation factor it is located under. Similarly, to arrive at the global weight of alternatives for each subfactor, the local weight of the alternative is multiplied by the global weight of the subfactor. At this step, the most important subfactors and selected alternatives are determined as the most appropriate solution to the MCDM problem as shown in

Table 9. Subfactor weights with respect to alternatives.

Ease of Operation	Fuzzy Weights	Crisp Weights	Normalized Weight
Ease of Operation
Flat	((2.09, 2.43, 3.06, 3.39; 1, 1), (2.14, 2.48, 2.97, 3.31; 0.8, 0.8))	2.5975	0.6270
VR	((1.41, 1.49, 1.71, 1.91; 1, 1), (1.44, 1.5, 1.68, 1.86; 0.8, 0.8))	1.5455	0.3730
Enjoyability
Flat	((0.31, 0.38, 0.5, 0.59; 1, 1), (0.33, 0.38, 0.48, 0.58; 0.8, 0.8))	1.2785	0.2483
VR	((0.26, 0.33, 0.56, 0.73; 1, 1), (0.27, 0.36, 0.53, 0.69; 0.8, 0.8))	3.8710	0.7517
Level of Engagement
Flat	((0.31, 0.38, 0.5, 0.59; 1, 1), (0.33, 0.38, 0.48, 0.58; 0.8, 0.8))	1.3203	0.2692
VR	((0.26, 0.33, 0.56, 0.73; 1, 1), (0.27, 0.36, 0.53, 0.69; 0.8, 0.8))	3.5843	0.7308
Response Accuracy
Flat	((0.37, 0.41, 0.48, 0.52; 1, 1), (0.39, 0.42, 0.47, 0.51; 0.8, 0.8))	1.2438	0.2297
VR	((0.34, 0.39, 0.5, 0.58; 1, 1), (0.36, 0.4, 0.48, 0.55; 0.8, 0.8))	4.1715	0.7703
Objective Usability
Flat	((0.31, 0.41, 0.61, 0.75; 1, 1), (0.33, 0.42, 0.58, 0.72; 0.8, 0.8))	1.5685	0.3809
VR	((0.3, 0.4, 0.62, 0.78; 1, 1), (0.32, 0.41, 0.6, 0.75; 0.8, 0.8))	2.5498	0.6191
Perceived Usefulness
Flat	((0.29, 0.38, 0.58, 0.74; 1, 1), (0.3, 0.4, 0.55, 0.71; 0.8, 0.8))	1.2643	0.2424
VR	((0.24, 0.35, 0.63, 0.88; 1, 1), (0.27, 0.38, 0.6, 0.82; 0.8, 0.8))	3.9505	0.7576
Ease of Recognition
Flat	((0.32, 0.4, 0.55, 0.67; 1, 1), (0.34, 0.41, 0.53, 0.64; 0.8, 0.8))	1.2810	0.2457
VR	((0.28, 0.37, 0.6, 0.78; 1, 1), (0.3, 0.38, 0.57, 0.73; 0.8, 0.8))	3.9323	0.7543
Immersion
Flat	((0.33, 0.36, 0.45, 0.51; 1, 1), (0.33, 0.37, 0.44, 0.5; 0.8, 0.8))	1.2320	0.2255
VR	((0.28, 0.33, 0.49, 0.6; 1, 1), (0.28, 0.35, 0.48, 0.58; 0.8, 0.8))	4.2315	0.7745
Realism
Flat	((0.3, 0.35, 0.45, 0.51; 1, 1), (0.31, 0.36, 0.43, 0.48; 0.8, 0.8))	1.2308	0.2198
VR	((0.24, 0.32, 0.49, 0.64; 1, 1), (0.26, 0.33, 0.46, 0.61; 0.8, 0.8))	4.3695	0.7802
Weight
Flat	((0.24, 0.35, 0.59, 0.83; 1, 1), (0.27, 0.36, 0.56, 0.76; 0.8, 0.8))	2.8550	0.6443
VR	((0.18, 0.31, 0.67, 1.06; 1, 1), (0.21, 0.33, 0.62, 0.95; 0.8, 0.8))	1.5765	0.3557
Sickness
Flat	((0.41, 0.45, 0.5, 0.55; 1, 1), (0.41, 0.45, 0.5, 0.54; 0.8, 0.8))	2.9223	0.6633
VR	((0.38, 0.43, 0.53, 0.58; 1, 1), (0.4, 0.44, 0.51, 0.57; 0.8, 0.8))	1.4833	0.3367
Interaction
Flat	((0.29, 0.38, 0.58, 0.74; 1, 1), (0.3, 0.4, 0.55, 0.71; 0.8, 0.8))	1.2810	0.2457
VR	((0.24, 0.35, 0.63, 0.88; 1, 1), (0.27, 0.38, 0.6, 0.82; 0.8, 0.8))	3.9323	0.7543

.

Step-9—Sensitivity Analysis: To understand the effect of variation in weightages over the outcome of the analysis and to test the robustness of the model, sensitivity analysis was performed. Global weightages of the evaluation factors and their subfactors were varied one by one, by changing the priorities to observe the effect of changed priorities and subfactor weightages on the final selection of alternatives. This step is presented in more detail as part of Section 4.

4. Results and Discussion

To analyze the efficiency of safety training and to answer research questions 1 and 2, an analysis of reaction times was conducted as obtained from each participant by their joystick inputs. The purpose of analyzing this data was to understand which type of hazards were correctly identified by each participant and correlate it to the perception offered by display types.

As discussed in Section 3, each participant was shown four scenes during the simulation—two scenes using a flat screen and two scenes using a VR headset. They were instructed to provide input using a joystick to register the time of identification of a hazard during the simulations. The break-up of the total number of hazards identified by the participants is shown in Figure 7.

It can be observed that participants were able to correctly identify a greater percentage of total hazards with a VR display when compared to the flat screen display. Moreover, this trend is magnified when trying to understand the distance to electric cables, which seem like 2D objects when viewed from a greater distance. Participants could only correctly identify the location of 18.5% of electricity cables when using a flat-screen display but could identify the location of 78.3% of electricity cables when using a VR headset.

A graphical representation of reaction accuracy—i.e., timely identification of hazards is shown in Figure 8. Participants were able to identify hazardous situations before the incident occurred in most of the cases when using a VR headset; 0 in Figure 8 represents the time when the incident occurs. All positive values represent the time of evasive reaction in seconds before the incident, while negative values represent seconds elapsed after the incident has occurred.

Another area where significant improvement was the accuracy of hazard identification. It was observed that the number of false identifications dropped by 50% when using VR headsets. This is shown in Figure 9.

To answer research question 3 using the procedure described in Section 3.2, fuzzy weights of all the criteria and alternatives are calculated and used for ranking the alternatives with respect to each criterion and sub-criterion. For ease of comparison and ranking, the fuzzy weights were converted into defuzzified weights using Equation (2), and these crisp weights were then further normalized. Aggregated IT2F comparison matrix for main evaluation factors and sub-factors are tabulated in Table 2, Table 3, Table 4, Table 5 and Table 6, while calculated fuzzy weights, crisp weights, and normalized weights of the main criteria, sub-criteria, and alternatives are given in Table 7, Table 8 and Table 9.

Using the relative weights of factors and subfactors in Table 7 and Table 8, the relative importance of each factor and subfactor is obtained. Accordingly, it can be seen that Performance Factors > Efficiency Factors > Device Factors > Presence Factors is the order of relative importance of evaluation factors. Using the local weights of subfactors and multiplying them with the priority weights of evaluation factors, the global weights of all subfactors were achieved, as shown in Figure 10. The same methodology is used to rank the subfactors to obtain the local and global weights. Table 9 provides the fuzzy, crisp, and normalized weights of display alternatives with respect to all subfactors.

Among all the subfactors, it can be seen that Realism and Immersion were the highest-ranked subfactors with individual global weightage greater than 0.1. As can be seen in Figure 10, the subfactors were divided into three distinct categories—High (Red), Med (Yellow), and Low (Green) which correspond to weightages greater than 0.1, between 0.1 and 0.08, and less than 0.08, respectively. This categorization refers to the contribution of the said factor toward depth perception leading to efficient hazard recognition during the simulations.

Similarly, the global weights of alternatives are calculated by multiplying the local weights of alternatives by the global weights of subfactors assessed by the participants. As seen in Figure 11, the difference in global weights between the flat screen and VR headset is highest in the case of immersion and realism. Another interesting observation in Figure 11 is the flat screen is the alternative of choice for ease of operation, Weight, and Sickness subfactors.

Finally, to answer the research question regarding aspects of construction machinery operator safety training that can be improved using VR, the fuzzy sets corresponding to each of the alternatives are calculated and defuzzified using the DTraT method—Equation (2), to calculate their crisp weights. The final representation of IT2F sets for both the alternatives can be seen in Figure 12, where the defuzzified values are represented at the center of these type-2 fuzzy sets. It can be seen that after the application of the DTraT method, the global weights of alternatives are 3.36 for the VR headset and 1.42 for the flat screen. Eventually, it can be observed that all aspects related to presence factors—i.e., perception of immersion and realism which contribute to distance perception can be effectively conveyed for better hazard identification. Overall, it can be observed that a VR headset is more effective compared to a flat screen for virtual safety training.

To find the effect of MCDM analysis techniques on the obtained results, Buckley’s type-1 fuzzy AHP [63] method served as a control technique. This technique was also used in this section to assess the effects of the uncertainty in the type-1 fuzzy sets on the obtained results. Analysis of the same set of participant responses was also conducted using type-1 fuzzy sets. The weights obtained compared to IT2F-AHP analysis are presented in

Table 10. Comparison of Type-1 and IT2F-AHP results.

Subfactors	Type-1 Fuzzy AHP		IT2F-AHP
	Global Weights	Ranking	Global Weights	Ranking
Realism	0.1187	1	0.1181	1
Immersion	0.1187	2	0.1181	2
Ease of Recognition	0.0888	3	0.0888	3
Sickness	0.0863	4	0.0863	4
Objective Usability	0.0850	5	0.0860	5
Perceived Usefulness	0.0837	6	0.0847	6
Interaction	0.0788	7	0.0802	7
Weight	0.0751	8	0.0757	8
Response Accuracy	0.0692	9	0.0682	9
Ease of Operation	0.0672	10	0.0664	10
Enjoyability	0.0649	11	0.0639	11
Level of Engagement	0.0636	12	0.0636	12

. Both methods produce equivalent results with the same rankings for all the subfactors considered for this study. Accordingly, type-2 fuzzy AHP results are considered to be sufficiently robust, and the findings of this study are independent of the types of analysis techniques.

Further, sensitivity analysis was conducted to validate the outcomes and understand if any changes to the weights of main evaluation factors can affect the overall selection of display technology for training involving depth perception. For the analysis, we defined 21 cases by varying weights for evaluation factors and subfactors. The definition of the first nine cases was conducted by gradual weight increases for the main factors one by one. Next, the weights of subfactors to performance factors were varied by assigning high weightage to one subfactor and low weights to other subfactors. Similarly, weights for other subfactors were varied. The case definitions can be seen in

Table 11. Sensitivity analysis case descriptions.

Cases	Description
Case 1	Current Case
Case 2	Medium-high weight of Performance Factors, Low for other factors
Case 3	High weight of Performance Factors, Low for other factors
Case 4	Medium-high weight of Efficiency Factors, Low for other factors
Case 5	High weight of Efficiency Factors, Low for other factors
Case 6	Medium-high weight of Presence Factors, Low for other factors
Case 7	High weight of Presence Factors, Low for other factors
Case 8	Medium-high weight of Device Factors, Low for other factors
Case 9	High weight of Performance Device, Low for other factors
Case 10	High weight for Ease of Operation
Case 11	High weight for Enjoyability
Case 12	High weight for Level of Engagement
Case 13	High weight for Response Accuracy
Case 14	High weight for Objective Usability
Case 15	High weight for Perceived Usefulness
Case 16	High weight for Ease of Recognition
Case 17	High weight for immersion
Case 18	High weight for realism
Case 19	High weight for weight
Case 20	High weight for Sickness
Case 21	High weight for interaction

, and the variations in weights across different cases can be seen in

Table 12. Sensitivity Analysis Case Weightages.

Cases	Weightages
	Performance Factors	Efficiency Factors	Presence Factors	Device Factors
Case 1	0.26	0.26	0.24	0.24
Case 2	0.51	0.18	0.15	0.16
Case 3	0.76	0.09	0.07	0.08
Case 4	0.18	0.51	0.14	0.15
Case 5	0.10	0.76	0.06	0.07
Case 6	0.18	0.18	0.49	0.16
Case 7	0.10	0.09	0.74	0.08
Case 8	0.18	0.18	0.15	0.49
Case 9	0.10	0.09	0.07	0.74
	Ease of Operation	Enjoyability	Level of Engagement	Response Accuracy
Case 10	0.95	0.01	0.01	0.03
Case 11	0.02	0.94	0.01	0.03
Case 12	0.02	0.01	0.94	0.03
Case 13	0.02	0.01	0.01	0.96
	Objective Usability	Perceived Usefulness	Ease of Recognition
Case 14	0.93	0.03	0.04
Case 15	0.03	0.93	0.04
Case 16	0.03	0.03	0.94
	Immersion	Realism
Case 17	0.9	0.1
Case 18	0.1	0.9
	Weight	Sickness	Interaction
Case 19	0.91	0.06	0.03
Case 20	0.01	0.96	0.03
Case 21	0.01	0.06	0.93

.

The variation in the final weights of the alternatives was graphed to observe any overall changes to alternative selection. It can be seen in Figure 13a that for all but three of the cases, the weight of the VR headset remains higher than that of the flat screen. However, the weight of the flat screen exceeds that of the VR headset in Case 10, Case 19, and Case 20. This variation can be attributed to high weights for ease of operation, device weight and Sickness subfactors. Accordingly, it can be observed from Figure 13b that the ranking for flat screens exceeds that for VR headsets only in Case 10, Case 19, and Case 20.

It can be seen from the results that realism and immersion are two of the most important factors which lead to better hazard recognition among novice heavy-machinery operators. Looking at the global weights of alternatives for subfactors in Figure 11, this finding is further magnified as the weightages for both immersion and realism are highest for the immersive VR display. This finding conclusively proves that immersive VR displays can portray realism better than semi-immersive displays. This result demonstrates the greater effectiveness of VR-based training with immersive displays due to their ability to provide better depth perception.

On the other hand, it can be observed from Figure 11 that flat screen display was preferred by the participants over VR display within the Device Factors category (Weight and Sickness) and Performance Factors category (Ease of use). As a part of feedback, most first-time VR users were initially uncomfortable with the immersive environment as the simulations provided a feeling of moving around in a tower crane cabin while they were actually sitting on a stationary chair. Additionally, not being able to see the actual joystick with a headset on contributed to the flat screen being preferred for ease of use.

Additionally, the effectiveness and suitability of VR displays for virtual training are also fortified by statistical details of hazards correctly identified by the participants presented in Figure 7. Immersive displays were able to provide participants with sufficient depth of field to identify the distance to seemingly two-dimensional objects such as overhead electricity cables. Additionally, Figure 9 shows that the identification of critical hazards improved with an immersive VR display by more than 300% when compared to that with a flat-screen display.

This study was able to demonstrate the importance of depth perception in hazard identification safety training by comparing the performance of novice tower crane operators in immersive and semi-immersive virtual environments. It can be concluded that better training efficiency can be achieved with immersive virtual displays when imparting hazard identification training to novice heavy machinery operators.

5. Conclusions

In this study, a comprehensive set of factors that are key to safety training of heavy equipment operations were analyzed. A group of operators from a course on tower crane operation was identified and shown simulations of tower crane operations in a virtual environment using flat-screen displays and head-mounted VR displays. Their hazard identification performance was recorded using a joystick during the simulations. Finally, the participants were asked to respond to a questionnaire to record their responses towards key factors regarding hazard perception in safety training. Interval type-2 fuzzy AHP was used to analyze participant feedback and convert linguistic responses to quantitative data to rank the effectiveness of factors and subfactors. Further, sensitivity analysis was conducted to observe the variations in hazard identification and depth perception.

Data analysis of participant response times reveals that immersive training with VR headsets improves the perception of hazards, as participants were able to identify more hazards, with sufficient time before incident occurrence and with higher accuracy when compared to their performance using flat screen display. Further, a very large improvement of about 300% was observed in the timely identification of critical safety hazards such as electricity cables, which account for the greatest number of fatal accidents involving tower cranes.

Observing the results from Interval Type-2 Fuzzy AHP and those from the sensitivity analysis, it can be concluded that VR simulations can provide effective training for heavy machinery operators when a VR headset is used as the display type for the training. Among the main evaluation factors, performance factors have the highest ranking as rated by the participants, followed by efficiency factors, device factors and presence factors. The IT2F AHP results highlight the areas related to immersion and realism that can be strengthened by using an immersive virtual environment. These include depth perception, spatial awareness and familiarization with construction sites. There were only three subfactors out of the total twelve subfactors—Ease of use, Sickness and Weight, where participants rated flat screen display better than VR headset for the training experience.

Sensitivity analysis reveals that higher weights across all the levels of the hierarchy can change the alternative preference but increasing the weight of only main evaluation factors or subfactors do not alter the preference for alternatives.

Combing the feedback from participants and data generated during the experiment, this study is able to demonstrate that depth perception conveyed by immersive VR displays such as head-mounted VR headsets can increase the efficiency of virtual safety training. Virtual safety training with HMDs can be used as an effective tool to increase awareness and hazard perception ability among novice heavy-machinery operators across most hazard types. This practice can drastically reduce injuries or fatalities on construction sites by reducing accidents caused due to lack of operator experience as novice operators can be trained in a virtual environment and can be taught effective ways of hazard identification by their experienced mentors. Acquisition of operator skills by novice operators and their safe implementation on construction should be continuously assessed in addition to safety training.