A Study on the Analysis of the Effects of Passenger Ship Abandonment Training Using VR

: Virtual reality (VR) has been adopted for education and training in numerous lines of work. In the ﬁeld of maritime education and training (MET), training with VR has been implemented in areas such as engine room system familiarization, dangerous work onboard familiarization, and emergency response. In addition, in line with current trends in maritime autonomous surface ship (MASS) technology developments, trial applications of VR have been implemented to understand the applicability of this type of vessel. Moreover, there is an increasing need to reinforce educa-tion/training in MET to reduce ship accidents. The purpose of this study is to investigate the level of satisfaction with VR education/training, the impact of that education, and matters requiring training emphasis for implementation. To this end, the training participants’ satisfaction with VR training was compared to that of video training, and items of strategic importance in the training content were identiﬁed using Borich’s needs assessment, after which the educational impact of VR training was compared to that of video training. The results of this study are expected to contribute to the extended application of MET using VR, and can help in identifying areas for emphasis in training to reduce maritime accidents.


Introduction
Virtual reality (VR) has been employed for education and training in various fields, including medicine, aerospace, entertainment, and education [1][2][3]. It enhances students' achievements in engineering education [1] and it has also been applied in anatomy education due to its proficiency in visualizing objects and moving to various locations [1]. Its positive impact confirms it as an effective training enhancement method [4]. In particular, using VR in education and workforce training has the advantage of resolving accessibility issues for places or systems that cannot be easily accessed [5]. VR's use in education and training has been widely adopted in vocational training and university education; in real-life applications, it has been frequently employed in education and training related to dangerous work [6]. In the field of maritime education and training (MET), educational content using VR is under active development. A previous study reported that VR was somewhat helpful to learning when applied to vocational safety training [7], and technology for the application of VR in the maritime safety training of seafarers for onboard use has been proposed [3]. Moreover, with the development of maritime autonomous surface ships (MASS), various studies utilizing VR for the operation of this class of ship have been conducted. These include research into the use of VR for the supervision of a MASS's operational status [8], the reproduction of an engine room using VR to provide engine-room system-failure scenarios to engineers [9], the application of VR for situational awareness (SA) in the operation of a MASS [10], and a study on the operation of unmanned surface vessels (USVs) through VR [11]. Moreover, in situations where it is necessary to improve and reinforce training as a method of resolving problems identified in maritime accident analysis [12][13][14], VR-based training has its merits, in that it enables safe training delivery in potentially dangerous situations, providing various types of training in a relatively short period of time. Clearly, the use of VR is expanding to many different fields based on its myriad advantages-primarily that of reducing risks by affording users a virtual experience of dangerous work. However, there have been a few studies on the effects of VR-based education and training in shipboard emergency response training. Since VR is recognized as an important technology in MET and in the operation of MASS, a study on the impact of education and training in these areas will be of great benefit in the application of VR to MET and MASS.

Literature Review
Research on the effects of training using VR was conducted in the field of medicine prior to any research in the maritime industry; that is, a training system using VR and a video trainer (VT) for surgery skills training was studied, the results of which showed VR-based training to be more effective [15]. It has also been reported that immersive VR through cognitive training is effective for children and adolescents with attention deficit problems [16]. VR training for acquiring surgical skills has been shown to be effective in reducing the training time required for learners [17]. Another prior study confirmed that VR-based education and training can be applied in different areas for various purposes, including education, military, sport, and medical training [18].
VR-based education and training has a wide range of applications in the field of MET [19]. As a representative example, a previous study confirmed VR-based training to be effective in enhancing the effects of education and delivering expert knowledge in the maritime sector [20]. In particular, studies on the advantages and disadvantages of providing education on the various control systems of a ship showed that VR-based training could be used to recognize the behavioral characteristics of seafarers [21], with the educational impact having been confirmed to some extent [22]. Another prior study showed virtual command bridge training to be cost-effective, with demonstrable educational impact when compared to the currently used simulation system [23]. Furthermore, since VR affords easy and convenient provision of various training programs-for example, in relation to bridge command, engine rooms, cranes, and fire safety-studies have suggested that education and training methods using this type of approach will play a key role in the future [24]. To enhance the effectiveness of education in MET, a study was conducted which proposed the use of a combination of VR and operator training simulators (VR-OTS) rather than VR-based training alone [25].
The educational impact of VR-based MET has been clearly demonstrated in the area of safety training. One of the most basic programs required for seafarers is familiarization training of a ship's structure. To provide such training in practice, problems such as cost, the difficulty of establishing an optimized plan, and the experience of the training facilitator are common; VR, however, can be effectively adopted in this type of familiarization training [26]. Moreover, advanced VR-based safety training in the offshore and maritime industries enhances the safety of workers in emergency situations [27]. In addition, VR can be used in search and rescue (SAR) training to support responses and decision making for SAR operators [28,29]. Another prior study showed that the use of multiple VR devices for firefighting training-a type of training most frequently performed on ships-allowed the participation of many people simultaneously, with effective educational outcomes through strong interactions and cost reduction [30]. VR-based training is also effective in the use of ship life-saving appliances-in particular, it is expected to be effectively used in training for free-fall lifeboats-where real-life training can be dangerous [31].
Another study confirmed that the use of VR in enclosed space entry training-the primary cause of casualties on ships-could ensure the safety and improve the performance of the training participants [32]. In addition to VR's application in safety training, a study was conducted showing that VR could be used to create a marine engineering English teaching and learning environment to enhance learners' levels of concentration and reduce the burden of English learning [33].
As can be seen from these previous studies, to apply VR, augmented reality (AR), and mixed reality (MR) to MET, evidence-based research is needed to identify the merits, drawbacks, and limitations of these technologies [34]. However, are only a few studies that quantitatively determine how effective VR-based training is for seafarers in the maritime sector. In particular, it has been even more difficult to find cases where various drills are performed on a regular basis onboard ships.
According to the International Standards of Training, Certification and Watchkeeping for Seafarers (STCW 1978), as amended, there are a number of training and educational courses that seafarers are obliged to complete [35]. Among these courses, in relation to requirements for safety training of personnel providing a direct service to passengers and passenger ship crowd management training-as specified in the STCW Code Part A/Section A-V/2-the IMO Model Course 1.41 (Passenger Ship Crowd Management Training) specifies the requirements for passenger ship crew to learn emergency passenger management skills through education and training. As can be seen from the Costa Concordia accident in 2012 and the Sewol ferry accident in 2014, passenger ship crews need to be trained to respond to emergencies at all times [12,36,37]. Unlike ship abandonment training on other types of ship, the importance of crew training with passengers has been emphasized [38]. There are, however, a number of practical difficulties in mobilizing passengers to conduct training during an actual passenger ship voyage-as well as the need to consider the safety of training participants during training-which lead to limitations in conducting training onboard a ship. Moreover, passenger evacuation and ship abandonment training actually conducted on passenger ships have been shown to have a reduced training effect due to the practical difficulties of mobilizing passengers and the use of life-saving appliances such as marine evacuation systems (MES) and life-rafts during training [39]. It can be anticipated-based on previous studies-that passenger ship abandonment training using VR will be effective in terms of safety [32]. However, the impact of VR-based training used in this case needs to be examined in more detail. Furthermore, it would be necessary to identify which items should be emphasized in the implementation of such training.

Materials and Methods
In this study, we aimed to quantitatively determine the impact of VR-based passenger ship abandonment training by means of a comparison with video-based training methods.
For video and VR content development, the scope of content was determined based on the IMO Model Course 1.41. "Assist passengers en route to assembly and embarkation stations" was selected from the contents of the Model Course 1.41 and developed as video and VR training content. Thereafter, the participants were divided into two experimental groups: 30 participants in Group 1 who were trained using video clips, and 30 participants in Group 2 who were trained using VR. Next, a survey was conducted in three areas: an area related to education and training satisfaction, an area related to the understanding and importance of education and training, and an area to check the participants' understanding of the training procedure.
Education satisfaction was investigated by assessing the overall satisfaction of the two groups with the training methods using a 5-point Likert scale, and Borich's needs assessment [40] was used to analyze the importance and performance of the training; the latter was expressed by the locus for focus models [41], and these models have been used in the field of education to identify changes in the education system and to conduct education demand analysis [12,42,43].
Finally, to establish whether participants understood the training procedure, they were examined on the related procedures. The examination was conducted immediately after the video and VR training-in accordance with the requirements of the 1978 SOLAS Convention that ship abandonment training should be performed once a month-and participants were examined on the same content a month later. The results between Group 1 and Group 2 were then compared. To analyze the results, data preprocessing and analysis were performed using Python, statistical analysis was performed using SPSS, and data were visualized using OriginPro. The experiment participants were ship officers who visited the Korea Institute of Maritime and Fisheries Technology (KIMFT) to undergo training. Figure 1 outlines the method and procedure of this study.

Development of Passenger Ship Abandon Training Content
According to the IMO model course 1.41, the following training content needs to be covered [44].
As shown in Table 1, among the items required in the model course 1.41, "2. Assisting passengers en route to muster and embarkation stations" was divided into 20 elements and steps, as shown in Table 2 (below). The video and VR content necessary for the experiment is described in the table. Table 1. Passenger ship crowd management training subject area.

1.
Contributing to the implementation of shipboard emergency plans and procedure to muster and evacuate passengers Ability to disembark passengers, with special attention to disabled persons and persons needing assistance  Figures 2 and 3, respectively. Figure 2 shows the partially captured video content and Figure 3 shows a part of the VR content. The equipment used for the experiment was the HTC VIVE VR Pro. A wireless kit was added so that participants in the actual experiment were not disturbed by cables when using the VR equipment. Figure 4 shows the video and VR training concept, as well as the equipment used for this study.

Participants' Education Satisfaction
A total of 60 participants took part in the experiment, with 30 participants using video training (Group 1) and 30 participants using VR training (Group 2). The positions of the participants comprised 38 third mates, 4 second mates, 1 chief mate, 15 third engineers, and 2 second engineers; 88% of the total participants were third mates or third engineers.
As for the age group composition, 48.33% of the participants were in their 20 s, 48.33% were in their 30 s, and 3.33% were in their 40s.
To examine the education satisfaction of the participants, nine questions were assessed using a 5-point Likert scale: (1) difficulty in use, (2) understanding of the content, (3) understanding of the procedures, (4) understanding of the roles, (5) realism, (6) efficiency of education, (7) effectiveness of education, (8) positive response to use of VR system for training, and (9) opinion on extending VR training courses in the maritime industry. As a result, the education satisfaction using the video-based training method scored 3.79 points, while the VR-based training method scored 4.32 points; that is, the seafarers participating in the study demonstrated a higher level of satisfaction with the VR-based training methods.
However, in terms of the difficulty of use, the score of the video-based training method was 4.17 points, whereas the score of the VR-based training method was 4.33 points, indicating more difficulty in use of the VR-based training method. Moreover, in terms of the extension of training using the relevant equipment, the video-based training method scored 3.80, whereas the VR-based training method scored 4.57 points. This confirms that most of the participants expressed a strong desire for an extension of training using the VR-based training method.
Furthermore, in terms of the understanding of content, understanding of procedures, understanding of roles, realism, efficiency of education, effectiveness of education, and positive response for use of the VR system for training, the VR-based training method received higher scores. Figure 5 shows the results of the education satisfaction comparison between the two methods.

Borich Needs Assessment and Locus for Focus Model
To check the statical significance of the importance and performance, a paired sample t-test was performed. If the p-value < 0.05 was satisfied, the alternative hypothesis could be adopted and applied to the Borich's needs assessment and locus for focus model. Table 3 shows the t-test results. It is confirmed that Borich's needs assessment and locus for focus model can be applied based on the results in Table 3.  The results presented used the locus for focus model after Borich's needs assessment was conducted to assess the differences in importance and performance. Equation (1) shows the formula for Borich's coefficient [12]: N: Denotes total number of cases; RCL: Denotes required competence level (importance level); PCL: Denotes present competence level (performance level); RCL: Denotes average of RCL (importance level).
By increasing the RCL value and decreasing the PCL value, the Borich's needs coefficient increased based on Equation (1). The calculated Borich's needs coefficient provides a basis for judging priorities using relative values instead of absolute values. Mink's locus for focus model divides priority areas into quadrants based on the mean value of the importance level (horizontal axis) and discrepancy (vertical axis) based on the current level of importance and performance [41]. Accordingly, it should be emphasized that items with a higher priority have a high Borich's needs coefficient, which should be corrected, if possible. Figure 6 shows a schematic representation of Mink's locus for focus model [12]. This study focused on the first quadrant (HH), which represents a high discrepancy between importance and performance, respectively. Figure 6. Locus for focus model [12]. Table 4 outlines Borich's needs values for 20 questions on video-based and VR-based training, respectively. Of the 20 items on the passenger ship abandonment training list, those that exhibited high importance but low performance from the results obtained from the seafarers participating in Group 1 (video), as well as Borich's needs values used for analysis with the locus for focus model, were Q1, Q2, Q3, Q5, Q11, Q13, and Q20. These seven items were identified as being of high importance and low performance in Figure 7 (up).  From the results obtained from the seafarers participating in Group 2 (VR), seven items of Q1, Q5, Q13, Q15, Q16, Q19, and Q20 were identified as items of high importance and low performance Figure 7 (down). The items that were found to be common in both Group 1 and Group 2 were Q1 (understanding and implementation the priority actions of duty officer in the event of an emergency on a passenger ship), Q5 (familiarity and implementation of the crew's behavior while guiding passengers to a safe place in case of an emergency), Q13 (understanding and implementation of the correct abandonship preparation on the passenger ships equipped with life-rafts and MES), and Q20 (understanding and implementation of launching procedures of a life-raft). This indicates that the training for these items needed to be emphasized more in both the video-based and VR-based training methods. Apart from these overlapping items, more reinforced training was required for items Q2, Q3, and Q11 in the video-based training, and more reinforced training was required for items Q15, Q16, and Q19 in the VR-based training.

Comparison of Educational Effects of Video and VR Training
Immediately after the training, 20 examination questions were prepared in relation to the contents of the 20 items on the passenger ship abandonment training content listed in Table 2; the participants in Group 1 and Group 2 were then examined based on their knowledge of the items. In the examination immediately after training, the score of Group 1 (the video-based training group) was higher by 2.3 points (on average) than that of Group 2. After 30 days, the 60 participants in Group 1 and Group 2 took an examination with the same questions; the score of the group with the video-based training method was 68.7 points, whereas the score of the group with the VR-based training method was 71.5 points. Clearly, the score of the VR Group 2 was higher by 2.8 points.
Regarding the decrease in scores over time, after 1 month, the "retention of knowledge" was higher in the VR group than in the video group; that is, the score difference was 3.5 for the VR group, whereas the score difference was 9.1 for the video group, confirming that the long-term effect on the training content using VR-based methods proved to be more effective. This result shows that training using VR was equivalent or more effective compared to the video-based training currently employed in MET. It can be argued that, in some respects, VR-based training serves as a superior training tool. Figure 8 shows the first and second results of the examinations of Group 1 and Group 2.

Discussion
In this study, experiments were conducted to investigate the education satisfaction of VR-based training, identify the items that need to be emphasized in the training, and confirm the impact of VR-based training. To this end, the participants were divided into Group 1 (video) and Group 2 (VR), with the same training content being provided in a different method. The resulting impact was comparatively analyzed.
In the case of education satisfaction, the results confirmed that education satisfaction using VR was considerably higher in all areas, except for "difficulty in use". However, it needs to be considered that most of the participants in this experiment belonged to a relatively young age demographic, and they exhibited no particular reluctance to use VR. Consequently, if the method were to be applied to seafarers in a relatively higher age group (50 or older), different results could be obtained in terms of education satisfaction [45,46]. Therefore, the higher score of education satisfaction in this study has a limitation in that this is a result of applying VR to seafarers in a relatively young age group.
Moreover, from the nine questions on education satisfaction, participants were of the opinion that training using video was more satisfactory in terms of understanding the passenger ship abandonment procedure. This can be understood to be a result of people familiar with conventional training methods showing their preference for more familiar training methods. In addition, with regard to the advantages of training using VR, the participants mentioned that VR with passengers was possible, the virtual space was similar to a real ship (which was effective in the training), accurate understanding of the procedure was possible, and there would be no injuries during training. Conversely, drawbacks that participants pointed out were the difficulty of using VR in the case of poor eyesight or unfamiliarity with the technology in the case of older participants [45,46]. In other previous studies, there have been cases where a learner felt dizzy when using VR for a long period of time [34] but, in this experiment, both Group 1 and Group 2 participated in the training for approximately 10 min, so this problem did not occur.
Through Borich's needs assessment and locus for focus model, it was possible to identify which parts of the training needed to be emphasized more. In this way-by means of ship accident analysis-the requirements for the implementation of training that reinforced the parts that were identified to be weak content areas in seafarer training could be met, instead of having to provide one-off seafarer training based on the requirements of the STCW 1978 convention [12]. In both video-based and VR-based training, understanding and practice of the highest priority actions of a duty officer on the bridge in the event of an emergency on a passenger ship, understanding and practice of the crew's instructions while guiding passengers to a safe place, clear understanding and practice of the abandon-ship preparation procedures on passenger ships equipped with life-rafts and MES, and a clear understanding and practice of life-raft dropping and preparation procedures on passenger ships were confirmed to be the areas that needed to be emphasized in passenger ship abandonment training.
This indicates that, in an actual passenger ship abandonment process, knowledge of the use of abandon-ship procedures and equipment is thought to be most important; however, the participants did not believe that they had appropriate levels of competence in these areas. This finding could be considered to be a point that should be reflected by actual seafarer training institutions and ships. Moreover, in the video-based training, reinforced training was required for Q2 (understanding and implementation of how to notify passengers of the occurrence of an emergency for the first time and what should be informed), Q3 (understanding and implementation about the external agency that should be reported first in the event of an emergency on a passenger ship), and Q11 (understanding and implementation of what the crew must check before boarding the MES). In the VRbased training, Q15 (understanding and implementing the most correct action when there is a missing person while identifying the number of people before abandoning the ship), Q16 (understanding and implementation of the role of the crew in the muster station), and Q19 (understanding and implementation of drop methods of a life-raft) need to be enhanced. In the video-based training, the importance of accurately explaining the necessary procedure to passengers in the event of an emergency requiring abandonment was emphasized while, in the VR-based training, the importance of the role of the crew in the emergency muster station and life-raft use procedure was emphasized.
It is thought that this result was obtained in relation to video-based training, and that this training's purpose was to guide passengers to abandon ship in the event of an emergency; this has been emphasized in the theory. However, in the case of VR-based training, the role of the muster station was not accurately implemented in the VR content that was developed for use in the experiment. Therefore, the associated items Q15, Q16, and Q19 were emphasized. There is thus a limitation, in that the scenario developed for this experiment could not be considered to include all procedures implemented on a real ship. This is because the production of the video and VR content for this study was meant for research rather than for practical application in the industry. Consequently, it is necessary to expand this study and verify the impact of education by implementing education and training with VR on various types of ship.
In terms of the applicability of VR system for MASS operation, remote operators will have access to traditional navigation systems enhanced with VR and augmented reality (AR) content [10]. When such information is displayed on a screen of in VR, there are several benefits compared to traditional tools [8]. Therefore, the results identified through this study will be helpful in applying VR to MASS operations. Notably, the inconvenience of using a VR device and the fact that it is helpful for long-term memory can be referred to.
The results of comparison between the examination results of Group 1 and Group 2 confirmed that VR-based training was effective in terms of long-term memory and recollection of related procedures and knowledge [32,47]. Consequently, the passenger ship abandonment VR-based training was confirmed to be more effective than that using video. Since predictions of the time and place of ship emergencies is not possible, long-term recollection of the contents of training may be helpful in taking action in the event of an actual emergency.
However, these results may differ depending on the experience on board ships and the types of ships themselves. Moreover, the results of this study have limitations, in that they depend on the experiences and knowledge of the participants in this experiment.
Similarly, the experiment was not performed with seafarers older than 50 years of age, and there may be limitations in the application of training using VR for seafarers in this age group due to problems related to using the equipment [45,46].

Conclusions
In this study, we investigated the education satisfaction and educational impact of passenger ship abandonment training using VR with comparisons to conventional training methods using video. In addition, areas for emphasis in passenger ship abandonment training were examined using Borich's needs assessment. It was confirmed that VR-based training was significantly better than video-based training in terms of its learning impact and participants' preference for the training method. In particular, in terms of training preference, training using VR showed higher scores in all items surveyed. However, the difficulty of using VR equipment remains a challenge to overcome in the implementation of VR-based training. With regard to areas to be emphasized in the training content, the common points of emphasis in Group 1 and Group 2 were Q1, Q5, Q13, and Q20, indicating that the passenger ship crew must be familiar with response procedures in the event of an emergency, and understand exactly how to use related equipment such as life-rafts, the MES, and life-boats. Lastly, it was confirmed that the VR-based training was somewhat superior to the video-based training in terms of memory and recollection of procedures related to passenger ship abandonment training. Seafarer training using VR could be extended to other areas, since the VR training equipment is relatively compact and VR-based training methods are not subject to significant time constraints. However, ease of using the equipment and discomfort when it is used for a long time are challenges in VR-based training that still need to be overcome. To overcome these difficulties, the development of VR equipment that can be used without difficulty or inconvenience by users of various age groups is essential. The advent of MASS calls for continuous research that reflects the limitations of VR identified in this study, and it also calls for the proper implementation of education and training in relation to MASS operation using VR.