Efficiency Comparison between Audible and Buzzle Alarms of Electronic Chart Display and Information System Alarm under the Simulated Environment

Abstract

Along with the development of ship navigational equipment, ship operators have to process a larger amount of information than before and be exposed to more alarm sounds. These ships’ bridge environment increases burdens to ship operators. One of the methods proposed to solve this problem is the audible voice alarm method. However, there is a lack of studies that objectively prove the efficacy of the method. Therefore, in this study, a comparative experiment was performed to confirm the effect by applying the method to an electronic chart display and information system (ECDIS), a representative navigation instrument. We analyzed collected data according to a data-driven process and confirmed the difference between a traditional alarm method and the audible voice alarm method by distinguishing groups through clustering.

1. Introduction

Reducing navigational accidents is crucial to improving maritime navigational safety [1]. Consistent efforts to systematically support ship operators’ decisions continue to develop enhanced navigational equipment [2]. An electronic chart display and information system (ECDIS) is a representative navigational tool from the effort of systematic development of navigational equipment [3]. This development of navigation equipment provides the operator with a lot of information related to safe navigation and supports an officer on watch(OOW) to make better decisions [4]. In addition, it helps the operator to quickly recognize the danger by providing an alarm for dangerous situations and malfunction of the ship’s equipment [5].

However, this systematic development of navigational equipment provides a large amount of information to ship operators as well as disturbs the ship operators owing to flood of alarm [6]. Alarms generated from many devices require the ship operator to take repetitive actions to confirm the alarm. Accordingly, the ship operator feels fatigued owing to these frequent alarms and gradually becomes insensible to the alarm or some operators mute the sound of alarms and set an improper threshold [7]. Although it has been found that maritime accidents are caused by a lack of situational awareness in various studies, the demands for a more effective alarm delivery method are raised because the alarm of the ship bridge still does not induce sufficient situational awareness for the navigator [8,9].

Research on the audible display of alarms has been conducted in various fields such as nuclear power research institutes, the aircraft industry, and hospitals. In the maritime field, there have been studies suggesting the application of auditory icons (environment sound alarm) instead of the existing abstract sound and studies to suggest the application of voice alarm to ECDIS [10,11,12].

Hence, this study focused on voice alarm, which has been applied to alarm systems since the early 1990s in the aviation field, as one of the alternative alarm methods suggested in previous maritime domain studies [13]. However, there is a lack of research into the efficacy of the application of the voice alarm to the ship’s bridge system.

Therefore, this study aimed to confirm the objective effect of audible voice alarm through comparative experiments. We collected data through simulation experiments using the traditional alarm method and an audible voice alarm method. The collected data were analyzed through a data-driven process; consequently, the effect of the audible alarm method was verified by distinguishing two groups through clustering.

2. Methodology

The workflow of this study is presented in Figure 1. The simulation log data and direct observation data were obtained during a scenario-based simulation experiment to explore objective differences due to the different types of ECDIS alarm methods. Feature extraction and selection were performed after preprocessing; then, data clustering was performed.

2.1. Data Collection

2.1.1. Simulation Environment

The experiment was performed in the full-mission-ship-handling simulator at the training ship SEGERO of Mokpo National Maritime University (Figure 2). The simulator uses the K-sim navigation’s full mission simulator of Kongsberg, which is suitable for objective data collection, as it can design detailed scenarios and record various information [14].

2.1.2. Scenario Design

The scenario was designed as shown in Figure 3 after brainstorming with experts, conducting a preliminary experiment, and considering the following variables.

• Independent variables

As the experiment is to verify the different results in navigation for different types of ECDIS alarm, the independent variable was set to types of ECDIS alarm in similar navigation situations. To select the type of alarm, alarms such as deviation from route, crossing safety contour, and closest point of approach(CPA) alarm were considered. For efficient data collection, the experiment time should be properly secured, and the action for the vessel maneuver should also be appropriately performed after the alarm is recognized; therefore, the collision-related alarm that has the urgent priority was selected as the type of alarm. In scenario (a), the alarm method was a traditional buzzer sound, whereas in scenario (b), the alarm method was an audible voice that gave the alarm ‘Dangerous CPA’ three times.

• Dependent variables

The dependent variables were set to time-domain variables, such as the response time of the navigator and a ship-maneuvering variable, which is a change in maneuvering characteristics.

• Control variables

The target ship (objective ship) was initially placed in a blind area covered by an island that cannot be observed with the naked eye so that the navigator’s risk recognition could begin from the occurrence of an alarm. In addition, to prevent recognition by monitoring ECDIS before an alarm occurs, participants were asked to write a logbook prepared in the simulator immediately after the simulation started. Therefore, the participants could recognize the risk of a collision from an alarm. Further, external forces that could affect ship maneuvering were controlled.

Scenario (a) is a scenario in which the traditional buzzer method was employed; the subject ship was northbound, and the target ship approached from the starboard, but the existence of the target ship could be checked only by ECDIS.

Scenario (b) is a scenario in which the audible voice method was employed; the subject ship was southbound, and the target ship approached from the port, but the existence of the target ship could be checked only by ECDIS. The distance and speed of approaching ships were the same, except that the approach direction of the target ship was symmetric.

2.1.3. Experiment

The experiment procedure was in the order of participant recruitment, experiment guidance, and experiment.

• Participant recruitment

The participants for the simulation experiment were recruited among deck cadets who were attending the third grade of the Mokpo National Maritime University with more than two months of boarding career. The statistical data of the participants are shown in

Table 1. Participant characteristics.

Characteristics	Mean (SD)
N	22
Female/male	8/14
Age (years)	21.5 (0.7)
Boarding experience	3.5 (1.4)

.

• Experiment procedure

Figure 4 shows the procedure of the experiment. All subjects participated in the experiment after guidance, including the purpose and reward of the study, through orientation before experiments (a). Each participant performed experiments under scenarios buzzer and audible voice alarm (b). To prevent the learning effect of the experiment, the sharing of information about the scenario among participants was controlled [15].

2.1.4. Collected Data

Collected data were simulation log recorded in simulation software (e.g., environment, ship maneuvering, and ship’s movement) and direct observations were recorded by the observer monitoring the participants during the experiment (e.g., time of alarm recognition). The frequency of the simulation log was set to 1 s so that the data are not unnecessarily large because the ship movement characteristics do not change rapidly like the human body movement [16]. The data from the experiment performed 44 times were collected in a table, and experiment’s data were converted into a cell array to facilitate data analysis.

2.2. Data Examination

We performed a data examination to confirm data consistency and quality. In this process, each variable was examined through data visualization as the first step for subsequent analysis [17]. In addition, we performed exploratory data analysis in parallel. Significant data patterns were also identified through this process.

2.2.1. Types of Data

The data collected through direct observation were time data collected in seconds for each section at the start of the experiment, alarm occurrence, recognition time, reaction, and end of the experiment. The data of the simulation log included variables for the subject ship, and other variables were continuous numerical data types, except for the nominal categorical data that recorded the use of timestamp and whistle. For each variable, different units were used according to the characteristics of the variable (e.g., m/s, m, degree, NM, degree/min, knots, kw, and %).

2.2.2. Outlier and Missing Values

The descriptive statistics of variables excluding the timestamp and variables with a fixed value were examined. Variables with fixed values were those variables whose values were fixed at zero because they were not used during the experiment, such as autopilot course setting and bow thruster setting values. As they had no meaning as variables, they were identified as variables that need to be removed in the preprocessing. Other variables showed different median values and distributions; we also identified outliers.

2.3. Data Preprocessing

2.3.1. Time Trimming

As this study examines the effect of the audible alarm method, the dataset was trimmed after the corresponding timestamp based on the alarm occurrence time recorded in the direct observation.

2.3.2. Time Cleaning

All timestamps were set to be collected at 1-s intervals. However, time cleaning was required because duplicate timestamps existed. We removed duplicate timestamps by selecting the data located in the first row among the same timestamp [18]. After time cleaning, all time-series data formed a dataset with regular time intervals.

2.3.3. Variable Handling

• Removing variables

As mentioned in the data examination, we removed variables with fixed zero values.The removed variables were 29 of 76, which were logs of ship control simulator functions unused during the experiment, such as autopilot control, bow/stern thruster control, and whistle signal control. In addition, although the navigation situations in scenarios (a) and (b) were similar, the external environmental variables differed. Therefore, we removed four under-keel-clearance-related variables and three distance-related variables, which were affected by environmental factors.

• Transform variables

Variables such as heading and course, which mean the ship’s moving direction, were collected in degree units ranging from 0° to 360°. As the subject ship’s directions of the two scenarios were designed to be opposite to remove the ordinal characteristics owing to the size of the value, we transformed the corresponding variables to the amount of change in degree. The initial value was set to 0. Then, other values of the variable were converted into a variable indicating a change with respect to the initial value by calculating the differences from the initial value.

• Creating variables

We expected that the distance from the target ship would also be meaningful in distinguishing the two experimental groups, so we created a distance variable from other ships, calculated using the Euclidean distance. As a result of checking the calculated distance, it was confirmed that the initial distance and the nearest distance between the two ships were different in the scenario of the two experimental groups. As distances cannot be simply compared under these conditions, we converted the distance variable to the amount of change in the decrease in distance to eliminate the difference between groups due to the difference in the initial value. After creating a distance variable, we removed the position variable of the subject and target ship’s data.

2.3.4. Outlier Handling

• Contextual outlier

As the data of each experiment participant were continuous time-series data, contextual outlier detection was employed [19]. We used the sliding window to consider the trend of the data for outlier detection [20]. We set the window size to 5, so two rows before and after in the corresponding row were selected as windows, and the sliding length was set to 1. The outlier threshold was set to 3 median absolute deviations, and data points exceeding the threshold were replaced with the nearest threshold value [21].

• Global outlier

The data comprise two experimental group datasets. As these two datasets were collected under different experimental conditions, we performed outlier handling for each group. We identified outliers by employing the quartile outlier detection method and replaced the outliers with the nearest outlier threshold [22].

2.3.5. Variables Obtained by Preprocessing

Data obtained from raw data through the process of refining meaningful data through data examination and variable removing, transformation, and creating of preprocessing were continuous time-series data related to the ship’s control, as shown in

Table 2. Variables obtained after preprocessing.

Number	Variable Description	Number	Variable Description
1	Acceleration (m/s)	21	Speed (knot)
2	Acceleration lateral (m/s)	22	Speed Lateral (m/s)
3	Acceleration longitudinal (m/s)	23	Speed Longitudinal (m/s)
4	Acceleration vertical (m/s)	24	Speed over the ground (knot)
5	Altitude (m)	25	Speed rate (m/s)
6	Course (degree)	26	Speed through water (knot)
7	Course over the ground (degrees)	27	Speed Vertical (m/s)
8	Course through water (degrees)	28	Surge (knots)
9	Draught Aft (m)	29	Sway (knots)
10	Draught Fore (m)	30	Trim (m)
11	Draught Mean (m)	31	Main Engine Power (kW)
12	Heading (degree)	32	Main Propulsor Engines Power (kW)
13	Latitude (degree)	33	Main Propulsor Pitch (degree)
14	List (degree)	34	Main Propulsor Pitch Order (degree)
15	Longitude (degree)	35	Main Propulsor Revolutions (rpm)
16	Pitch (degree)	36	Main Rudder Angle (degree)
17	Rate of pitch (degree/min)	37	Main Rudder Order (degree)
18	Rate of roll (degree/min)	38	Distance between target ship (m)
19	Rate of turn (degree/min)	39	Distance between target ship decrease rate (m/s)
20	Roll (degree)

.

2.4. Feature Extraction

We extracted features via two approaches. The first approach was to extract descriptive statistics of variables that showed differences through data visualization as features, and the second approach was to create features from the perspective of a ship handling expert using domain knowledge.

2.4.1. Data-Driven Features

We visualized the dataset obtained after preprocessing using a box plot as shown in Figure 5. Through this visualization, we could intuitively identify that descriptive statistics could be the practical feature that distinguishes the two groups.

Therefore, we calculated descriptive statistics for each variable and extracted statistical values that most effectively distinguished the groups’ features.

2.4.2. Knowledge-Driven Features

In the domain knowledge approach, we categorized items affected by the alarm delivery method into ‘agility to alarm’, ‘ship maneuvering’, and ‘correlation between variables’. Then, features were extracted for each category.

Table 3. Extracted feature list.

Domain	Index	Feature
Ship maneuvering	1	Max of acceleration lateral
	2	Standard deviation(STD) of acceleration lateral
	3	Max of absolute course change
	4	STD of absolute course change
	5	Max of absolute course over the ground(COG) change
	6	STD of absolute COG change
	7	Max of absolute course through the water(CTW) change
	8	STD of absolute CTW change
	9	Max of absolute heading change
	10	STD of absolute heading change
	11	Max of absolute rate of turn(ROT) change
	12	STD of absolute ROT change
	13	Max of lateral speed
	14	STD of lateral speed
	15	Max of sway
	16	STD of sway
	17	Max of absolute rudder angle
	18	STD of absolute rudder angle
	19	Max of absolute rudder angle order
	20	STD of absolute rudder angle order
	21	Average rate of change in distance reduction
Agility to alarm	22	Elapsed time (alarm to recognition)
	23	Elapsed time (alarm to reaction)
	24	Elapsed time (recognition to reaction)
	25	Elapsed time (rudder reaction to alarm)
Correlation	26	Correlation between rudder total usage and distance
	27	Correlation between heading change and distance
	28	Correlation between ROT and distance

is the extracted feature list including data-driven features and knowledge-driven features. The features were examined to confirm that the extracted features express the intended data properties.

2.4.3. Feature Selection

The number of extracted features was 28, which is a high dimension to input all features into the classification algorithm. High-dimensional input data are time-consuming to calculate, and they are inappropriate for generating a good model [23]. Therefore, to select the most compelling feature for the model, we performed feature selection.

• Correlation coefficient between features

We checked the correlation between the extracted features and identified overlapping features [24]. As the features were continuous numbers, the Pearson correlation coefficient was employed [25]. As shown in Figure 6, a high correlation (over 0.8) was identified between the variables (max of acceleration lateral~STD of absolute ROT change) related to the ship’s course change. Moreover, a high correlation between time-domain variables was confirmed. On the other hand, a negative correlation was confirmed between the variable related to the course change and the time domain variable.

If there are many strong correlations between features, the problem of multicollinearity is also a concern [26]. Therefore, it was necessary to select representative features among overlapping features and exclude the rest. The result of the correlation coefficient test was referred to when confirming the feature selection results.

• Stepwise regression

We employed stepwise regression as a feature selection method. Stepwise regression starts from a model that includes all variables, deletes the variable that is least helpful to the standard statistic, or adds the variable that improves the base statistic the most among the missing variables from the model, and repeats the addition or removal of these variables [27]. Various criteria were used to add or remove a feature: sum of squared error, Akaike information criterion (AIC), Bayesian information criterion, R², and adjusted R² [28]. By repeating feature selection and clustering, the feature selection criterion with the best clustering result was AIC.

Total ‘Ten’ features were selected (

Table 4. Selected features.

Domain	Index	Feature
Ship maneuvering	2	STD of acceleration lateral
	3	Max of absolute course change
	6	STD of absolute course over the ground change
	10	STD of absolute heading change
	12	STD of absolute ROT change
	14	STD of lateral speed
	15	Max of sway
	21	Average Rate of change in distance reduction
Agility to alarm	22	Elapsed time (alarm to recognition)
	23	Elapsed time (alarm to reaction)

); ‘Eight’ features were related to the maneuvering characteristics of the vessel, and the other ‘Two’ features were related to the agility characteristics of the alarm.

2.5. Clustering

The feature data were applied to clustering algorithms. The used clustering algorithms were K-nearest neighbor (KNN), K-medoids, and K-means algorithms [29]; we obtained the optimal result by performing parameter tuning.

3. Result

3.1. Result of Clustering

Table 5. Clustering accuracy.

Clustering Algorithm	Accuracy
KNN	75.0%
K-medoids	84.1%
K-means	90.9%

shows the results of the clustering algorithms. The results of clustering were validated by comparison with the labels recorded during the experiment. The algorithm that distinguished the two groups with the highest accuracy was the K-means algorithm, with 90.9% accuracy.

3.2. Result of Features

Features were visualized to check the features that affected the clustering results, as shown in Figure 7. The values in Figure 7 were normalized using min-max scaling because the unit and scale of each variable are different. Thus, a value of ‘zero’ means the minimum value among the two group values of the corresponding feature, and a value of ‘one’ means the maximum value. The blue and red colors indicate the buzzer and audible voice alarm groups, respectively.

Features 1–8 were the features of the maneuvering vessel domain and features 9 and 10 were the representative values of agility to alarm.

Features from 1 to 4, the distributions and median value of the groups, show a clear difference, so we can see that the maneuvering vessel characteristics of the audible alarm group were more aggressive than those of the buzzer alarm group. On the other hand, as for the remaining features of the maneuvering vessel domain, 5–8, it can be seen that the median values are closer than the features of 1–4 and the distributions are overlapped. So, the distinction between the two groups was relatively less noticeable compared with features 1–4.

Features 9 and 10 are the ability to alarm domains. What can be confirmed from the distribution of feature 9 is that, in the audible voice alarm group, all participants recognized the alarm at the same time that the alarm sounded, whereas in the buzzer alarm group, there was a relatively larger deviation in time to recognize the alarm between the participants.

In feature 10 (elevated time to reaction), the distribution of the buzzer group is wider than the audible voice alarm group and outliers close to a value of 1 (late reaction) occur, whereas in the audible voice alarm group, all participants reacted to an alarm within a relatively short time.

4. Discussion

Through the data-driven approach, we verified the difference in objective navigation performance between the group to which the traditional (buzzer) alarm delivery method was applied and the group to which the newly proposed (audible voice) alarm delivery method was applied. The K-mean cluster was divided into two groups with an accuracy of about 91%. The features used for clustering were ten features composed of ship maneuvering and agility to alarm domains.

When an alarm occurs, the vessel maneuvering characteristics of each group are clearly distinguished, which indicates that the audible voice alarm group showed more active collision-avoidance action against the vessel expected to collide than the traditional alarm group. These characteristics were found in features (1, 2, 3, and 4), which explain the course/heading change of the ship.

However, as the motion of the hull appears after the course change has continued for a certain period, it is assumed that the differences between the two groups are not clearly distinguished relatively in features that express ship motion (features 6 and 7).

The difference in vessel handling characteristics described above had an effect in changing the rate of distance reduction from the approaching vessel in consequence. As shown in feature 8, the audible voice alarm group effectively controlled the distance reduction from the opposite vessel compared with the buzzer alarm group.

It seems that the reaction time to the alarm (feature 9 and 10) could not indicate a clear difference between the two groups compared with other features. However, we were able to derive a significant result from feature 9. Some of the participants in the buzzer alarm group did not recognize (missed) the alarm or recognized it late after a substantial period of time. In contrast, the voice alarm group showed that all participants had recognized the alarm within a certain time range.

According to the analysis of the causes of 800 collision accidents decided by the Korea Maritime Safety Tribunal over the past five years (2016–2020), 757 cases were linked to human error, accounting for 94.6% of the cases. In addition, 514 cases, 67.9% of the human error cases, were caused by a lack of situational awareness [30]. Considering the above statistics, the improvement in the collision danger recognition owing to the change in the alarm delivery method was a meaningful result in that it improved the root cause of the accident.

Although this research achieved clear academic outcomes, there were several limitations and constraints in this study.

First, as the experimental participants had a relatively short onboard career, it could be different from the alarm response of experienced ship operators. Therefore, in future research, it is necessary to research by sampling ship operators with sufficient experience.

Second, in the scenario used in the experiment, as the subject ship of the buzzer alarm group is a give-way ship and the audible voice alarm group is a stand-on vessel, this may cause bias in the experimental results. Although the audible voice alarm group showed more effective maneuvering action, nonetheless, the bias due to this scenario design should be noted in future studies.

Third, in order to increase the reliability of the experiment, it was attempted to reduce the learning effect by limiting information sharing between the experimenters. However, there is still a learning effect due to the participant’s scenario execution, and there may be a bias in the experimental results. To suppress this learning effect, methods such as randomizing the sequence of scenarios or applying a “Latin square” design should be considered in future research.

Fourth, in this study, the effect of ECDIS’s alarm delivery method was compared by applying two types of alarm methods to CPA alarm. However, as the primary purpose of ECDIS is not collision avoidance, the interpretation of the study is limited. In future research, it seems necessary to select an alarm considering the primary purpose of navigation equipment.

Fifth, although the objective effect of audible voice alarm was confirmed through this study, downsides, such as being more confusable, language problem of the voice, and increase in alarm fatigue that may occur owing to the application of voice alarm, also exist [31].

Therefore, in future research and application, sufficient awareness of this downside is required, and in particular, selective application of voice alarm is required to control the increase in additional alarm fatigue.

Sixth, this study quantitatively presented the effect of audible voice alarm through a data-driven approach, but a qualitative questionnaire to support the results of the study was not conducted. Further investigation requires the collection of qualitative data (survey or interview) to validate these data-driven results.

5. Conclusions

We conducted a comparative experiment to objectively prove the effectiveness of audible voice alarm, a newly proposed alarm delivery method of ECDIS. The navigation performance of two groups (traditional alarm method and proposed alarm method) was collected through a simulation experiment, and the collected data were analyzed through a data-driven approach.

As a result, the two groups were clustered with an accuracy of about 91%, showing a difference in navigation performance, and it was possible to confirm the difference between the two groups by analyzing the features used in the cluster as follows.

First, the audible voice alarm group took more active avoidance action than the buzzer alarm group for the dangerously approaching vessel. This characteristic could be found as the changes in heading, course, and ROT were relatively large. In particular, as the maximum course change was analyzed as the most distinctive feature, it was found that the maximum deviation of course for the avoidance of the audible alarm group was greater.

Second, the audible voice alarm group effectively controlled the approach of the opposite vessel as a result of the active change as above.

Third, the audible voice alarm showed a remarkable improvement effect on alarm recognition. While 6 out of 22 participants in the buzzer alarm group did not recognize the alarm or recognized it late, all the participants in the audible voice alarm group recognized the alarm early.

The most remarkable research achievement of this study is the improvement in participants’ alarm recognition according to the alarm delivery method. Considering that the background of this study was the requirement to improve the alarm effect due to insensitivity or numbness to alarms, the audible voice alarm method seems to be a more appropriate alarm delivery method than the conventional alarm method to solve such a problem.