Validation of the Disaster Preparedness Evaluation Tool for Nurses—The Korean Version

(1) Background: The purpose of this study was to validate a Korean version of the disaster preparedness evaluation tool (DPET-K) for nurses and to verify its validity and reliability for use in community healthcare centers and hospitals in South Korea; (2) Methods: In total, 497 nurses (248 for exploratory factor analysis and 249 for confirmatory factor analysis) at public health centers, public health sub-centers, public health clinics, and general hospitals in Seoul and Gyeonggi, Chungcheong, and Gangwon Provinces participated in this study. The tool went through translation and back-translation, content validity verification, a pilot survey, and validity and reliability testing; (3) Results: The DPET-K had 28 items with five factors (disaster education and training, disaster knowledge and information, bioterrorism and emergency response, disaster response, and disaster evaluation). The Cronbach’s α values for internal consistency were 0.766–0.953 for the five subscales of the DPET-K. A structural equation model was built through confirmatory factor analysis for goodness of fit (χ2/df = 2.193, SRMR = 0.060, RMSEA = 0.069, GFI = 0.831, CFI = 0.927, NFI = 0.875); (4) Conclusions: The DPET-K was confirmed to be a useful tool for assessing the disaster preparedness of nurses in Korea.


Introduction
Disaster preparedness refers to all action plans and efforts prior to the occurrence of a disaster to establish a disaster response system, which actively involves nurses [1]. Nurses are considered to be experts in disaster management and healthcare, and through their expert knowledge and skills, they provide nursing care that reduces the risk of harm to people's lives and health caused by disasters [2,3]. Therefore, understanding nurses' level of disaster preparedness is critical for understanding the status of the disaster management process.
Four main tools have been used to assess the disaster preparedness of nurses in Korea. First, Noh (2010) developed a 44-item tool with reference to the International Council of Nurses (ICN) disaster nursing competencies and the 2003 Wisconsin Health Alert Network Emergency Preparedness Questionnaire (EPIQ) [4]. Second, a tool developed by Kang et al. in 2012 assessed nursing college students' level of interest in disasters and essential supplies, as well as their awareness regarding the importance of disaster nursing, individual disaster preparedness, and the core competencies of disaster nursing (emergency care, communication, and disaster response system) [5,6]. Third, a tool was developed to measure the disaster preparedness of nurses based on the disaster nursing competencies proposed by the ICN, of which the validity and reliability were verified [7]. Fourth, the disaster preparedness evaluation tool (DPET) was created by Bond and Tichy (2007) to measure nurse practitioners' knowledge and skills regarding disaster and post-disaster response and management [8]. Bond and Tichy (2007) originally developed the DPET to assess nurse practitioners' knowledge and skills regarding disaster management [8]. The development of the original DPET was based on the recommended disaster preparedness competencies for nurse practitioners found in the American Association of Colleges of Nursing's Essentials of Master's Education (1996) [9]. The DPET has been translated and used in many different countries, including Asian countries such as Japan, Hong Kong, Indonesia, and Taiwan and Middle Eastern nations such as Iran, Jordan, and Saudi Arabia. The studies conducted in Japan [10], Indonesia [11], Iran [12], the Asia-Pacific region [13], and Saudi Arabia [14] focused on finding out the level of nurses' disaster competency in various stages of the disaster. Meanwhile, the studies conducted in Hong Kong [15], Taiwan [16], and Jordan [17] verified the validity and reliability of the DPET among nurses in their respective countries.
One previous study used the DPET in Korea, and only measured the disaster preparedness capability among public health nurses [18]. Disaster preparedness should be a competency of both public health nurses and nurses at general hospitals. Therefore, this study was conducted to verify the validity and reliability of the DPET among nurses in both public healthcare centers and general hospitals. After adaptations were made to align DPET with Korean culture and language (DPET-K), various methods were used to evaluate it.

Data Collection and Ethical Considerations
Survey data were collected from nurses working at 13 public health centers, three public health sub-centers, 30 public health posts, and two general hospitals with consent from the department heads of each institution. Convenience sampling was used in the process of distributing explanations of the survey, informed consent forms, and survey forms to nurses who volunteered to participate in the survey. The procedure is as follows. After explaining the purpose and method of the research to the institution or department's head, permission for data collection was obtained. Then, the researcher delivered the questionnaire directly to the head of the institution or department. A person in charge of the institution or department distributed the questionnaires and contacted the researchers after collecting the completed questionnaires. Researchers visited and collected the questionnaires.

Participants
The participants were nurses working at public health centers, public health subcenters, public health posts, and general hospitals in Seoul and Gyeonggi, Chungcheong, and Gangwon Provinces in Korea and they understood the purpose of this study and provided written consent.
The ideal sample size to determine the validity of a tool is about 5 to 10 times the number of items in the tool as this helps ensure the stability of analyses like factor analysis or item correlation analysis [19]. A total of 550 copies of the survey were distributed among the nurses to examine the construct validity of the initial version of the tool, which consisted of 46 items. Among them, 497 survey responses were used for further analyses, after excluding 39 surveys that were never returned (93% response rate) and 14 responses that did not have sufficient information.

Instruments
The tool consists of items relating to three stages of a disaster: pre-disaster preparedness, mitigation and response, and evaluation. Each item provides responses on a 6-point Likert scale (1 = strongly disagree, 2 = disagree, 3 = somewhat disagree, 4 = somewhat agree, 5 = agree, and 6 = strongly agree, with higher scores representing higher disaster preparedness). The original version Disaster Preparedness Evaluation Tool (DPET) consists of 46 items. The items (25) in the pre-disaster stage probe disaster knowledge, skills, and individual preparedness, while the items (16) in the mitigation and response stage assess knowledge of disaster and patient care. The final six items in the evaluation stage measure readiness for disaster knowledge and management. The Cronbach's alpha values of each criterion in the original survey were between 0.91 and 0.93.

Validation of the DPET-K Version
This study followed the instrument development guidelines presented by DeVellis [19]. After obtaining permission from Elaine A. Bond to translate and use the tool for this study, the study proceeded in the following order: translation and back-translation, content validity verification, a preliminary survey, the actual survey, item analysis, validity verification, reliability verification, and selection of the final items.

Translation and Back-Translation
The translation was carried out in the following steps: First, the forward-translation of the tool (English to Korean) was completed by an organization specializing in translation. Next, a nursing professor, who is fluent in both Korean and English, drafted the translation. Second, the researcher of this study reviewed the translation to confirm its accuracy as well as the adaptability of the terminology and content in the Korean scenario. Third, a public health nurse and an ER head nurse with experience in disaster education and disaster response provided a consultation to confirm the adaptability of the content. Fourth, a bilingual individual back-translated the content into English. Fifth, a professor (native English speaker) of nursing and public health from the U.S. compared the original and backtranslated surveys to confirm that there were no deviations in the meaning of each question. Finally, a professor of the Korean language reviewed the final draft of the translation to fine-tune the meaning and word order.

Content Validity
The DPET-K had the same format as the original tool. A total of seven academic experts and clinical experts were selected to determine the validity of the content. As academic experts, four nursing professors in the fields of community health nursing and adult nursing (specializing in emergency nursing) were selected. As clinical experts, three nurse managers who had worked or were currently working as ER head nurses at tertiary hospitals were selected. Content validity was measured using the content validity index (CVI), using a 4-point scale with the following responses: 1 = not relevant; 2 = revision required: unable to evaluate relevancy or appears not to be relevant without revision; 3 = relevant but requires modest revision; and 4 = highly relevant and concise. In the first round of verification, items were removed if four or more out of the seven experts rated the CVI with 3 or 4 points, reflecting a level of expert agreement less than 80%. This was followed by a second round of content validity verification, in which corrections and/or revisions were made to items that experts indicated were unclear, too long, or unsuitable for the Korean context. In the second round of verification, the group of seven experts reached an agreement of 80% or higher regarding the CVI for all 46 items.

Preliminary Pilot Study
The purpose of a pilot study is to develop and test the acceptability of the study's instruments. Experts generally recommend 10% of the items as the number of samples to be tested [20]. A pilot study is a small exploratory study for determining the understanding of the items. DPET-K analyzed the validity and reliability, excluding pilot study data.
The DPET-K developed by this study had the same structure, arrangement, and format as Bond and Tichy's tool [8]. In order to check the format, linguistic expressions, and response time, a pilot survey was conducted among 55 nurses; they were from hospitals and public health centers in the Seoul metropolitan area and provided written consent.
After the pilot survey, there were two requests for the clarification of the definition of a "disaster," and one concern that the items of the survey instrument were not easy to read. To resolve these issues, the research team added brief descriptions of key terms such as "disaster," "natural disaster," "social disaster," and "disaster preparedness stages" after discussion, so that the definition of a disaster would be clearer to the participants. Through this process, the survey instrument was finalized.

Construct Validity
Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were carried out to evaluate construct validity [21,22]. All 497 responses were randomly assigned for use in EFA (248 responses) or CFA (249 responses). The model fit, convergent validity, discriminant validity, and concurrent validity were evaluated.

Reliability
To verify the reliability of the DPET-K, internal consistency was analyzed using Cronbach's alpha. The Cronbach's alpha values of the preliminary survey of this study were between 0.94 and 0.96.

Data Analysis Methods
The collected data were analyzed using SPSS (Armonk, NY, USA) and AMOS for Windows (v.22) (Armonk, NY, USA). The general characteristics of the participants were analyzed using descriptive statistics, while construct validity was determined using EFA and CFA. Items were analyzed using item-total correlations, and the Kaiser-Meyer-Olkin (KMO) test and the Bartlett test of sphericity were performed to determine the fit of EFA. When the fit of the model was confirmed, the number of items and factors was decided based on a total explained variance of 60%, eigenvalues of 1.0 or higher in the major component factor analysis, and factor loading of 0.50 or higher using the varimax orthogonal rotation method [19].
Once the number of factors was decided, the goodness of fit of the model was determined using indexes such as χ2/df, root mean square error of approximation (RMSEA), goodness of fit index (GFI), and adjusted goodness of fit index (AGFI) through CFA. Factor loading, significance, average variance extracted (AVE) > 0.50, and construct reliability (CR) > 0.70 were confirmed to verify convergent validity, which reflects whether the items included in the factorial model consistently measured the concepts that were included in the instrument. To verify discriminant validity, the AVE for each factor was checked to determine whether it was larger than the square of the correlation coefficients between the factors [23].

General Characteristics of the Participants
The participants included 381 public health nurses (76.7%) and 116 hospital nurses (23.3%), and 96.4% (479) of them were women. The average nursing experience of the participants was 11.9 ± 8.8. Among the participants, 64.4% stated that they had participated in disaster education or training. The majority of them (64.2%) said that they received disaster education during on-the-job training. Almost all the participants (94.4%; 469) responded that they would like to receive disaster education. No significant differences were found between the EFA (N = 248) and CFA (N = 249) groups according to the working place, gender, age, education, marital status, and nursing experience (Table 1).

Item Analysis
The validity of each item was verified using the mean, standard deviation, skewness and kurtosis, and item-total correlation of the 46 items. For skewness, items with absolute values less than 1.96 were selected to ensure normality, and only items with an item-total correlation coefficient of 0.30 or higher were included.
All items of this instrument had skewness values between −1.28 and 1.15, showing a normal distribution, and the communality of all items was also over 0.5. Item-total correlation analysis was conducted to evaluate the correlation between each item and the entire questionnaire. The correlation coefficients for each factor were all higher than 0.30 (range, 0.338-0.776), suggesting a good fit. Internal consistency of the 46 items was confirmed, with a Cronbach's alpha of 0.96.

Content Validity
The CVI of all 46 items, as evaluated by the seven experts, showed agreement of 80% or higher.

Construct Validity Exploratory Factor Analysis
In the EFA group (N = 249), the goodness of fit of the model was confirmed prior to factor analysis by the KMO test and the Bartlett sphericity test. The KMO value was calculated to be 0.948, which indicated a suitable fit for factor analysis, and the chi-square value of the Bartlett sphericity test was 9601.826 (p < 0.001). Factor analysis was performed after confirming the goodness of fit of the model.
Eight factors were extracted as a result of the first factor analysis, with a total explained variance of 70.41% and then, seven items with a factor loading were lower than 0.5 (items 6, 14, and 29) or a factor loading of 0.4 or higher for two or more factors (items 27, 35, 38, and 39). Those items were not statistically significant. Moreover, item 6 (I am aware of classes about disaster preparedness and management that are offered, for example, at my workplace, the university, or community) was considered to be included in item 4 (I participate in one of the following educational activities on regular basis: continuing education classes, seminars, or conferences dealing with disaster preparedness) in Korea and deleted. The nurses of Korea currently are considered to be healthcare providers limited to the roles of a nurse and a team member [24,25]. So, item 14 (In case of a disaster situation, I think that there is sufficient support from local officials on the county or state level), and 38 (I am familiar with the organizational logistics and roles among local, state, and federal agencies in disaster response situations) were deleted. Nurses at hospitals and even nurses at public health centers in Korea do not perform the roles, as nurses cannot participate in government policy-making and implementation or play the role of a community leader. For the same reason, item 39 was deleted. Item 27 (I can manage the common symptoms and reactions of disaster survivors that are of affective, behavioral, cognitive, and physical nature) included PTSD. Disasters in Korea are managed by the government control system [24]. The general public and media all controlled by government. Considering this, item 29 (I am able to describe my role in the response phase of a disaster in the context of my workplace, the general public, media, and personal contacts) was discarded. In the scenario of a bioterrorism/biological attack, health history and assessment are conducted by a special control system [24]. Therefore, general nurses are not assigned the task and item 35 (In case of a bioterrorism/biological attack, I know how to perform focused health history and assessment, specific to the bioagents that are used) was discarded. As stated above, various items were discarded considering the situation in Korea and finally, 39 items remained.
Six factors were extracted in the second round of factor analysis, with a total explained variance of 66.698%. Then, three items were with a factor loading lower than 0.5 (item 26), a factor loading of 0.4 or higher for two or more factors (item 16), or a factor loading of 0.5 or lower for communality (item 15). Korea has a disaster medical response system which is in a centralized top-down format, and thus, nurses only provide the nursing care [25]. Item 15 (I participate/have participated in creating new guidelines, emergency plans, or lobbying for improvements on the local or national level), and 16 (I would be considered a key leadership figure in my community in a disaster situation) were discarded for this reason. The Korean nurses follow the procedures rather than identifying indicators of mass exposure according to the disaster medical response system [25]; therefore, item 26 (I can identify possible indicators of mass exposure evidenced by a clustering of patients with similar symptoms) was deleted. Items 5 (I read journal articles related to disaster preparedness) and 7 (I would be interested in educational classes on disaster preparedness that relate specifically to my community situation), which were grouped together as a single factor, were also removed because those items contained the basic meaning of disaster preparedness and could be included in item 4.
After this process, the final questionnaire consisted of 34 items and five factors. The percentage of explained variance for each factor was 19.2% for recovery, 17.6% for bioterrorism and emergency response, 14.0% for disaster knowledge and information, 8.3% for disaster response, and 7.9% for disaster education and training, with a total explained variance of 66.949% (Table 2).

Confirmatory Factor Analysis
Model fit was confirmed by structural equation modeling of the five factors determined from EFA and the items in each factor. The indexes (χ 2 /df = 4.401, RESEA = 0.083, GFI = 0.805, and CFI = 0.886) fell short of the standard. Six items (17,23,24,25,30,31) that had high correlations with multiple factors or were similarly correlated with two factors were informed, with reference to the modification index results. Item 17 (I am aware of what the potential vulnerabilities in my community are, e.g., earthquake, floods, terror, etc.) was included in item 18 (I know the limits of my knowledge, skills, and authority as an RN to act in disaster situations, and I would know when I exceed them). Item 23 (I am familiar with accepted triage principles used in disaster situations) was considered to be included in item 36 (I would feel confident working as a triage nurse practitioner, and setting up temporary clinics in disaster situations) and deleted. Since the family's disaster plan was judged to be less related to the nurse's disaster readiness, item 24 (I have personal/family emergency plans in place for disaster situations), and item 25 (I have an agreement with loved ones and family members on how to execute our personal/family emergency plans) were deleted. Diagnosis and treatment in bio-terrorism situations of Korea are performed by the disaster medical assistance team according to the disaster medical response system [25]. So, item 30 (I am familiar with the main Groups (A, B, C) of biological weapons (anthrax, plague, botulism, smallpox, etc.), their signs and symptoms, and effective treatments), and item 31 (I feel confident discerning deviations in health assessments indicating potential exposure to biological agents.) including 'bio-terrorism and response' were deleted.

Convergent Validity
Convergent validity was examined to determine the shared variance of any latent variables. The composite reliability (CR) of the DPET-K items ranged from 0.659 to 0.937; and the AVE ranged from 0.413 to 0.710. (Table 3).

Discriminant Validity
Discriminant validity is considered to be present if the AVEs are greater than the squared correlation coefficient. All AVEs were observed to be greater than the square of the correlation coefficient between each factor. Thus, discriminant validity was supported; that is, each factor was found to be different from all other factors (Table 4).

Concurrent Validity
The disaster nursing tool developed by Noh (2010) was chosen for correlational analysis [4]. For criterion validity, items are considered to be correlated if the correlation coefficient is between 0.40 and 0.60, and closely correlated if it is between 0.60 and 0.80. The correlations between the items of this tool and Noh's (2010) tool ranged from r = 0.481 (p < 0.001) to 0.740 (p < 0.001) ( Table 5).

Reliability and Mean Scores for Each Factor
The internal consistency of the items of the five factors and item score distributions were calculated. Cronbach's alpha for all 28 items was 0.954. For each subdomain, the values were 0.766 for the four items on disaster education and training, 0.892 for the seven items on disaster knowledge and information, 0.926 for the four items on bioterrorism and emergency response, 0.886 for the five items on disaster response, and 0.953 for the eight items on disaster evaluation. The mean scores were calculated for each of the 28 items and they ranged from 2.92 to 4.52 (Table 6).

Discussion
This study examined the properties of DPET-K and various methods were used to assess its content, construct validity, and reliability. The tool consists of five factors and 28 items, and its reliability and validity were confirmed through a survey of Korean community health nurses and hospital nurse practitioners. The findings suggest that DPET-K is structurally valid with five distinct yet convergent subscales and possesses internal reliability. This study is meaningful in the sense that it streamlined the tool by discarding items not suitable to the Korean context and organizing items in a statistically significant way, improving the utility of the tool.
It is also notable that this study was the first to conduct factor analyses of a version of the widely used DPET tool specifically targeting Korean nurses. In particular, the translation and back-translation were based on consultations with nurses who had experience in disaster situations, which led to the development of a final version that can be understood by all nurses working in public healthcare centers and hospitals. The content validity of this tool for both theoretical and clinical aspects of disaster preparedness was verified by four professors with expertise in disaster nursing and three clinical head nurses. The questionnaire was also revised after a pilot survey of nurses to ensure that it could be easily understood both theoretically and clinically, and it was established that this tool can be utilized in a broad range of public healthcare and hospital settings.
The DPET-K is suitable for measuring the disaster preparedness of nurses in Korea and is adapted to their culture and society; however, we also inferred that several additions/improvements were needed in the nurses' role in the future. First, the role of nurses should extend to coordinators, organizers, and leaders. Disaster management in Korea focuses on the disaster response system of the government [24,25]. For this reason, various roles of nurses in disasters covered in the original DPET were excluded in DPET-K. This foregrounded the absence of nurses in policy positions, which were included in the role of nurses in disaster by the international council of nurses [26]. Second, the role of a nurse needs to include daily life and community. On this regard, Korean society is making a distinction between a nurse's role and that of the individual/family. For the same reason, disaster preparedness of nurses in Korean nurses was based on the workplace. However, it would will be difficult to perform active disaster nursing if individuals and families are not prepared. Therefore, disaster preparation for individuals/family needs to be introduced. Third, it is necessary to provide general nurses with more information and skill about the symptoms, assessment, and treatment related to the disaster. Since Korea has a separate disaster medical assistant team (DMAT) dedicated to disasters, it was difficult to cover all aspects of disaster nursing for general nurses working in public health centers and hospitals through DPET-K. But in a pandemic situation such as COVID-19, it is imperative that all nurses participate in disaster nursing.
In this way, the DPET-K has been reorganized in the Korean situation. However, there are many aspects to be considered as in the original DPET by referring to the necessary elements of disaster nursing. Nevertheless, DPET-K has become closer in terms of cultural accessibility for nurses working in various field in Korea. Therefore, DPET-K is considered to be a useful tool to measure the disaster preparedness of Korean nurses of public healthcare centers and hospitals.
In a disaster situation, nurses' leadership and empowerment strengthen interprofessional collaboration and help solve problems [27]. Education to enhance nurses' disaster preparedness, such as an education curriculum reflecting the ICN framework, is essential [28]. In particular, this education encompasses practical as well as theoretical knowledge [29,30]. Therefore, to increase Korean nurses' disaster preparedness, it is necessary to develop an applicable curriculum that grants nurses leadership and authority during a disaster. Through this, it is thought that nurses' core competence can be improved to increase disaster preparedness [31].

Limitation
This study adapted and translated Bond and Tichy's DPET and verified the resulting DPET-K using EFA and CFA. Some items were removed because they did not reflect Korea's circumstances and did not satisfy the criteria for statistical power. DPET-K was identified through statistical analysis, and the final decision was made based on Korean cultural suitability. In the EFA, the disaster data for items 5, 6, and 7 and patient symptom management for item 27 were excluded for statistical power in that they could be included in other items. Items 14, 15, and 16 were excluded because the general nurse's role in Korea is limited to team members. Items 26,29,35,38, and 39 were excluded because they could not be generalized to all Korean nurses because it is the Korean government's role. In the CFA, items 17, 23, 30, and 31 were excluded in the aspect that they could be included in other items. Items 24 and 25, related to the family's disaster readiness, were also excluded because they were considered task-oriented. Therefore, adding and verifying other items reflecting Korea-specific factors to the original DPET could be another way to develop the tool.
DPET-K examines only the perceptions of nurses participating in the study, as it is challenging to measure disaster readiness in real situations accurately. Therefore, attention should be paid to the purpose and interpretation of DPET-K. Moreover, since this study was only for nurses, it is necessary to expand and verify the DPET-K to include various occupations like disaster paramedics.

Conclusions
This study adapted the DPET to the Korean context, determined its validity and reliability, and streamlined its content. The original DPET was condensed into five factors and 28 items. The DPET-K consists of a three-phase pre-disaster stage, disaster stage, and post-disaster stage. The pre-disaster stage (prevention) has 15 items composed of disaster education, disaster knowledge, and bioterrorism; the disaster stage (mitigation) has five disaster response items; and the post-disaster stage (recovery) has nine items of disaster evaluation. The DPET-K is expected to help understand disaster preparedness at each stage and analyze and interpret each of the five factors.