Infrastructure Cyber-Attack Awareness Training: Effective or Not?

The purpose of this study is to provide insight as to how infrastructure countermeasures awareness training will impact individuals dealing with a nationwide catastrophic cyber-attack. Can this awareness training lessen the psychological effect of an attack? This study showed no value for this type of training. Reading about such an attack, the subjects had lower technical optimism and cyber self-efficacy. Reading about infrastructure countermeasures, before or after reading about a cyberattack, did not improve or maintain the subjects’ optimism and self-efficacy. A possible explanation is that emotional arousal may override or block rational thinking. Another explanation may be that a nationwide attack is towards the infrastructure and not the personal computer. Here the individual lacks any control. Future research needs to look at personal preparation and response training to see if it will help the psychological effects of a catastrophic cyber-attack.


People's Responses 1
When people feel threatened, their anxiety, fear, unrest, crisis mood, will lead to panic (Brickenstein, 1980). When a disaster occurs, panic could lead to selfish or irrational flight behaviors due to a loss of control, functionality, lack of knowledge and lack of communication. "Lack of control over a situation that is perceived as threatening or dangerous can give rise to feelings of emotional distress, fear and insecurity. Such strong emotions can on occasion lead to irrational behaviour" (Sutherland, 2007). "These are all aspects that can influence how members of the public (psychological) . . . are impacted by an attack" (Bada & Nurse, 2020). Quarantelli (2001) and Tierney et al. (2006) find that people's initial response to an emergency is prosocial rather than selfish or irrational flight. Singer (1982) discusses people's reactions and responses to disasters and the need for disaster planning and training. However, this discussion deals with physical destruction (i.e., earthquakes and building collapses), including deaths and injuries. A cyber-attack does not generally create physical destruction, it disrupts computer systems. Hence, the panic that a cyber-attack generates will be of a different nature, a disruption of the ability to function while infrastructure is still intact. To overcome panic, clear communication from authorities is extremely critical (Loong, 2018). The problem here is with a cyber-attack, clear communication can be blocked. Prior communication, such as awareness training before the cyber-attack may be needed to reassure people.

Response Types 1
Four responses to a cyber-attack include: 1. Optimism: Trumbo et al. (2014) define optimism as a "person's belief of being at less risk from the dangers of the environment." Over the years, students were more optimistic about the impact of computers on their performance. Males are more optimistic than females (Walstrom, et al., 2010). 2. Self-Efficacy: Self-efficacy is the perceived ability to perform the needed response to cope with a specific risk (Ajzen, 2002). "Self-efficacy is considered not as skills themselves, but as the evaluation of what one can do with skills. It considers a person's belief in themselves and their abilities" (Bandura, 1991). Self-efficacy is the confidence to successfully perform an action (Bandura, 1977) or deal with a threat (Liang & Xue, 2010). For example, Ng e. al. (2009) showed that self-efficacy determines employees' email-related security behaviors. Yoon et al. (2012) proposed a model that identified self-efficacy as a variable which significantly affects the decision of home wireless network users to implement security features on their networks. They 4 found that self-efficacy has a significant impact on students' intentions to practice information security. "This concerns issues such as cyberattacks because it is important that individuals believe that they stand a chance of protecting themselves and responding successfully to an attack's occurrence" (Bada & Nurse, 2020). In this study, we define self-efficacy as the confidence in using current cyber knowledge and software. 3. Worry: Panic disorder is a form of worry (Davey & Wells, 2006). Worrying is perceived as feelings of inability to control the situation, which is a feature of generalized anxiety disorder (GAD) (Davey & Wells, 2006). Worry can become a source of extreme emotional discomfort (Davey & Wells, 2006). It is an attempt to engage in mental problem-solving whose outcome is uncertain but contains the possibility of one or more negative outcomes. Worry relates closely to the fear process (Borkovec, et al., 1983).

Education/Awareness Training
Organizations realize the importance of user security education and awareness training (Dodge et al., 2007;Schultz, 2004). Education makes users more security conscious (Ng et al., 2009) and changes users Internet behavior (Albrechtsen & Hovden, 2010;D'Arcy, Hovav, & Galletta, 2009;Kruger et al., 2010). Refresher courses will be needed to lower unrealistic thinking, such as optimistic bias. Users must constantly be reminded to be aware of security issues (Peltier, 2005). An educational program must continually keep users aware and proactive and must build proper security habits (Yoon et al, 2012). The belief is that as more people are informed, the better off society will be when a massive attack does occur. They will be trained, educated and psychologically prepared to deal with an attack. However, repeated awareness training can lose its effectiveness over time (Wolf et al., 2011). Most business organizations conduct security awareness training to deal with policy, procedures, and tools (Ku et al., 2009;Peltier, 2005;Rotvoid & Landry, 2007). Research about security awareness and education has focused on protective behavior (Britt, 2008;Claar & Johnson, 2012;McLaughlin, 2006;Mensch & Wilkie, 2011;Pollitt, 2005;Puhakainen & Siponen, 2010;Wagley, 2010). The training focuses on the organization's system instead of a national catastrophic cyber-attack. However, the literature lacks any evidence showing training and education impacting psychological reaction to a national catastrophic cyber-attack.

Hypotheses
This study explored subjects reading about a massive national cyber-attack and reading about infrastructure countermeasures (awareness training) and how it impacts a general population's technology optimism, cyber optimism bias, cyber self-efficacy, and general worry. The readings did not focus on personal computer and personal actions and preparedness. For the reading, physical infrastructures, such as roads and bridges, were not affected Reading a scenario of national catastrophic cyber-attack after reading awareness training of infrastructure countermeasures will: H2-1b, not change Technology Optimism. H2-2b, not change Cyber Self-Efficacy H2-3b, not change General Worry.

METHodoLoGy
Subjects: "Internet participants in online studies are a purely self-selected sample of participants and thus maybe more homogeneous than desired" (Weiser, 2000). This was avoided by using a random sample of 579 adults from the general population of the U.S.A. (via Qualtrics). They were invited to participate in this research. They were divided into two groups: Group A, countermeasures awareness after attack, and Group B, countermeasures awareness before an attack. They accessed a Qualtrics survey that contained four instruments and two readings. The two readings were written by a Certified Information System Security Professional of (ISC)2 and a Certified Cyber Security Professional of ICCP. Some of the demographics are mean age 45 + 17; 49% male, 51% female, and 46% full time employment. Age, gender, employment status, and job type were not considered in this study. Those variables will be used in a future study dealing with demographic impacts.

Four Instruments Used (see Appendix C):
The three instruments were designed with either a 5-point or 7-point Likert scale to indicate level of agreement. For example, strongly agree to strongly disagree. This provided discrete tiered numbers with a restricted range (1 to 5 or 1 to 7).

Technology Optimism Instrument
Items for technology optimism were taken from the Technology Readiness Index (TRI) (Parasuraman, 2000).

General Worry Instrument
The Penn State Worry Questionnaire (PSWQ) represents excessive and/or uncontrollable general worry (Startup & Erickson, 2006). The content was not cyber. It was taken from a published paper by Meyer et al. (1990). The 16-item PSWQ has high internal consistency for both clinical and non-clinical groups (Molina & Borkovec, 1994). Cronbach's alpha coefficients have been between 0.88 and 0.95 for clinical samples.
Of the 16 items, seven were selected. The items were selected based on three research paper factor analysis loadings (Brown, 2003;Meyer et al. 1990;Fresco e. al. 2002). For an item to be selected, two of the loading factors had to be greater than .7, and the third had to be greater than .6.
Five phases of this study: 1. Determine current state (Data Set 0): The first phase was the administration of the four instruments to determine the current state of worry, optimism, optimistic bias, and cyber selfefficacy. This was to establish a baseline as the control prior to treatments for comparisons. The subjects were then placed in one of two groups, group A or group B.

First Treatment:
Group A read a scenario of a national catastrophic cyber-attack. See Appendix A. Group B read a scenario of countermeasures. See Appendix B.

Determination of state after reading the Scenario A & B (Data Set 1): After reading the first
Scenario, the four instruments were administered again to determine the current state of worry, optimism, optimistic bias, and cyber self-efficacy.

Second Treatment:
Group A read a scenario of countermeasures. See Appendix B. Group B read a scenario of a national catastrophic cyber-attack. See Appendix A.

Determination of state after reading scenario A & B (Data Set 2): After reading the second
Scenario, the four instruments were administered again to determine the current state of worry, optimism, optimistic bias, and cyber self-efficacy.

Analysis
Winter & Dodou (2012) wrote a paper on using five-point Likert items with a t-Test (parametric test) and Mann-Whitney-Wilcoxon (MWW) (non-parametric test). They found both tests have similar power. However, Winter & Dodou (2012) found MWW had a power advantage with non-normal distributions. Their conclusion for five-point Likert data were that both tests will not find a significant difference in a population when there is none. This was consistent with Rasmussen (1989). That study showed parametric and nonparametric tests were similar regarding false positives (Type I error rate) for Likert items (Rasmussen, 1989). Therefore, for analysis, paired-wise t-Tests of the four measures of technology optimism, optimistic bias, self-efficacy, and worry were done to determine if there were differences. If the significant data had skewed or peaked distributions, the non-parametric related-samples Wilcoxon Signed Rank Test was used to confirm the t-Test.

descriptive Statistics
Tables 1A & 1B shows the descriptive statistics of the three sets of data for Technical Optimism, Cyber Optimistic Bias, Cyber Self-Efficacy, and General Worry for Groups A and B. The data were non-normal. The statistics were more than two standard errors. To confirm any significant findings with the t test, the non-parametric related-samples Wilcoxon Signed Rank Test was used.

Face Validity of the Readings (Scenarios)
Nunnally & Bernstein (1994) defined face validity as the extent a measure reflects what it is intended to measure. Anastasi (1988), and Nevo (1985) defined face validity as the degree that respondents judge the appropriateness of instrument items. To determine if the two readings were meaningful and appropriate to the subjects, four post-survey questions were given to check the quality and validity of the readings (scenarios). See Table 2.
As shown in Table 2, 63% of the subjects believed they learned a good or large amount and gained good or large insight from the readings. Table 2 also shows 66% judged the readings as insightful Valid N 289 * data were non-normal due to skewness and/or kurtosis; significant differences from zero. The related-samples Wilcoxon Signed (WS) rank test was warranted for significant t-tests.
or very well done. Finally, half of the subjects were surprised by the extent of the disruption and countermeasures response. These perceptions by the subjects suggest good face-validity.

Validity and Reliability data Analysis
Validity and reliability of the data were checked using Cronbach's Alpha, Kaiser-Meyer-Olkin (KMO) Measure of Sampling Adequacy, and Bartlett's Test of Sphericity before the data were analyzed. Except for one value (.78), the Cronbach's Alphas were over .82, which indicates high internal consistency. The Alpha values were considered "excellent." See Tables 3A & 3B. For this analysis, the factors for each data set were tested for validity by performing the Kaiser-Meyer-Olkin (KMO) Measure of Sampling Adequacy and Bartlett's Test of Sphericity. Since KMO was greater than .87, and Bartlett's Tests were significant (p < .001), variables had a strong relationship supporting the use of factor analysis. Although these items are self-reporting perception, they have significantly high validity and reliability. See Tables 4A & 4B. Normalization) was performed on each data set to ensure all items of the survey loaded correctly on the factors intended. Group A cumulative total variance explained through rotation sums of squared loadings for Data Sets 0, 1, 2 were 80.31%, 83.76%, 84.96%, respectively. All Rotated Component Coefficients were >.79, with most >.85. Refer to Appendix D-A for Group A factor analysis.
Group B cumulative total variance explained through rotation sums of squared loadings for Data Sets 0, 1, 2 were 78.78%, 83.95%, 86.82%, respectively. All Rotated Component Coefficients were >.76, with most >.85. Refer to Appendix D-B for Group B factor analysis.
To ensure there were no initial differences between groups A and B, an Independent Samples t-Test was performed between the two groups variables before treatment. There were no significant differences initially between the two groups for Technology Optimism, Cyber Self-Efficacy, and General Worry.

Paired-wise t-Test and Mean Plots
Since an ANOVA treats each data set as coming from different subjects rather from the same subject, Pair-Wise t-Tests were performed. Pair-Wise is used when the difference between the two variables came from the same subject. The subject becomes their own control. If significant, a Plot Means graph was generated and the non-parametric related-samples Wilcoxon Signed Rank Test was used to confirm the t-Test.

Group A: Reading about a Cyber-Attack Before Awareness of Countermeasures
Technology Optimism (H1-1) Since the data were non-normal, the related-samples Wilcoxon Signed (WS) rank test was performed on both pairs. The first pair had p < .001 and the second pair had p < .003. WS was consistent with the t-test. The null hypotheses of the differences between OptTech0/1 and OptTech1/2 equals 0 was rejected.
However, the Effect Sizes based on Cohen's d (.134 & .085) were found to be small, < .2. The Effect was trivial. Table 6 and Graph 2 show that reading about the results of a massive cyber-attack lowered cyber self-efficacy (Pair 1: t = 5.408, df = 289, p < .001; Cohen's d = .324). However, awareness of countermeasures, after the fact, does not restore cyber self-efficacy (Pair 2: t = 1.044, df = 289, p As Graph 2 indicates, Cyber Self-Efficacy decreased after the reading the scenario of the attack and the awareness scenario failed to restore Cyber Self-Efficacy. The confidence with installing and configuring security software on their computer to deal with the attack decreased.

Cyber Self-Efficacy (H1-2)
General Worry (H1-3) Table 7 shows that General Worry is not impacted by reading about an attack or awareness of countermeasures.
Since the data were non-normal, the related-samples Wilcoxon Signed (WS) rank test was performed on the significant second pair. The second pair had p < .029. WS was consistent with the t-test. The null hypotheses of the differences between OptTech0/1 and OptTech1/2 equals 0 was rejected. However, the Effect Sizes based on Cohen's d (.084) was found to be small, < .2. The Effect was trivial.

Figure 2. Mean Plots for Cyber Self-Efficacy
Cyber Self-Efficacy (H2-2) Since the data were non-normal, the related-samples Wilcoxon Signed (WS) rank test was performed on the significant second pair. The second pair had p < .007. WS was consistent with the t-test. The null hypothesis of the differences between Self-Eff_1 and Self-Eff_2 equals 0 was rejected.   As Graph 4 indicates, Cyber Self-Efficacy decreased after reading the scenario of the attack. The awareness training before the attack reading failed to change Cyber Self-Efficacy. The confidence with installing and configuring security software on their computer to deal with the attack decreased.

General Worry (H2-3)
Table 10 and Graph 5 show that General Worry decreases with the awareness of countermeasures. However, reading about an attack after awareness training, leaves General Worry the same.
Since the data were non-normal, the related-samples Wilcoxon Signed (WS) rank test was performed on the significant first pair. The first pair had p < .012. WS was consistent with the t-test. The null hypotheses of the differences between Worry_0 and Worry_1 equals 0 was rejected. However, the Effect Sizes based on Cohen's d (.059) was found to be small, < .2. The Effect was trivial, and the difference is unimportant.

Hypothesis Results
Based on these results, the findings for the hypotheses are as follows: Hypothesis 1: Reading about a Cyber-attack before Awareness Training of Infrastructure Countermeasures--Group A

Figure 4. Mean Plots for Cyber Self-Efficacy
Reading a scenario of a national catastrophic cyber-attack will: H1-1a, lower Technical Optimism. Significant but Trivial H1-2a, lower Cyber Self-Efficacy. Significant but Trivial H1-3a, increase Worry. Not Significant Reading a scenario of countermeasures after reading a scenario of an attack will: H1-1b, increase Technical Optimism. Significant Decrease but Trivial H1-2b, increase Cyber Self-Efficacy. Not Significant H1-3b, decrease General Worry. Not Significant Hypothesis 2: Awareness Training of Countermeasures before reading about a Cyber-attack -Group B Awareness training of infrastructure countermeasures will: H2-1a, increase Technical Optimism. Not Significant H2-2a, increase Cyber Self-Efficacy. Not Significant H2-3a, decrease Worry. Significant Decrease but Trivial

Figure 5. Mean Plots for General Worry
Reading a scenario of national catastrophic cyber-attack after reading awareness training of infrastructure countermeasures will: H2-1b, change Technology Optimism. Significant Decrease but Trivial H2-2b, change Cyber Self-Efficacy Significant Decrease but Trivial H2-3b, change General Worry. Not Significant

dISCUSSIoN
As Rhee et al. (2012) indicated, since technology alone cannot completely protect information systems from potential threats, there needs to be more effort in addressing the human dimensions when dealing with information security events.
Technical Optimism decreased before and after reading about a catastrophic cyber-attack on the infrastructure, resulting in the increase in the belief of dangers. After awareness training of countermeasures, Technical Optimism continued to decrease. A possible explanation is that the awareness training stressed the lack of personal control since the control was with the infrastructure. Awareness training did not increase Technical Optimism before or after the attack reading. Findings indicate that awareness of infrastructure countermeasures has no positive impact on Technical Optimism.
Cyber self-efficacy decreased after reading about an attack whether awareness training of countermeasures was done before or after. This is consistent with Technical Optimism. It appears a massive cyber-attack will increase the belief of being at greater risk from the dangers of the technical environment. The subjects consciously lost faith in technology and lost their confidence to control the attack's effect on the internet. This may be explained by the lack of control over the situation.
Their personal computer is not the major target of massive national cyber-attack. It is the infrastructure computers that are the major targets (i.e., ISP, computers controlling the power grid and water systems). They may have realized that much of cyber operations are out of their control because it was infrastructure computers were attacked.
General Worry (non-cyber) appears to not be affected by a cyber-attack. This may be due to the PSWQ's focus on self-feelings in general (non-cyber). This suggests that subjects can differentiate their worry consciously and/or sub-consciously between cyber and non-cyber situations. However, General Worry does decrease after awareness training of countermeasures before reading an attack scenario.

CoNCLUSIoN
Will awareness training of countermeasures before or after reading about a cyber-attack provide an immunization effect? Can education override the psychological effect of a catastrophic cyber-attack? Based on these results, the answer is no.
This study shows that awareness training of infrastructure counter measures has little or no impact on the psychological consequences of a cyber-attack. Considering Sutherland (2007) research, it appears the emotional distress inhibits psychological recovery. Since an attack on cyber infrastructure deals with the subject's inability to control infrastructure, self-efficacy, the ability and confidence to control, drops (Ajzen, 2002). This can lead to emotional distress and irrational behavior (Sutherland, 2007). This may explain why awareness training has no effect as found in this study.
The outcomes of this study failed to support the value and need for infrastructure awareness training and education before and after reading about a catastrophic cyber-attack. Overall, most findings were non-significant. The few significant findings were trivial as defined by Cohen's d. Awareness training of infrastructure countermeasures before or after reading about a massive national cyber-attack for a general population appears to be of little value when it comes to internal responses for technical optimism, cyber self-efficacy, cyber optimistic bias, and general worry. However, an actual experience, rather than just a reading, may have a greater impact on psychological dimensions.
Reading about a catastrophic cyber-attack did lower technical optimism and cyber self-efficacy. Half of the subjects were surprised about the extent of disruption. It appears the emotional response when reading about a cyber-attack may cause awareness training to be ineffective. A reason for this may be that the countermeasures were for the infrastructure, which is out of the control of the subject. The point here is that awareness training needs to be proactive and focus on local or personal computers. Instead of reassuring people the infrastructure can deal with the massive national cyberattack, awareness of action and preparation the subject can take may be to show better results with an actual attack.
Another consideration for future research would be intervening variables such as demographics. Age and computer knowledge may have an impact on the effectiveness of awareness training and internal responses.

Implications
Singer (1982) discussed people's reactions and responses to disasters and the need for disaster planning and training. There is a need for more training and education on how to prepare and respond to security incidents before an attack happens. This study used a reading scenario stressing infrastructure countermeasure instead of stressing what the individual can do.
Awareness training and education have traditionally tended to focus on infrastructure systems and protocols, aiming to be reassuring of the security of the infrastructure. Learning how to prevent and respond to security incidents on the individual/local level before incidents happen may improve technical optimism, cyber optimistic bias, cyber self-efficacy, and general worry. These are actions to take to build confidence of what you can control.

Three instruments
Technology Optimism Indicate your optimism on the following topics by indicating: Strongly disagree; Disagree; Somewhat disagree; Neither agree nor disagree; Somewhat agree; Agree; Strongly agree.
• New technologies contribute to a better quality of life. • Technology gives me more freedom of mobility • Technology makes me more productive in my personal life • Technology gives people more control over their daily lives Cyber Self-Efficacy Compared to others in the U.S. that are similar age as you, answer the following questions. (NOT at all confident; NOT confident; Somewhat NOT confident; Neutral; Somewhat confident; Confidant; Totally confident).
• I can select the appropriate security software for my home computer. • I can correctly install security software on my home computer. • I can correctly configure security software on my home computer.
• I can find the information needed if I have problems using security software on my home computer.

Penn State worry Questionnaire
Rate each of the following statements on a scale of 1 ("not at all typical of me") to 5 ("very typical of me"). Please do not leave any items blank.