Competency Models for Information Security and Cybersecurity Professionals: Analysis of Existing Work and a New Model

2.1 Information Security and Cybersecurity

Information security and cybersecurity can be differentiated by considering the origin of the threats and the assets that are to be protected [112]. Although competing definitions exist, information security can be understood as an ongoing process [83] concerned with the protection of analog and digital information, its security properties, and the Information Technology (IT) that stores valuable data from intentional and unintentional threats that arise from physical and virtual sources [2, 57, 112, 119]. In contrast, cybersecurity is a computing-based approach [60] that focuses on the protection of information systems (e.g., hardware and software), the information stored on them, and non-information-based assets (e.g., humans and society) that are vulnerable to intentional or unintentional threats originating from cyberspace [53, 112]. IT security referring to the protection of information systems can be seen as a subset of both information security and cybersecurity [112].

From the perspective of security, assets have security properties assigned, including confidentiality, integrity, availability, authentication, authorization, and nonrepudiation [39]. These security properties are defined as follows [2, 23]. Confidentiality refers to the ability to ensure that information is not disclosed to unauthorized individuals, processes, or devices. Integrity ensures that information is not maliciously or unintentionally modified or altered. Availability ensures that information is accessible by authorized individuals when required. To establish whether a claim of identity is true, authentication is used. Implemented using access controls, authorization decides what an authorized entity can or cannot do. Last, nonrepudiation is achieved when the people taking action cannot successfully deny that they have done so [2].

In recent years, several efforts [46, 60, 95] have been made to collect, systematize, and codify the foundational information security and cybersecurity knowledge in a Body of Knowledge (BOK). Given that cybersecurity is a broad and interdisciplinary field, different bodies have different foci. For the competency model analysis, we have selected the CyBOK [94], because it (i) is an up-to-date body, (ii) has a strong focus on cybersecurity [124], and (iii) consists of a reasonable number of Knowledge Areas (KAs) that allow for a fine-grained content analysis that is neither too abstract nor too specific. The CyBOK is a comprehensive BOK with a more technical focus than other BOKs, such as the Certified Information Systems Security Professional BOK or the Cybersecurity Curricula 2017 [124]. The CyBOK’s purpose is to codify foundational knowledge and serve as a guide for cybersecurity knowledge. The CyBOK’s basis is formed by 19 KAs that are grouped into five broad categories [94]. Table 1 provides a brief definition of each area.

2.2 About the Concept of Competency

Competency is a widely adopted concept in cognitive, social, and educational science [63] and has been introduced in psychology as a counterterm to intelligence [50, 81]. Theoretical views, national context, and application area influence the concept’s meaning [110], and different approaches to conceptualizing competencies coexist [33, 71, 114]. For instance, Schippmann et al. [99] revealed that experts’ answers to the question of what a competency is vary. Given its different meanings, some authors have referred to the term as a fuzzy concept [109]. Nonetheless, the concept promises to help bridge the gap between education and the labor market [71, 90, 109].

Which components should be included in the competency construct is an ongoing debate. Focusing on a narrow notion of competency, Klieme and Leutner [64] defined competency as a context-specific, cognitive performance disposition, thereby reducing the concept to specialized cognitive prerequisites [49, 50]. In contrast, the computing curricula 2020 report went beyond the cognitive realm and defined competency as “composed of K-S-D dimensions observed within the performance of a task” [22, p. 47]. According to this notion, competency integrates knowledge, skills, and dispositions that are causally related to the accomplishment of a task [41]. The integration of cognitive and noncognitive components into a complex competency system is also frequently found in the concept of action competency [64, 114]. For example, the German Qualifications Framework Working Group [45] defined competency as “the ability and readiness to use knowledge, skills, personal, social, and methodological competencies and to behave in a considered, individual, and socially responsible manner” [45, p. 17]. In this article, we adopt a holistic approach to competency and refer to the definition of Weinert [116, pp. 27–28]: competencies are the cognitive abilities and skills possessed by or able to be learned by individuals that enable them to solve particular problems, as well as the motivational, volitional, and social readiness and capacity to use the solutions successfully and responsibly in variable situations. This notion of competency implies that competencies are comprised of “all those cognitive, motivational, and social prerequisites” [115, p. 51] that are necessary for achievement. Specifically, this holistic approach to competency integrates cognitive and noncognitive components into a complex system of knowledge, skills, attitudes, and cognitive abilities [114]. Although not explicitly stated, knowledge is a component of Weinert’s definition [62]. Here, knowledge refers to the mastery of core concepts and topics acquired through learning [74, 114, 120]. Cognitive abilities refer to general intellectual abilities that are less learnable [69, 114]. Relying on the CC2020 Task Force [22] and the German Qualifications Framework Working Group [45], we define skill as the proficient application of knowledge to successfully meet demands in a particular action context. The construct described as “motivational, volitional, and social readiness and capacity” refers to attitudes respectively dispositions [41] and bridges the gap between the mere ability to do something and the actual behavior [89]. Dispositions are affective by nature and can be understood as tendencies toward a certain behavior and the sensitivity to know how and when to engage in a task [89]. In this sense, the affective component is what transforms the mere ability to act into appropriate action [1]; it establishes the connection between what a person can do (ability) and what a person does do (action) [41]. Last, the internal structure of the competency is derived from the structure of the task, with the task unfolding and framing the purpose and meaning of competency. The task serves as a crystallization point of competency (i.e., the task renders competencies concrete and visible) [90, 115].

For analytical and organizational reasons, competencies can be categorized into competency classes [34, 77, 104]. By default, competency classes can be differentiated according to task or demand [33, 34]. If the subject’s action relates to other people or groups of people (the task), it is a question of social competencies. Personal competencies refer to tasks concerned with oneself (e.g., self-control in stressful situations), and professional competencies pertain to domain-specific and work-related tasks. Methodological competencies are somewhat different, as their task application is more general. Here, we refer to methodological competencies as personal qualities that apply to a broad range of tasks (e.g., problem solving). Depending on the task, the relative emphasis of the competency components varies [41]. Thus, some competencies are strongly knowledge focused, whereas others are more skill- or disposition focused.

2.3 Competency Models

The concept of the competency model has been defined in three ways. First, the term can refer to the modeling of the internal structure of competency in general terms, specifying personal qualities, such as dispositions and skills, as competency components [41]. Second, there are competency structure and level models used to model the dimensionality and differentiate between the levels of proficiency of a concrete competency, such as foreign language competency or programming competency [6, 64, 67]. Third, competency models, as understood in this work, refer to organized catalogs or lists of competencies required by individuals to achieve goals, meet demands, and perform effectively in a specific role within a job, job family, organization, industry, or process [20, 32, 75, 78, 79]. Specifically, the term competency framework is also frequently used in the literature to refer to a structured competency collection [3, 30, 34].

Because models can contain a large number of competencies [34], organization becomes crucial. To organize competencies, different structures have been proposed [106], including hierarchies [32, 108] and typologies [45, 52, 104]. For instance, models by the Employment and Training Administration [32] organize competencies into stacked tiers that form a hierarchically structured pyramid shape. In contrast, the “KompetenzAtlas” [52] classifies competencies based on a competency class typology. Regardless of the underlying structure, competencies constitute the core of a competency model, and models often record competencies in detail. A competency usually consists of (i) a label or title highlighting the name of the competency, (ii) a detailed description of the competency in behavioral terms, and (iii) proficiency levels or behavioral indicators outlining how a competency unfolds in action [20, 97]. Grouping behavioral indicators into proficiency levels (e.g., novice, intermediate, and expert) facilitates the application of competency models in many HR activities, including performance management, appraisal systems, and workforce development [98, 106]. Owing to their many applications, competency models can be considered the backbone of an organization’s competency management [97]. To maximize the benefits of using competency models, many models are highly tailored to an organization’s context and strategy, use organization-specific language, and are graphically elaborated [20].

2.4 Related Work

Against the backdrop of a lack of qualified workers and with the aim of tackling the workforce shortage, several studies have examined competency models to provide input for the preparation of cybersecurity programs. For instance, Manson et al. [76] asked faculty experts to assess the content of several standards, including the IT Security Essential Body of Knowledge (EBK) [150]. According to the results, the EBK’s competency area “data security” was considered most important, whereas “strategic security management” was deemed least important [76]. To determine industry priorities regarding the competencies of entry-level professionals, Whitman [118] asked participants to rate the competencies of the Cybersecurity Competency Model [153]. Results showed that all competencies were in demand, although some were more favored than others. Moreover, the preference for specific competencies did not vary between organization size and industry [118]. With the aim of informing curriculum development, Armstrong et al. [4] and Jones et al. [61] investigated the relative importance of the National Initiative for Cybersecurity Education (NICE) Cybersecurity Workforce Framework’s Knowledge, Skills, and Abilities (KSAs)¹ [140] that are listed under specialty areas within the “protect and defend” category. In sum, the most important KSAs common to two or more specialty areas dealt with networks, vulnerabilities, threats, and programming [61], and nontechnical skills were rated highly important for achievement [4]. Next, several studies have compared competency models to identify differences and similarities. From these efforts, we can see that models can vary in several ways, including in the treatment of nonprofessional competencies (e.g., methodological, social and personal competencies), the number of competencies [15], the basic structure [84], and the concepts referred to [40]. Another line of studies has analyzed the content coverage of models and whether predefined characteristics have been met. Using the CyBOK areas to map the content of several frameworks, Hallett et al. [48] found that the NICE Framework, although not exhaustive, covers most KAs, with “security operations and incident management” and “risk management and governance” being the most emphasized. Focusing on the analysis of the e-Competence Framework (e-CF), Plessius and Ravesteyn [91] showed that the e-CF [132] covers the IT domain to a great extent and fulfills many quality criteria. Miloslavskaya and Tolstoy [82] analyzed four models in terms of their applicability to the Internet of things (IoT) and cloud areas and found that the NICE Framework best fulfills the requirements. To inform a possible drafting of an e-competency framework for Malta, Camilleri [19] analyzed the usability of existing e-competency frameworks in Europe. Findings shed light on best practices regarding usability. Examining three models (e.g., e-CF) in terms of user expectations, Brown and Parr [15] found that models did not fully comply with user expectations, such as utility, portability, and simplicity. Moreover, all three models lacked automation features, limiting their usefulness in advanced skill management tasks [15]. Elsewhere, Brown [14] discussed the issue of backward compatibility between the Skills Framework for the Information Age (SFIA) 7 [145] and 6. Another study [25] examined the applicability of the e-CF 3.0 and SFIA 6 to the profile of a data scientist and concluded that both models adequately represented the profile. Last, relating to our investigation are studies extracting competencies from cybersecurity and information security job advertisements. From these efforts, we can draw the conclusion that it is not only professional competencies that are in demand but also methodological, social, and personal competencies [13, 92, 93]. Reducing the competency profile of the cybersecurity workforce to professional competencies is therefore an invalid process.

3.1 Research Goal and Questions

To achieve consistency between goals, research questions, and metrics, the Goal–Question–Metric paradigm [7] has been used. The goal of this study is

The goal leads to two research questions:

• RQ1 Which competency models for cybersecurity and information security are available and what are their characteristics?

• RQ2 Can we use existing competency models to build a new security competency model, and which components and properties should characterize the new model?

To answer the research questions, we collected, analyzed, and synthesized evidence for several metrics:

The search and selection of sources, the data analysis, and the construction and validation process for the new competency model are described in Sections 3.2, 3.3, and 3.4, respectively. The competency model analysis results answering RQ1 are presented in Section 4, and the new competency model that provides answers to RQ2 is presented in Section 5.

3.2 Searching and Selecting Sources

The search process and source selection are critical to our research, as they lay the foundations for all of the results. To optimize the search process and source selection, we adopted recommended strategies stated in the guidelines of systematic and multivocal literature reviews [43, 122].

3.2.2 Source Selection.

Source selection deals with defining and applying inclusion and exclusion criteria to identify relevant sources for answering research questions [43]. Similarly to Garousi and Mäntylä [44], we only defined inclusion criteria, as these criteria already indirectly excluded irrelevant sources. Additionally, we used some quality assessment criteria because GL requires special treatment. First, we applied inclusion criteria to the title and the abstract. Subsequently, we applied the criteria to the body of content. Table 3 shows the inclusion criteria and some sources that were excluded. For clarification, we selected sources that solely contained a competency model and sources in which the competency model was only one element among many (e.g., in curricula). Additionally, we included any accompanying material to which no selection criteria were applied (e.g., material of the e-CF [131, 133]). Figure 1 presents the entire search and selection process.

Table 3.

#	Type of Criteria	Inclusion Criteria	Excluded
1	Content	The publication contains a stand-alone security competency model or a model integrating security concepts (i.e., a list of competency descriptions, behavioral indicators, or related concepts).	[31, 34, 85, 102]
2	Content	The competency model describes competencies that practitioners or graduates of a tertiary program should possess.	[96]
3	Language	The literature is in English or German.
4	Access	The full text can be accessed.	[11, 35]
5	Bibliographic information	The producer (author, institute, organization, etc.) and the date of publication are indicated.	[58]
6	Bibliographic information	The source was published online during the time frame from 1990 to 2020.

View Table

Table 3. Inclusion Criteria

Fig. 1.

View Figure

3.3 Data Analysis

To evaluate the content of the 29 sources, we performed a Qualitative Content Analysis (QCA) using MAXQDA.² QCA is a qualitatively oriented, category-based method that systematically condenses qualitative material, reduces complexity, and deciphers the meaning of qualitative data [70, 80, 101]. QCA does so by assigning text passages (coding units) to categories of a category system [101]. Several forms of QCA exist, and a decision regarding a specific technique depends on the project’s research questions. For this study, we favored a content structuring QCA [70, 80]. A content structuring QCA allows for specific topics to be filtered out of the material and summarized [80]. The category system used in this method usually consists of deductive and inductive categories [70]. We derived some of our deductive categories by transforming the metrics (see Section 3.1) into categories. Additionally, competency classes (e.g., social, methodological, personal, and professional competencies), as well as the five CyBOK categories and 19 CyBOK areas [94] (see Table 1), were converted to categories that, together, formed a theoretically derived hierarchical category system.

When constructing a coding manual [80], we defined the categories and underpinned them with illustrative coding examples. Where required, coding rules were added to support coding decisions [101]. Additionally, we determined appropriate content analytical units (coding unit, context unit, and recording unit) for each research question. When coding competency statements, for example, we coded text snippets that clearly stated what an individual should be able to do. Thus, we coded competency statements such as “develop processes and procedures to mitigate the introduction of vulnerabilities during the engineering process.” After coding the material with the main categories, subcategories were derived using inductive category formation, a strategy to derive categories from material [80]. This way, we developed a deductive-inductive, hierarchically structured category system that served as the basis for answering our research questions.

3.4 Building and Validating the New Competency Model

To build the competency model, we chose an empirical rather than theoretical approach [12] and followed best practice recommendations [20, 79]. The 27 models served as the data basis upon which we developed the new model, and the competency model analysis (see Section 3.3) served as the method for deriving the structure and content of the model. The developed category system already represented the structure of the competency model (i.e., it categorized the competencies according to competency clusters and dimensions). Thus, by converting the category system to a competency model, we obtained the structure of the model. Determining the granularity of the competency model was another critical step in the construction process. Granularity concerns not only the number of competencies included but also the level of detail of each competency [20]. For the model to be exhaustive, we included a large number of competencies, namely 72. When determining the detail of a competency, we followed recommended guidelines [20, 79] and constructed a basic competency anatomy for every competency. Up to six behavioral indicators were selected to anchor the definitions [97]. Because of the coding process, competency categories were already assigned to behavioral indicators. We extracted the indicators and added them to the respective anatomies. In doing so, we avoided the frequently mentioned criticism of “using empty, overly general phrases or a listing of meaningless buzzwords” [102, p. 398].

The next step in the development process was to check and ensure curricular validity [59]. This validation step examined the extent to which the model’s content corresponded to the curricular content. Other competency modeling studies [67, 68] have also regarded this step as essential. Because we are familiar with the cybersecurity education landscape in Austria and consider it representative, we used the Austrian information security and cybersecurity curricula as our basis. The collection process resulted in 10 curricula [154, 155, 156, 157, 158, 159, 160, 161, 162, 163] that were analyzed according to content structuring QCA [70, 80]. To check curricular validity, we used the competency model as a deductive coding scheme. During the first coding session, similarly to Bröker and Magenheim [12], we found that the richness in detail of the content of the curricula varied, with many curricula only stating titles, topics, and content knowledge. Consequently, the corpus included many implicit competencies. Hence, we refined our coding rules to fit the data. Additionally, whenever competency candidates not previously captured by the category system emerged, a new competency category was inductively developed [80].

4.1 Demographic and Bibliographic Aspects

Figure 2 shows the cumulative number of sources per year. In 2006, the first competency model containing security content emerged. Since this emergence, interest has steadily increased, peaking in 2017 and 2019. With exception of 2009, competency models were released regularly, resulting in a continual supply of such models. With regard to the publication type, most sources constituted GL (23). Only some of the models (6) were formally published. As shown in Figure 3, the GL published in this area surpassed the formally published literature many times.

Fig. 2.

View Figure

Fig. 3.

View Figure

In the next step, we examined which countries have produced the most competency models. To rank countries based on the number of models, we extracted the countries of the universities to which the authors belonged. If several authors from several different countries had developed a model, one credit for each country was assigned. Figure 4 shows the top countries in terms of the releasing of competency models. According to Figure 4, the United States significantly outnumbered the rest of the countries. Noticeably, only 17 countries contributed to the growing body of competency models. Moreover, international collaborations seemed to be the exception, as only two models were developed through collaboration.

To evaluate the influence of models in terms of citation count, we extracted the respective information from Google Scholar and Web of Science. Of particular interest was the relationship between the citations for each article and articles’ years of publication. Showing the relationship, Figure 5 indicates that the actual citation count of the models depended on the database used. In effect, Google Scholar reported higher citation counts than Web of Science, whereas Web of Science indexed more models than Google Scholar. Contrary to expectations, recently published models were more influential than earlier work. Based on data from Google Scholar, the NICE Framework [140] led the list of the most influential models and was followed by the models of Prifti et al. [142] and Topi et al. [147]. Concerning data provided by Web of Science, the model developed by Prifti et al. [142] was cited the most.

Fig. 4.

View Figure

Fig. 5.

View Figure

4.2 Usages and Target Groups

We developed 18 categories describing the usages and applications of the competency models. According to the results, competency models provide several uses, ranging from performance management to policymaking. Table 4 provides an insight into the results, as well as descriptions and frequencies for all usages. As Table 4 indicates, the category “learning and competency development” led the list of the most frequently coded usages and was followed by the categories “assessment” and “development and evaluation of qualification programs.”

Subsequently, several target groups were identified, including job seekers, technical professionals, HR experts, qualification providers, and students. To understand how target groups can use competency models, we investigated the relationships between the target groups’ subcategories and the subcategories of usage by looking for co-occurrences. The findings indicate that most target groups can use competency models in several ways. Although market researchers and legislative bodies constitute target groups, the examination did not find any concrete use for these groups. Considering these findings, Table 5 presents a matrix that relates target groups to usages.

4.3 CyBOK Categories and KAs

As mentioned previously, the CyBOK [94] consists of 19 KAs grouped under five categories. To evaluate the content of competency models, we transformed these categories and KAs into a deductive category system and applied the system to the material. After the coding process, the categories were quantitatively evaluated. Figure 6 shows the category frequencies of the five CyBOK categories. The category “human, organizational, and regulatory aspects” topped the list of the most frequently coded KAs and was followed by the CyBOK category “attacks and defenses.” The categories “infrastructure security” and “systems security” were the least frequently named topics. Evaluating the codings quantitatively, we discovered an imbalance in terms of content. In fact, the competency models analyzed favored less technical content.

Fig. 6.

View Figure

Fig. 7.

View Figure

Fig. 8.

View Figure

To further analyze the models’ content, we conducted a simple configuration according to Kuckartz [70]. Figure 7 shows nine code configurations. Interestingly, the most frequent code configuration was formed by all five CyBOK categories. Put another way, many models included content covering a diverse set of knowledge and competencies, ranging from systems security to regulatory aspects. Excepting two models, most of the models included content related to at least two CyBOK categories. Moreover, the CyBOK category “human, organizational, and regulatory aspects” ran through all code configurations except two. Similarly, most of the configurations, except three, contained the CyBOK category “attacks and defenses.”

Separating the CyBOK categories into 19 KAs allowed the content to be analyzed in more detail. After the coding process, the material was not only reduced to 19 categories describing the models’ content from a bird’s-eye view but also quantitatively evaluated. Figure 8 shows a bar chart revealing the category frequencies of the 19 CyBOK areas. The most frequently coded KA was “security operations and incident management,” which was followed by “risk management and governance” and “secure software lifecycle.” Noticeably, the remainder of the categories occured to a considerably lesser extent. As with the five CyBOK categories, the quantitative evaluation of the 19 KAs revealed an imbalance in terms of content coverage. By comparison, areas such as “risk management and governance” and “security operations and incident management” were emphasized much more than the more technically oriented areas, such as “hardware security” and “physical layer and telecommunications security.”

4.4 Evaluation of Competency Models

Inspired by Plessius and Ravesteyn [91], we evaluated the competency models regarding content coverage. Assuming that the CyBOK areas represented the full range of possible cybersecurity topics, we used the 19 CyBOK categories as a deductive coding scheme to uncover the models’ content coverage. Because models that integrate security content do not claim to be exhaustive, the evaluation process focused on stand-alone models. Table 6 presents the results of the evaluation. In Table 6, beginning with the oldest model, the models are ordered by date. It is to be noticed that Table 6 indicates the presence of a specific KA, not the extent to which the models cover the KA.

As shown in Table 6, none of the competency models were exhaustive (i.e., no model studied could be considered complete in terms of content coverage). Missing only one KA, the Cybersecurity Competency Model [153] came closest to being complete. Additionally, the NICE Framework [140], as well as the Competency Model for Cybersecurity [127], omitted relatively few areas compared to other models (e.g., [136, 141]). Consequently, some models offered a general view of cybersecurity, whereas others were better understood as specialized models. However, although models differed, all of the models contained content relating to the KAs “security operations and incident management” and “risk management and governance.” To a lesser extent, the KAs “secure software lifecycle,” “software security,” and “human factors” also represented common ground. Last, each KA was addressed by two models at least.

4.5 Definitions of the Concept of Competency and Proficiency Levels

In the next step, we coded all of the text passages that provided clarification on the concept of competency. Of the 27 models, only 10 actually defined the term explicitly; the rest refrained from doing so. To better understand the competency construct, we examined the characteristics of the coded passages. Inspection of Table 7 reveals that the term is associated with various features, ranging from learnability to measurability, yet no single coded definition of the competency construct contained all of the attributes listed in Table 7. Instead, the list serves as an overview of all of the possible characteristics that a definition of the term competency could provide. To further elaborate on the characteristic “gradual expression,” we extracted the number of competency levels from each model. Table 8 shows that between two and seven competency levels were deployed to express different degrees of proficiency. Approximately half of the models did not group behavioral indicators into varying levels of proficiency.

4.6 Competency Classes

A common way to categorize competencies is to use competency classes. After inductively constructing 240 competencies, we counted the competencies per competency class. As shown in Figure 9, the class “professional competencies” encompassing those competencies associated with the solution of domain-related technical problems was the largest general competency category. Examples of professional competencies are inter alia, penetration testing, risk management, cloud security, and secure operating systems. Although the analysis identified a large set of professional competencies, only a few competencies were assigned to the classes “methodological competencies” (e.g., problem solving), “social competencies” (e.g., teamwork), and “personal competencies” (e.g., self-control). Since only 13 methodological, 10 social, and 17 personal competencies were identified in the analysis, it can be stated that competency models included a limited variety of nonprofessional competencies required by security experts.

Subsequently, we conducted a simple code configuration [70] to analyze the relationship between competency classes and models. By performing a simple configuration, code combinations can be examined. In other words, code configurations provide information about which competency classes are present in the respective competency models. Inspection of Figure 10 reveals that of the 27 models, most models only included professional competencies (18). Conversely, only a small number of models (4) covered the complete range of competency classes. Moreover, all of the other possible code configurations were addressed by two models at most. Consequently, methodological, social, and personal competencies for security professionals were not only underrepresented compared to professional competencies in terms of variety but also seldomly covered in competency models in general.

Fig. 9.

View Figure

Fig. 10.

View Figure

4.7 Competencies

The analysis identified a unique set of 240 competencies. To elaborate on the competency descriptions for each of the 240 competencies, we conducted a category-based evaluation [70]. In essence, we listed all material under one category and summarized the material’s meanings in a few sentences. When formulating the descriptions, we ensured that all definitions followed the same sentence structure and expressed the competency in an observable manner. Constructing all competencies based on this approach led to the formation of a competency pool, which can be found on Zenodo [8]. It is worth mentioning that the competencies extracted from the models did not prescribe any technologies to be used. Table 9 provides examples of competencies with their respective descriptions and associated KAs. For instance, with regard to the competency “network defense,” cybersecurity experts should be able to design, maintain, install, and apply a range of network defense systems. As regards the competency “secure design,” experts should be able to apply different design principles and perform threat modeling. Regarding the competency “malware analysis and defense,” security professionals are required to analyze different features of malicious software and combat malware.

To identify the category frequencies of each competency, we conducted a quantitative evaluation. By calculating category frequencies, a list of the 20 most frequently coded competencies could be produced. Inspection of Figure 11 reveals that although no competency was shared by all 27 models, the competency “risk management” topped the list of the most coded competencies and was followed by the competencies “risk assessment” and “incident management.”

Fig. 11.

View Figure

Interestingly enough, the competencies listed were associated with only eight areas of expertise: “risk management and governance,” “security operations and incident management,” “network security,” “human factors,” “law and regulations,” “forensics,” “secure software lifecycle,” and “software security.” The remaining KAs are not covered in Figure 11. Similarly, nonprofessional competencies, such as teamwork and stress tolerance, do not appear on the list. Consequently, the ranking of competencies shows that not only was the level of diversity of nonprofessional competencies lower than that of professional competencies but also their level of importance.

5.1 Competency Model for Information Security and Cybersecurity Professionals

By transforming the empirically developed category system into a competency model, we produced the competency model for information security and cybersecurity professionals, which is shown in Figure 12. The four competency classes serve as the high-level structure of the model. Unlike the nonprofessional competency classes, the professional competency class was divided into additional subcategories according to the structure of the CyBOK. As the CyBOK areas were insufficient to incorporate all of the identified competencies, we added three additional areas: physical security, job-specific skills, and CyBOK introduction. The latter refers to the foundational professional competencies of the security domain. The competency dimension of job-specific skills highlights the need for professional competencies beyond the security domain (e.g., technology watching). By design, the model incorporates not only professional competencies but also social, personal, and methodological competencies, thereby providing a holistic view of the competency profile of an cybersecurity expert. Furthermore, the model can be considered exhaustive, as the model’s content covers all of the CyBOK areas. In that regard, the proposed model is unique. As shown previously, none of the existing models fulfill this criterion.

When constructing the model, we did not include all 240 competencies. Instead, we selected the three most frequently mentioned competencies per KA from the generated pool. This approach is in line with the advice of the scientific literature, which recommends a manageable number of competencies [20, 75]. In sum, the model displays 72 competencies, which are underpinned with up to six competency indicators expressing the competency in action. Although we set a size limit, we consider this model to be a minimal framework that is expandable. For example, additional competencies from the competency pool could be added to the model. Regarding the definition of the competency construct, the model refers to the definition of Weinert [116]. Due to limited space, the full model, an in-depth description, and key data are provided online [8].

5.2 Curricular Validation

Overall, the model proved to be applicable to the categorization of the curricular content of 10 Austrian security programs. When checking for curricular validity, we found that most of the content of the 10 Austrian curricula could be integrated into the competency model. Consequently, the competencies of our model matching with competency aspects of the curricula reflect a significant number of abilities that are thought to be relevant to information security and cybersecurity experts and can be considered as approved on that basis. Table 10 provides a brief overview of the coding results. However, 11 competencies of the competency model for information security and cybersecurity experts were not mentioned in the curricula (e.g., customer service and technical support, hardware testing, secure hardware design, personal information, and creative thinking). At the same time, it also became apparent that it was impossible to integrate all curricular content into the model immediately, and some of the competencies suggested in the curricula were missing from the proposed model. Hence, 25 new competencies had to be developed inductively to be able to integrate all of the curricular content. Table 11 provides an insight into some of the newly developed competencies. Concerning the competency areas and dimensions, the structure of the model proved to be sufficient to categorize the competencies emerging from the curricular analysis.

6.1 Discussion in the Context of Literature

To better understand the field of information security and cybersecurity competency modeling, we explored competency models’ characteristics using a QCA. This study addresses the limitations of previous efforts. First, unlike related research, this study analyzes a broad array of competency models, namely 27. Previous research on competency modeling in the information security and cybersecurity domains and beyond [113] has focused on a smaller set of models, ranging from 1 model [14] to 14 models [111]. Second, this study adopts a systematic research method to uncover new, previously missed insights into competency modeling.

First, this study provides a complementary contribution to the discussion on the importance of cybersecurity topics. From this perspective, the competency models’ creators consider “human, organizational, and regulatory aspects” to be more important than knowledge about “systems security” and “infrastructure security.” Diving deeper, we found that less technical content, such as “risk management and governance,” was emphasized more than more technical areas, such as “cyber-physical systems security” and “hardware security.” These results are in line with recent work. Mapping four curricula against the CyBOK areas, Hallett et al. [48] also noticed an overemphasis on the areas “risk management and governance” and “security operations and incident management” in comparison with more engineering-focused areas. Similarly, Cabaj et al. [17] analyzed cybersecurity master’s programs and identified an increased interest in less technical content, such as human, societal, and organizational security. Nevertheless, from the results of other work, we can see that the discussion on the importance of topics continues to be a source of debate. For instance, the work by Parekh et al. [86] highlighted the importance of privacy, ethics, operating system security, and the rooting of trust in hardware. However, the CyBOK areas covering these topics are underemphasized in competency models. Another piece of work [107] analyzing the content of 71 cybersecurity education papers suggested that human, societal, and organizational security are much less important than data security and connection security, for example. In contrast, the present study rather suggests the opposite. Hence, what constitutes the core topics remains controversial at this point.

Next, the findings regarding the competency classes suggest an imbalance that could have profound consequences. Studies analyzing job advertisements have agreed that employers value professional competencies, as well as social, personal, and methodological competencies [13, 87, 92, 93]. Additionally, a recent review of the cybersecurity workforce’s future has argued that the skill set of cybersecurity experts must consist of more than just technical skills [27]. However, social, methodological, and personal competencies are not only underemphasized in number but are also completely missing from many competency models. Consequently, most of the studied competency models paint an incomplete picture of the competencies required in the security domain. Indeed, if security professionals lack personal and social competencies, they may not be successful at work. As discussed by Dawson ahd Thomson [27], lifelong learning is a valuable personal competency, and the absence of a commitment toward lifelong learning could render a security professional useless as the technology and threat landscape changes. Similarly, an inability to communicate complex security issues to nontechnical personnel and a lack of team playing skills reduce job performance [27]. Therefore, most of the analyzed competency models are only partially suitable for curriculum and workforce development, as they miss essential competency dimensions. Purely subject-oriented competency models must not be the only basis for curriculum and workforce development; they must be complemented by other sources.

The evaluation of the models’ content coverage pointed to a similar problem. As some models provided a general view of the domain and others were better understood as specialist frameworks, curriculum designers must carefully select models for curriculum design. For instance, if a designer wishes to build a program providing a holistic view of the security domain and chooses the work of the Hong Kong Monetary Authority [136] or the Personal Data Protection Commission [141] as the basis, they could achieve the opposite. Conversely, these models could meet expectations if a specialist focus were to be desired. As information about the content of models is crucial for the selection process, we believe that the information provided in Table 6 would facilitate decision making.

In accordance with previous efforts [13, 15], our work suggests that more professional competencies are required in terms of variety than nonprofessional ones (e.g., methodological, social, and personal competencies). Additionally, our findings stress the importance of professional competencies. However, the analysis of job advertisements by Brooks et al. [13] showed that teamwork was the most frequently sought-after competency of a security professional. Additionally, the work of Whitman [118] and Rahhal et al. [93] underscored the importance of soft skills. The World Economic Forum’s list of the top 10 most in-demand skills across industries also stressed the importance of nonprofessional competencies [121]. Nonprofessional competencies, however, do not appear in our top 20 list. However, concerning the importance of domain-specific professional competencies, our results comply, to a large extent, with the results reported in the literature. Similarly to our work, previous work has also suggested that competencies related to risk [13, 55, 93], networks [61, 93], incidents [13, 123], audits [13, 93], vulnerabilities [13, 61], and compliance [13] are among the most important competencies required by security professionals. In summary, although our results disagree with those of related work on the importance of nonprofessional competencies, the findings regarding the importance of professional competencies agree with those of the literature.

Furthermore, this study adds to the discussion on what characterizes a competent cybersecurity professional. First, unlike other efforts [13, 93], we underpinned the competencies with a thick description in an observable manner to provide clarification and avoid confusion. Second, we substantially expanded the set of competencies required in the industry and painted a more nuanced picture of the profession. However, the vast number of competencies could also be indicative of challenges for educational programs. Because a single program cannot promote all 240 competencies, curriculum designers must carefully select the competencies that are most important for security jobs [61].

Last, the findings suggest that competency models could help to tackle the skills gap. Given their uses related to competency development, workforce development, and curriculum design, competency models can help to address many of the pressing issues and challenges facing the cybersecurity education system and the labor market, including outdated curricula [18], the low responsiveness of the education system to changes in the cyber domain [10], the poor alignment between educational and industry requirements [28, 47], the insufficient communication between employers and educational institutions [26], the difficulties in hiring and retaining employees [55], the lack of investment in employees [26, 28], and the lack of clear career pathways [18]. Since competency models are applicable to the education system and the labor market, they can support the elimination of deficits with regard to supply and demand, which, in sum, are at the root of the shortage in skills [28]. With regard to employers, competency models, for example, can help to ensure and sustain the professional development of employees by facilitating the identification of skills gaps and ways to address them, supporting the identification of appropriate training opportunities, and providing a means to manage talent and plan succession. With regard to supply, competency models can support the construction and evaluation of educational programs. By mapping the curriculum against a competency model, curriculum designers can identify gaps and ways to address them. Additionally, competency models can complement proactive, cost-effective curriculum maintenance strategies based on monitoring and integrating changes to certification schemes [65]. Competency models are frequently updated (e.g., the e-CF and the Cybersecurity Competency Model), and considering such updates can strengthen curriculum maintenance efforts. In terms of curriculum design, the models facilitate discussion among key stakeholders and provide a clear indication of what cybersecurity professionals should be able to do. Additionally, competency models help to advance the professionalization of individual security occupations. Because a spectrum of different cybersecurity occupations exists, Burley et al. [16] argued against oversimplified one-size-fits-all professionalization mechanisms and recommended tailored occupation-specific activities. When considering occupation-specific activities, competency models can be used to identify occupation characteristics (e.g., competency requirements) and deficits (e.g., competency gaps). Using competency models also addresses some of the disadvantages associated with professionalization activities, such as high barriers to entry based on credentials [16]. In fact, the notion of competency highlights a worker’s actual capacity and job readiness in terms of competencies rather than formal achievements, which provides opportunities for people who have not undertaken formal training but have nevertheless developed competencies informally to enter the cybersecurity labor market [16, 71]. Therefore, we believe that the competency model analysis helps to address workforce issues, especially the qualitative aspects of the issues.

6.2 Application Scenarios

In this section, we wish to draw attention to the potential applications and uses of the competency model for information security and cybersecurity professionals. In principle, the model can be used in all application scenarios identified during the competency model analysis. However, here, rather than discussing all applications in detail, we focus on two important application scenarios and conclude by highlighting the model’s ability to narrow the skills gap. Tailoring the model to the concrete context could be beneficial when using the framework. For example, organizations could pick competencies linked to their missions and goals [20]. Educational staff might wish to adapt the model to fit with the regional and national contexts or the institute’s capacities.

Developing and evaluating qualification programs is one of the main applications of the competency model. The competencies of the model used to define programs’ learning outcomes constitute in-demand abilities that are not technology-specific. Consequently, curricula based on those competencies not only align with industry needs but are also more sustainable than curricula focusing on specific tools, as they are less subject to technological advances [13]. Moreover, as pure knowledge is insufficient to meet industry expectations [103], educational programs must provide opportunities to develop competencies. To support competency development, educational experts must rethink the learning culture. Competencies cannot be traditionally taught, and they can only be developed through hands-on experience in authentic learning environments [63]. Authentic learning environments and tasks can be constructed using the competency model’s behavioral indicators. These indicators suggest how a competency unfolds in action and provide guidance for creating test items and learning tasks. The nature of the learning task (well defined versus ill defined) and the learner’s familiarity with the task are hypothesized to influence the integration process of knowledge, skills, and attitudes (e.g., low-road integration and transformative integration) [5]. Hence, designers should think carefully about the nature of the task. In addition to serving as a tool for the creation of new curricula and content, the competency model is helpful in analyzing existing programs. As outdated curricula are seen as one rationale for the shortages in the security workforce [29, 94], evaluating the currency of curricula is imperative. The proposed model provides a holistic and up-to-date view of the security domain, making the model particularly useful in evaluating educational programs.

Competency models form the basis for competency-based HR management in organizations. Competency-based HR management aims to inform and improve HR systems, including recruitment and selection systems [3]. Evidence has suggested that recruiting competent cybersecurity experts is challenging for organizations [29, 55]. Our model can help to improve the effectiveness and efficiency of recruiting and selecting talent inside and outside the organization. By using the model’s competencies as building blocks for constructing job profiles, recruiters do not have to build profiles from scratch. Moreover, the competency model helps to build attractive job descriptions. Because the competencies have been defined and anchored with indicators to avoid misunderstandings, they are particularly useful in communicating an organization’s needs and attracting candidates who fit the profile. However, when informally studying job advertisements, we found that many organizations do not use a structured process to convey the meaning of competencies. By using the model, organizations can avoid this problem. Additionally, the competencies can be used in competency-based interviews during selection. Again, the behavioral indicators are particularly useful for this process and should help HR experts to decide which candidate should be appointed to the job in question. Consequently, using the model reduces the risk of recruiting and selecting the wrong people, thereby helping to avoid increased costs.

Moreover, the competency model can facilitate other approaches to narrowing the skills gap. For example, some organizations do not exactly know which qualifications and certifications are required for a particular job [42]. By mapping the qualifications and certifications against the competency model, organizations can determine which competencies are actually covered by the educational programs and sort out those requirements and certifications in the job advertisements that do not fit the role. This way, organizations can avoid mismatches [42]. Similarly, by mapping external educational platforms against the model, organizations can compare their offerings and find the training that best fits employees’ training needs. In doing so, organizations ensure cost-effective professional development and retain indispensable personnel. To satisfy workforce needs, the literature has also recommended the hiring of applicants with nontraditional backgrounds [26]. In this case, the competencies of the model are well suited to the assessment and validation of the person’s abilities and support decision making regarding the applicant’s employability. With regard to education, using the model to align industry requirements with educational efforts facilitates the development of cybersecurity experts with sought-after competencies who successfully transition from an academic environment to the industry. To resolve the tension between education and training [24], which represents a long-standing issue in alignment efforts, educational designers can construct authentic learning environments through practical tasks, group work, or internships, for example, using the model’s competencies. Last, the model can serve as a source of input for future competency-based cybersecurity curriculum guidelines and standards.

6.3 Limitations

The main issues related to threats to the validity of this article are an inaccurate category system and an incomplete dataset. Potential issues during the process of searching for and selecting sources can arise from limitations of the search terms, the databases used, and biases when applying inclusion and exclusion criteria. To minimize biases when applying inclusion criteria, we discussed controversial sources as a team and made consensus-based decisions. To minimize the risk of sources being missing, we used formal search terms and considered synonyms, which was followed by full forward and backward snowballing. Moreover, we used a wide range of databases to avoid issues resulting from the limitations of the search engines. The exclusive use of Austrian curricula to validate the competency model threatened external validity. The question concerns whether the curriculum corpus is up to date and covers all types of competencies. In that respect, we argue that the curricula contained enough information to map almost all of the competencies of the competency model. Hence, we are confident that the outcome of these processes constitutes a solid and inclusive basis.

Now, we wish to discuss the validity of the category system. Concerning inductive categories, signs of validity issues are high coding frequencies of residual categories, disproportionately high coding frequencies of subcategories, and disproportionate abstract categories [101]. The validity of the inductive categories is supported by the fact that no residual categories were used. Furthermore, disproportionately high frequencies for one subcategory within a main category were absent in most cases. However, in cases where they were not absent, we are confident that it was not a sign of an undifferentiated category but rather an empirical finding. This assumption is supported by the sheer number of inductive categories, which also indicate appropriate abstraction and differentiation. For evaluation, the second author, who was familiar with the study’s objectives and the procedure of content analysis, reviewed the category system so that the coding frame could also be considered valid from an expert’s perspective. Hence, we consider the validity of the category system as approved.