Introduction

Background/rationale

The Korean Nursing Licensing Examination (KNLE) is a representative criterion-referenced assessment, based on the latest guidelines for nursing practice. It evaluates the minimum knowledge and skills that nursing graduates must possess to efficiently perform their duties [1]. However, the subjects that this licensing examination should cover are under constant debate. It has been suggested that the learning objectives of the 8 major nursing subjects overlap and that the KNLE may not be adequately equipped to appropriately evaluate the job competency of new nursing graduates [2,3]. Hence, continuing efforts are being made to verify the job-based knowledge of nursing graduates and to resolve the issue of overlap among the objectives of nursing subjects [4-7]. As part of this effort, Kim et al. [1] reviewed the feasibility of linking nurse jobs with the licensing examination. Song et al. [3] proposed subjects for an integrated nursing examination by extracting 1,303 statements from 481 knowledge statements. Consequently, 134 activity statements based on graduate nurse job activities in the field of nursing practice and 481 knowledge statements have been developed for the KNLE. These knowledge statements can be grouped into the following 8 activity categories: management of care and improvement of professionalism, safety and infection control, management of potential risk, basic care, physiological integrity and maintenance, pharmacological and parenteral therapies, psychosocial integrity and maintenance, and health promotion and maintenance. However, despite ongoing efforts to design a more appropriate licensing examination, the process of sharing information, collecting opinions, and securing agreement among academic societies and related organizations, such as those involved in nursing education, and nursing practice experts is still insufficient.

Objectives

This study aims to suggest the number of test items in each of 8 nursing activity categories of the KNLE. The specific goals of the study were as follows: first, to elicit opinions on the distribution of the number of test items from 7 major academic societies affiliated with the Korean Society of Nursing Science; and second, to obtain perspectives from 4 professional associations on those opinions from 7 academic societies. It is hoped that the results will help resolve redundancy among the objectives of the 8 nursing subjects by proposing job-based item development for the KNLE.

Methods

Ethics statement

Institutional Review Board approval was not required for this process, because the survey questionnaire items did not contain information about the individual characteristics of the participants. Instead, it had information about the policies of societies and associations. No sensitive or individually identifiable information was obtained from the participants.

Study design

This was a survey-based descriptive study and content analysis, described according to the STROBE (Strengthening the Reporting of Observational studies in Epidemiology) statement and 6 steps for conducting thematic analysis [8].

Setting

A job-based item model for the KNLE to resolve redundancy among the objectives of the 8 nursing subjects was provided by the authors (Supplement 1). Two opinion surveys were conducted from members of 7 academic societies to assess the suitability of this model. The first opinion survey was conducted from March 19 to 31, 2021. The second opinion survey was conducted from April 30 to May 14, 2021, after sharing the results of the first survey. Approximately 12–27 members from all academic societies and associations participated in the opinion-collection process. Next, 4 experts from 4 nursing associations reviewed the opinion-survey results from May 21 to June 4, 2021. The review results were summarized by content analysis. Official e-mails were sent to each academic society and association, and the executives of each academic society and association gave their opinions. After summarizing the survey and analysis results, these results were compared with those reported by Tak et al. [9] and the National Council License Examination for Registered Nurses (NCLEX-RN) of the United States [10]. The overall workflow of the study is depicted in Fig. 1. Although a public hearing was held, the content was not reflected in this article. For more information on the public hearing, refer to Supplement 2.

Participants

Seven academic societies participated in this survey, including the following: the Korean Academic Society of Community Health Nursing, the Korean Academic Society of Nursing Administration, the Korean Academic Society of Fundamentals of Nursing, the Korean Society of Adult Nursing, the Korean Academic Society of Child Health Nursing, the Korean Academic Society of Psychiatric and Mental Health Nursing, and the Korean Society of Women Health Nursing. The 4 expert associations participating in the review were the Korean Society of Nursing Science, the Korean Association of College of Nursing, the Korean Deans Association of Nursing College, and the Korean Hospital Nurses Association. The executives from the 7 academic societies included nursing education experts, while the reviewers from the associations included both nursing educators and practice experts.

Variables

The primary outcome was the number of items for each category.

Data sources/measurement

The results of the opinion survey of the KNLE among the 7 academic societies and 4 associations were used as data sources. The survey questionnaire for 7 academic societies consisted of the items and the responses by the participants (Supplement 3). A more precise source report for the article is available in Supplement 2.

Bias

There was no bias in selecting participants because this was a descriptive study in which no comparison was conducted.

Study size

We did not perform any sample size estimation because this was a descriptive study without any further analysis.

Statistical methods

The collected quantitative data were organized through content analysis by focusing on key concepts and analyzed with descriptive statistics such as frequencies and percentages.

Results

Participants

Table 1 presents the participating societies and associations.

Number of items suggested for the 8 categories by 7 academic societies

The KNLE items comprising 8 activity categories based on the results of the second opinion survey of the 7 member societies are listed in Table 2 (Dataset 1). The exam items included the following: management of care and improvement of professionalism, 41 items; health promotion and maintenance, 33 items; basic care, 37 items; physiological integrity and maintenance, 63 items; psychosocial integrity and maintenance, 41 items; safety and infection control, 16 items; pharmacological and parenteral therapies, 11 items; and management of potential risk, 31 items. In total, 273 items were confirmed. Two items on nursing history were exempted. Another 20 test items related to medical laws were also exempted.

Opinions of experts from 4 professional associations

The 4 professional associations emphasized that the safety and infection control activity category and pharmacological and parenteral therapies activity category play major roles in clinical and community nursing practice in real-life situations. Therefore, in the guideline on item-making, the proportion of items in the pharmacological and parenteral therapy activity category increased from 10% (in the guideline on item-making presented by Tak et al. [9] in 2019) to 12%, and that of the safety and infection control activity category also increased from 10% to 12%. The results of the opinion survey of the 4 expert associations on the KNLE standards for each activity category presented by the 7 academic societies mentioned above are summarized in Table 3.

Comparison with research on other examination items

We compared the results obtained with those of the NCLEX-RN evaluation conducted in 2019 [9] and those reported by Tak et al. [9] in a study that was also conducted in 2019 (Table 4). Among the items presented by Tak et al. [9], the percentage of items related to the management of care and improvement of professionalism, basic care, psychosocial integrity and maintenance, and health promotion and maintenance activity categories were the same as that determined by the opinion surveys conducted for this study. In the examination subjects developed by Tak et al. [9], the percentage of items in the management of potential risk activity category was 16% and that of items in the physiological integrity and maintenance activity category was 20%, which were higher than those in the NCLEX-RN Test Plan (2019) [10]. Referring to the opinion-survey results, we noted that the maximum percentage of the management of risk potential activity category in the distribution of the NCLEX-RN Test Plan (2019) was 15% (9%–15%) [10]. The percentage of the physiological integrity and maintenance activity category in the NCLEX-RN Test Plan (2019) was adjusted to 17%, corresponding to the maximum range of the distribution (11%–17%) [10]. In the opinion survey of the academies, the percentages of items in the safety and infection control and pharmacological and parenteral therapies activity categories were 4%–6%p lower than those in the item model suggested by Tak et al. [9] and 6%–10%p lower than those in the NCLEX-RN Test Plan (2019) [10].

Discussion

Interpretation

This study proposes a guideline for item-making based on the item model of Tak et al. [9] in 2019 and the NCLEX-RN Test Plan (2019 NCLEX-RN), opinion surveys of 7 academic societies and 4 associations, interviews with experts on clinical and community nursing practice, and the results of a content validity survey. The distribution of items in 4 activity categories—management of care and improvement of professionalism, basic care, physiological integrity and maintenance, and health promotion and maintenance—were maintained. However, the number of items in the management of potential risk and psychosocial integrity and maintenance areas were adjusted downward. The number of items in the areas of pharmacological and parenteral therapies and safety and infection control was increased.

In the expert opinion survey of the associations, some differences were noted in the understanding of the following 4 activity categories: management of potential risk, basic care, physiological integrity and maintenance, and pharmacological and parenteral therapies. To gain a clear understanding of the 8 activity categories, the NCLEX-RN Test Plan should be referred to [10] and the definition and goals for each activity category should be presented more clearly. An opinion was expressed that the 8 activity categories should be arranged systematically in a sequence. Furthermore, a specialist society needs to be established to collect opinions and reach a consensus regarding clinical and community nursing practice. Additionally, the 481 knowledge statements have different levels of specificity for each group. Some of the knowledge statements are not linked to any major subject, while others are linked to 2 or more major subjects. Therefore, the process of collecting opinions and reaching a consensus regarding knowledge statements should be modified and take precedence over other procedures. Because the competency of new nursing graduates is directly related to the quality of nursing and patient safety [11,12], it is necessary to prepare a guideline on item-making that can redirect the aim of the examination to a knowledge-verification model to evaluate examinees’ competency.

Limitations

Given the basic research nature of this study, which investigated the opinions of experts on the job-based integrated national examination model and a guideline on item-making for new nursing graduates, there is a possibility that it lacks objective justification.

Generalizability

The problems and directions for improvement proposed in this study relate to the KNLE. The study results are based on the cultural and situational background of the Republic of Korea. Hence, the results of this study have limited applicability to nursing licensing examinations of other countries or licensing examinations for other job groups.

Suggestions for further studies

Future studies should increase the level of objective legitimacy achieved in this study. For this, a task force team composed of nursing education and practice experts should be formed, and a consensus should be reached through sufficient discussion and careful collection of experts’ opinions.

Conclusion

The study results suggest that the distribution of items in the following 4 activity categories should be maintained: management of care and improvement of professionalism, basic care, physiological integrity and maintenance, and health promotion and maintenance. However, the number of items of the management of potential risk and the psychosocial integrity and maintenance area should be reduced. The number of items in pharmacological and parenteral therapies, and safety and infection control should be increased.

Article information

Authors’ contributions

Conceptualization: KK, YK, YS. Data collection: KK, SYK, YK, YK, HK, YS, JC, MC, HSL. Methodology/analysis/validation: KK, YK, SYK, HK, JC. Project administration: KK, HSL. Funding acquisition: KK, HSL. Writing–original draft: KK, SYK, YK, HK, JC, MC, HSL. Writing–review & editing: KK, YK, YS.

Conflict of interest

No potential conflict of interest relevant to this article was reported.

Funding

This work was supported by the Korea Health Personnel Licensing Examination Institute (Fund ref ID: 10.13039/501100003647) research fund (RE02-2111-05).

Data availability

Data files are available from Harvard Dataverse: https://doi.org/10.7910/DVN/S4CZRO

Dataset 1. Raw response data from 7 academic societies on the 2 rounds of the opinion survey.

jeehp-20-18-dataset1.xlsx

Acknowledgments

Supplementary materials

Article information

Editor’s note

This article is a part of a research report titled “Basic research on the development of item criteria based on the integrated model of the Korean Nursing Licensing Examination,” supported by the Korea Health Personnel Licensing Examination Institute. The source report has an enormous amount of research and survey results. Only a part of the content was included in this article. For more specific content, the original report is provided in Supplement 3. Although it is written in Korean, readers who are interested in this topic will be able to obtain more information. Research has long been conducted on the same topics and subject classification based on job descriptions. However, the test subjects of the Korean Nursing Licensing Examination are still based on the major fields of nursing, including nursing management, fundamental nursing, women’s health nursing, adult nursing, children’s health nursing, psychiatric and mental health nursing, community health nursing, and medical law. Therefore, competency, activity, or job-based integrated subjects should be developed based on consensus among interested groups.

Fig. 1.

Flow of the study. KNLE, Korean Nursing Licensing Examination; NCLEX-RN, National Council License Examination for Registered Nurses.

References

• 1. Kim YK, Kim KH, Kim MH, Kim SH, Kim YH, Shin S, An SH, Oh SE, Lee KM. A study on the improvement of test questions development standards and development procedures for the practice-based nurse national examination [Internet]. Korea Health Personnel Licensing Examination Institute; 2018 [cited 2022 Oct 31]. Available from: https://rnd.kuksiwon.or.kr/last/lastList.do?MENU_ID=C-01-01
• 2. Park HR, Kim KH, Kang YS, Kim SY, Park EH. A comparative study of the educational contents of national examination subjects by interdisciplinary system for nurses [Internet]. Korea Health Personnel Licensing Examination Institute; 2011 [cited 2022 Oct 31]. Available from: https://rnd.kuksiwon.or.kr/last/selectFileDown.do?attach_id=2015030500032
• 3. Song R, Kim YK, Kang SY, Park YW, Park HS, Shin S, Shin YS, Oh SE. The model construction for integrated test items for the nursing licensing examination. Korean Nurses Activity Association; 2018.
• 4. Shin S, Kim GS, Song JA, Lee I. Development of examination objectives based on nursing competency for the Korean Nursing Licensing Examination: a validity study. J Educ Eval Health Prof 2022;19:19. https://doi.org/10.3352/jeehp.2022.19.19 ArticlePubMedPMC
• 5. Park IS, Suh YO, Park HS, Ahn SY, Kang SY, Ko IS. The job analysis of Korean nurses as a strategy to improve the Korean Nursing Licensing Examination. J Educ Eval Health Prof 2016;13:24. https://doi.org/10.3352/jeehp.2016.13.24 ArticlePubMedPMC
• 6. Park IS, Suh YO, Park HS, Ahn SY, Kang SY, Kim KS. Relevance of the test content of the Korean Nursing Licensing Examination to nursing job. J Educ Eval Health Prof 2016;13:23. https://doi.org/10.3352/jeehp.2016.13.23 ArticlePubMedPMC
• 7. Park IS, Suh YO, Park HS, Kang SY, Kim KS, Kim GH, Choi YH, Kim HJ. Item development process and analysis of 50 case-based items for implementation on the Korean Nursing Licensing Examination. J Educ Eval Health Prof 2017;14:20. https://doi.org/10.3352/jeehp.2017.14.20 ArticlePubMedPMC
• 8. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol [Internet]. 2006 [cited 2022 Oct 31];3:77-101. Available from: http://eprints.uwe.ac.uk/11735
• 9. Tak YR, Kang YH, Song RY, Song JA, Han MK. Improvement plan for nurse national examination and license management system in accordance with changes in health and medical environment [Internet]. Korea Health Personnel Licensing Examination Institute; 2019 [cited 2022 Oct 31]. Available from: https://rnd.kuksiwon.or.kr/last/selectLastDetail.do?MENU_ID=&reportno=2020011301&sYear=2019&sJssfc_code=05&sResearchindexcd=&researcherid=&sRsrchtasknm=&PAGE_NUM=1&PER_PAGE=10&IS_PAGE_NEW_SEARCH=Y&TOTAL_PAGE=2
• 10. National Council of State Boards of Nursing. 2019 NCLEX-RN Test Plan [Internet]. National Council of State Boards of Nursing; 2018 [cited 2022 Oct 31]. Available from: https://www.ncsbn.org/publications/2019-nclexrn-test-plan
• 11. Victor J, Chavez LS, Podlesney C. Do predictor exams really predict readiness for professional nursing practice? Clin Simul Nurs 2021;50:48-54. https://doi.org/10.1016/j.ecns.2020.10.005 Article
• 12. Kwon IG, Cho YA, Cho MS, Lee YH, Kim MS, Kim KS, Choi AS. Status of training for new nurses’ practical adaptation support and experience of role fulfillment of new nurses [Internet]. Hospital Nurses Activity Association; 2018 [cited 2022 Oct 31]. Available from: https://khna.or.kr/home/pds/utilities.php?bo_table=board1&page=4&page=3

Figure & Data

References

Citations

Citations to this article as recorded by