Predictive Validity of a New Triage System for Outcomes in Patients Visiting Pediatric Emergency Departments: A Nationwide Study in Korea

Triage is essential for rapid and efficient intervention in patients visiting an emergency department. In Korea, since 2016, the Pediatric Korean Triage and Acuity Scale (PedKTAS) has been implemented nationwide for the triage of patients visiting pediatric emergency departments (PEDs). The aim of this study was to evaluate the validity of the PedKTAS in patients who visit PEDs. This study was a retrospective observational study of national registry data collected from all emergency medical centers and institutions throughout Korea. We analyzed data from patients aged <15 years who visited emergency departments nationwide from January 2016 to December 2019. The hospitalization and intensive care unit (ICU) admission rates were analyzed on the basis of triage level. In total, 5,462,964 pediatric patients were included in the analysis. The hospitalization rates for PedKTAS Levels 1–5, were 63.5%, 41.1%, 17.0%, 6.5%, and 3.7%, respectively, and were significantly different (p < 0.001). The ICU admission rates for PedKTAS Levels 1–5 were 14.4%, 6.0%, 0.3%, 0.1%, and 0.1%, respectively, and were significantly different (p < 0.001). The hospitalization and ICU admission rates were highest for PedKTAS Level 1, and differences were significant based on the level. We identified that the PedKTAS is suitable for predicting the emergency status of pediatric patients who visit PEDs.


Introduction
A triage system that classifies the emergency status of a patient when visiting the emergency department is essential to provide preferential emergency treatment to patients [1]. In addition, triage systems can ameliorate inefficient operation and resource use caused by overcrowding in emergency departments. Therefore, various triage tools are used in many countries. The Emergency Severity Index [2], Manchester Triage Scale [3], and Canadian Emergency Department Triage and Acuity Scale (CTAS) [4] are the most widely used tools.
In 2001, the Canadian Pediatric Triage and Acuity Scale (PedCTAS) [5], a triage tool applied to pediatric patients, was developed in Canada and is used in most pediatric emergency departments (PEDs) in Canada. By using this tool, a nurse selects one of 38 main symptoms and then assigns one of five triage levels by synthesizing additional information about the patient, such as vital signs, baseline status, or age.
In Korea, the problem of inefficiency of the emergency medical system because of chronic overcrowding of PEDs has been continually raised. Before 2015, different triage systems were applied to children in each emergency department nationwide. Each emergency department classified patients' urgency into three levels by applying ESI, classified into five levels by applying the PedCTAS, and some emergency departments did not even apply the triage system. For this reason, it was difficult to share information about patients' emergency levels among hospitals or among 119 emergency medical services and medical staff at the pre-hospital stage. As a result, in 2010, an incident occurred in which a child with intussusception died because the emergency department of several hospitals did not recognize the emergent situation of the patient. Given these circumstances, a nationally unified triage system was needed to evaluate the degree of urgency in pediatric emergency patients. The Korean Society of Emergency Medicine (KSEM) developed the Pediatric Korean Triage and Acuity Scale (PedKTAS) [6] in 2015 by modifying and supplementing the PedCTAS to suit the situation in Korea because the PedCTAS was known as a triage tool whose reliability and validity were verified. The PedKTAS has been applied to pediatric patients visiting nationwide PEDs since 2016.
In the past few years, the PedKTAS has been implemented. However, studies on the validity of the PedKTAS have not yet been reported, and whether the PedKTAS is appropriate for predicting the clinical outcomes in patients visiting PEDs is unknown. The aim of this study was to identify the hospitalization rate and intensive care unit (ICU) admission rate on the basis of triage level after applying the PedKTAS and to compare them with the results of applying PedCTAS.

Study Setting, Design, and Population
This study was a retrospective, observational study. The data used in this study were retrieved from the National Emergency Department Information System (NEDIS). The NEDIS was established in 2003 as a nationwide emergency medical network in South Korea. Since 1 January 2016, all 31 district emergency medical centers (EMCs) in Korea, 120 regional EMCs, and 257 of 262 regional emergency medical institutions (EMIs) have participated and transmitted data to the NEDIS in real time. District EMCs are emergency centers assigned by the Minister of Health and Welfare to provide final care for critical emergency patients at tertiary general hospitals or general hospitals with more than 300 beds and to prepare for disaster situations. The average number of visits to District EMCs per year is approximately 40,000, and the average number of pediatric patients is approximately 12,000 per year. Regional EMCs are assigned by the provincial governor if they are suitable for emergency patient care among general hospitals; the average number of visits per year is approximately 30,000, and the average number of pediatric patients is approximately 8000 per year. Regional EMI is assigned by the mayor among general hospitals; the average number of patients visiting each year is approximately 6000, and the average number of pediatric patients is approximately 3000 per year.
Data for 4 years-from 1 January 2016 to 31 December 2019-were provided by the NEDIS. Patients younger than 15 years of age who visited the emergency department were included. Patients with KTAS 8 and 9 who were not classified by the PedKTAS or who visited a PED for certificate issuance were excluded from the analysis.
We considered each PED visit from arrival to departure as an independent case. The following variables were extracted from the NEDIS data during the study period: demographic variables, triage level, shift of arrival, EMC class, arrival mode, diagnosis at the PED, and final disposition. The demographic variables included age and sex. We divided the participants into four age groups: <1 year (i.e., infant), 1-4 years (i.e., young child), 5-9 years (i.e., preschool-and early-school-aged child), and 10-14 years (i.e., school-aged child). Diagnoses were determined by using the diagnostic codes of the Korean Standard Classification of Diseases-7 and the Korean version of the International Classification of Diseases-10th Revision [7,8]. "Hospitalization" included patients directly admitted to general wards, ICUs, operating rooms, and patients transferred to other hospitals for hospitalization. "ICU admission" was defined as direct admission from the PED to the ICU, admission to an ICU from an operating room, or transfer to another hospital for ICU admission. The PedKTAS was developed by consensus of experts from KSEM. The PedKTAS is applied to children under the age of 15, and patients are divided into 17 symptom groups. In total, 166 specific symptoms are included in each symptom group, so the trained nurses can select the symptoms corresponding to the patient. In the last step, the final PedKTAS level is determined by selecting the appropriate item from the primary or secondary considerations (consciousness, hemodynamic signs, degree of dyspnea, body temperature, hemorrhagic disease, immune state, and accident mechanism) to evaluate the severity and urgency of the symptom. A screenshot of the process of implementing the PedKTAS is provided in Appendix A. The priority of pediatric patient care is determined according to the degree of emergency, and if they are classified as Levels 1 or 2 in the first impression evaluation, they receive treatment with the highest priority. For other patients, the treatment area may change depending on whether an infectious disease is suspected, and the priority of treatment is determined according to the severity. In this process, if the severity is low, the waiting time may be long. Re-evaluation is recommended within 30 min, 1 h, and 2 h for Levels 3, 4, and 5, respectively, in order to evaluate whether the waiting patient's symptoms worsened. There are differences between the two triage tools in terms of application age, chief complaint list, and criteria for abnormal vital signs. Details of the comparison between the PedKTAS and the PedCTAS are described in Table 1.

Outcomes
The primary outcomes were the hospitalization and ICU admission rates based on the PedKTAS level. The secondary outcome was to compare the hospitalization and ICU admission rates with application of the PedKTAS and the PedCTAS, respectively.

Statistical Analysis
Continuous variables were expressed as the median and interquartile range, and categorical variables were expressed as the frequency and percentage. The PedKTAS level was treated as an independent variable, and the hospitalization and ICU admission rates were treated as the dependent variables. The hospitalization and ICU admission rates were compared using the odds ratios based on the PedKTAS level; a logistic regression analysis was conducted at 95% reliability, adjusted for age, sex, mode of arrival, and diagnosis. All analyses were conducted using R version 4.0.0 (R Foundation for Statistical Computing, Vienna, Austria). Because the sample size of the analysis data was very large, the probability level of significance was set at p < 0.001.

Results
From January 2016 to December 2019, 6,847,960 patients were extracted by using the NEDIS. Among these, 1,322,689 patients without a triage classification and 62,307 patients with missing data were excluded. Ultimately, 5,462,964 patients were analyzed in this study ( Figure 1). On the basis of the NEDIS data, the 1-to 4-year-old age group constituted the highest proportion of patients (48.1%), followed by the 5-to 9-year-old age group (24.9%). Boys accounted for 57.2% of the total participants. The proportions of patients were 0.2%, 3.4%, 31.6%, 55.8%, and 9.0% for PedKTAS Levels 1, 2, 3, 4, and 5, respectively. The demographic data of the patients in the study group are summarized in Table 2.  On the basis of the NEDIS data, the 1-to 4-year-old age group constituted the highest proportion of patients (48.1%), followed by the 5-to 9-year-old age group (24.9%). Boys accounted for 57.2% of the total participants. The proportions of patients were 0.2%, 3.4%, 31.6%, 55.8%, and 9.0% for PedKTAS Levels 1, 2, 3, 4, and 5, respectively. The demographic data of the patients in the study group are summarized in Table 2. The hospitalization rates were 63.5%, 41.1%, 17.0%, 6.5%, and 3.7% for PedKTAS Levels 1, 2, 3, 4, and 5, respectively, and were significantly different (p < 0.001). The ICU admission rates were 14.4%, 6.0%, 0.3%, 0.1%, and 0.1% for PedKTAS Levels 1, 2, 3, 4, and 5, respectively, and were significantly different. Deaths in the PED were 0.6% of 75 patients at PedKTAS Level 1, and 47 patients, 29 patients, and 9 patients at PedKTAS Levels 2, 3, and 4, respectively. The emergency department length of stay (LOS) was the longest at PedKTAS Level 1 at 268.8 min, and the emergency department LOS decreased from PedKTAS Level 1 to Level 5 (Table 3). The PedKTAS Level 1 in the trauma group had the highest ICU admission rate at 29.5%. Regarding medical diseases, ICU admission rates were the highest in the cardiac disease group at PedKTAS Level 1 and Level 2, at 26.4% and 14.4%, respectively. The results of hospitalization rate and ICU admission rate according to PedKTAS Level for each disease group are shown in Table 4.

Discussion
The emergency triage system, which determines patients' severity, is essential for the priority treatment of emergency patients and the efficient distribution of medical resources. This study is the first validation study conducted after the implementation of the PedKTAS in Korea. The results of this study will provide sufficient evidence for predicting the emergency status of patients in PEDs.
In this study, we found that when the PedKTAS was applied to patients visiting PEDs, the hospitalization and ICU admission rates were the highest among patients in PedKTAS Level 1. We also found that the hospitalization and ICU admission rates significantly decreased as the severity of the PedKTAS level decreased from 1 to 5. These results indicated that the PedKTAS can reflect differences in patients' statuses, depending on their level. On the basis of this study, it can be said that the higher the PedKTAS level from 5-1, the more severe the patient's condition, and the faster the examination and treatment that are needed. Previous studies on the validity of the PedCTAS, which were conducted on the basis of the PedKTAS, also reported similar results. An Israeli study [11] reported  Figure 3B shows the ICU admission rates by triage level based on hospital type. In district EMCs, the ICU admission rates based on the triage level were 17.9%, 7.4%, 0.4%, 0.1%, and 0.2% for PedKTAS Levels 1, 2, 3, 4, and 5, respectively. In regional EMCs, the ICU admission rates based on triage level were 11.8%, 4.9%, 0.3%, 0.1%, and 0.1% for PedKTAS Levels 1, 2, 3, 4, and 5, respectively. In regional EMIs, the ICU admission rates based on the triage level were 0%, 3.0%, <0.1%, <0.1%, and <0.1% for PedKTAS Levels 1, 2, 3, 4, and 5, respectively.

Discussion
The emergency triage system, which determines patients' severity, is essential for the priority treatment of emergency patients and the efficient distribution of medical resources. This study is the first validation study conducted after the implementation of the PedKTAS in Korea. The results of this study will provide sufficient evidence for predicting the emergency status of patients in PEDs.
In this study, we found that when the PedKTAS was applied to patients visiting PEDs, the hospitalization and ICU admission rates were the highest among patients in PedKTAS Level 1. We also found that the hospitalization and ICU admission rates significantly decreased as the severity of the PedKTAS level decreased from 1 to 5. These results indicated that the PedKTAS can reflect differences in patients' statuses, depending on their level. On the basis of this study, it can be said that the higher the PedKTAS level from 5-1, the more severe the patient's condition, and the faster the examination and treatment that are needed. Previous studies on the validity of the PedCTAS, which were conducted on the basis of the PedKTAS, also reported similar results. An Israeli study [11] reported that when the PedCTAS was applied, as the severity of the PedCTAS level increased from 5 to 1, a strong correlation existed between the hospitalization rate and ICU admission rate. Another study [12] conducted in Costa Rica also reported that, as the severity of the PedCTAS level increased, the hospitalization rate and ICU admission rate increased.
In a multicenter study [9] using the PedCTAS, the hospitalization rates based on triage level were 61%, 30%, 10%, 2%, and 0.9% for Levels 1-5, respectively [9]. These results were lower than the expected hospitalization rates based on PedCTAS at all levels except for Level 5. In the current study, the hospitalization rates for PedKTAS Levels 2 and 5 were 41.1% and 3.7%, respectively, which was within the expected hospitalization rate and lower than the expected hospitalization rate, respectively, of the other levels.
However, compared with the findings of a previous multicenter study, [9] the current study result was closer to the expected hospitalization rates. The reason for the lower-thanexpected hospitalization rate was likely related to over-triage. When triaging pediatric patients, the important criteria are the vital signs and whether the patient appears unwell. However, when pediatric patients visit PEDs with fear, their irritability worsens, and their condition may be evaluated as more severe than it actually is [13]. Another study [14] reported no difference in the hospitalization rates of children who visited PEDs because of fever within 24 h when the triage level was lower than the actual triage level. This over-triage may explain why the actual hospitalization rate was lower than expected.
The ICU admission rate based on triage level differed between the PedKTAS and the PedCTAS. The ICU admission rate at Level 1 was lower for the PedKTAS than for the PedCTAS, but the ICU admission rate at Level 2 was higher for the PedKTAS than for the PedCTAS. The PedCTAS multicenter study included 12 tertiary care universityaffiliated hospitals with ICUs; therefore, admitting Level 1 patients to the ICU may have been easier. However, this study analyzed nationwide data and included hospitals of all sizes. Thus, the ICU admission of Level 1 patients would be restricted in a tertiary hospital without a pediatric ICU. However, in a secondary hospital with an ICU, admitting Level 1 patients and Level 2 patients to the ICU would be easier. In Korea, the ICU admission rate of PedKTAS Level 1 patients in a single tertiary care university-affiliated hospital was 34% [15]. The fact that the ICU admission rate of PedKTAS Level 1 patients is lower than that of PedCTAS Level 1 patients has important clinical significance. The PedKTAS may be overestimating the severity and urgency of pediatric patients. It may be better to overestimate than underestimate the condition of pediatric patients. However, in places where the use of medical resources is limited, such as in an emergency department, too many Level 1 patients requiring prompt treatment may be screened, and treatment of patients actually requiring emergency treatment may be delayed. Therefore, it may be necessary to modify the evaluation items for selecting PedKTAS Level 1 patients.
In this study, hospitalization and ICU admission rates were identified via the PedKTAS level based on hospital type through subgroup analysis. District EMCs had the highest hospitalization rate at all levels, followed by regional EMCs and EMIs. The ICU admission rate also showed a similar trend in the hospitalization rate based on hospital type. This finding is likely because higher-level hospitals are equipped with specialized medical staff and facilities to accommodate many patients. A previous study [16] in the United States also reported that tertiary hospitals had a higher hospitalization rate because of the availability of 24 h specialist care, advanced testing, and other services that are unavailable in small hospitals. We identified that PedKTAS Level 1 patients were not admitted to ICUs in regional EMIs, likely because small hospitals, such as regional EMIs, have several restrictions on the treatment of Level 1 pediatric patients, such as lack of specialists or equipment. Therefore, they are transferred to higher-level hospitals.
The triage of pediatric patients visiting the emergency department is performed by nurses. Nurses who perform triage using the PedKTAS must complete 6 h of training [17]. The educational process consists of basic learning lectures on severity and urgency classification, group discussions on severity classification cases, and written tests. Nurses who completed this training course showed a considerable level of reliability in severity classification using the PedKTAS. One study reported a weighted kappa value of 0.76 for inter-nurse agreement [18]. This result shows that the range of weighted kappa values identified in the meta-analysis study of inter-rater agreement of the PedCTAS is 0.41 to 0.74, which is significant [19].
This study had several limitations. First, the data for this study were provided anonymously. Therefore, information on the same patient is likely to have been included more than once. However, this factor is an inherent limitation of studies with anonymized data and is not specific to this study. Second, this study analyzed the triage level results classified in actual clinical practice. Therefore, conducting in-depth analysis, such as inter-rater reliability analysis, was not possible. However, this inherent limitation is unavoidable in retrospective observational studies. Third, 1,309,867 patients not classified by the PedKTAS were excluded from this study. They accounted for approximately one-fifth of the total number of pediatric patients analyzed in this study, and most visited local EMIs. This was identified to have occurred in the early days of implementing the PedKTAS nationwide. Due to this, there is a possibility that sample selection bias exists. In the future, it is thought that additional analysis will be needed by extending the study period except for the initial period of KTAS implementation. Finally, this study reflected the medical situation in Korea; the results may differ depending on the medical situation and the environment in other countries.
The emergency department has the characteristics that the number of patients visited and the condition cannot be predicted, and treatment must be performed with limited medical resources. Therefore, overcrowding in emergency departments is an important problem that is directly related to the threat to patient safety. Efforts are being made in various ways to relieve the overcrowding of emergency departments. Using the triage system, the flow of patients can be controlled in the pre-hospital stage, and the patient's condition can be accurately evaluated at the initial stage so that patients visiting the emergency department can wait safely in the hospital stage. Through this, overcrowding in the emergency department can be improved. Continuous efforts and research of many researchers will be required for the development and performance improvement of the triage system.

Conclusions
The hospitalization and ICU admission rates based on PedKTAS levels were the highest for PedKTAS Level 1 and were significantly different based on the level. We identified that the PedKTAS is suitable for predicting the emergency status of pediatric patients who visit PEDs.
Funding: This study received no external funding.

Institutional Review Board Statement:
The study was conducted in accordance with the Declaration of Helsinki and was approved by the Institutional Review Board (IRB) of the Catholic University of Korea (IRB approval no. KC21ZASI0798).

Informed Consent Statement: Not applicable.
Data Availability Statement: Data are available on request owing to restrictions, such as privacy or ethics.

Conflicts of Interest:
The authors declare no conflict of interest.

Appendix A
Children 2023, 10, x FOR PEER REVIEW 11 of 12 Figure A1. Screenshot of the process of implementing Pediatric Korean Triage and Acuity Scale.