The Reliability of an Articial Intelligence Tool, ‘Decision Trees’, in Emergency Medicine Triage

Objective: Overcrowding is a challenge for emergency departments throughout the world. Triage systems categorize the patients based on medical emergencies in order to avoid the malpractices. The present study aimed to test the validity of an articial intelligence tool, ‘Decision Trees’, in emergency medicine triage. Methods: This prospective, cross-sectional, clinical study was conducted in an emergency department of a tertiary care hospital. A total of 1999 patients over 18 years were included into the study. The triage stuff were trained before the study with the Australasian Triage Scale. Two independent observers rate the ultimate triage category of study patients. A new algorithym by ‘Decision Trees’ was constructed at the end of the study. Results: The mean age of the study patients were 41.1±17,2 and 49.1 % of them (n=981) were male. There were 867 patients (43.3%) with triage category of ve and 14 (0.7%) patients with triage category one. The most common clinical descriptors of the patients were minimal pain with no high risk features 20.5% of them (n=409) and minor symptoms of low risk conditions 18.1% of them (n=362). There was an excellent consistency between two independent observers (kappa value: 0.997. The new algorithm by ‘Decision Trees’ rated wrong in only one patient. The accuracy rate was 99.9%. The consistency between ATS and ‘Decision Trees was excellent (kappa value: 0.999). There was average consistency between physicians and paramedics. (kappa value: 0.541). Conclusion: Decision trees as an articial intelligence model should be used for producing practical triage algorithms as a decision support tool in emergency departments.


Introduction
Overcrowding is one of the most important issues of emergency departments (ED) in recent years. Increase in overcrowding leads to delays in the evaluation and the treatments of patients that results with a negative impact on patient satisfaction and the quality of the medical care (1,2). The requirement for triage was arisen from to the necessity of selecting patients need urgent care.
Triage systems have had a common use in so many countries in recent years. The triage scales performed by the medical stuff categorizes the patients by using various medical parameters and presentation symptoms (3).
Arti cial intelligence, which models the logic of human brain, has been used since 1969 in some elds of medicine with promising results (4). There are so many techniques for arti cial intelligence, which have the capacity of solving problems, and the usage of these systems in medicine makes sense.
This study aimed to reveal the validity of an arti cial intelligence method, 'Decision Trees', in triaging the patients presented to ED.

Material And Method
This prospective cross sectional trial was conducted in an ED of a tertiary care hospital with an annually census of 73000 patients between 1-10 July in 2011. The local ethical committee approved the study.
Patients presented to the ED with any complaints over 17 years old included into the study. Patients under 18 years old and denied to give inform consent was excluded from the study.
Eight paramedics and one nurse working in the triage area were trained for Australia Triage Scale (ATS) (Supplement) before the study. During the training, the categories of ATS were de ned rstly and clinical scenarios composed the second stage. In the third stage, a practical application was implemented with 100 patients in the triage area.
Blood pressure, body temperature, oxygen saturation, heart and respiration rate, age, gender, presenting symptom, co-morbidities, Glasgow Coma Scale (GCS) score, verbal pain score that classifying the patients as serious pain, average pain, less pain and no pain and the triage category according to ATS were recorded to the study form. Blood pressure, pulse oxymeter and pulse rate were measured by a monitor. Body temperature was measured by temporal route (Exergen Temporal Artery Thermometry).
Triage Categories for ATS Category 1: Patients who need resuscitation and admitted to the resuscitation area immediately Category 2: Patients with a life-threating condition or risky for any limb. These patients were admitted to the telemetry unit or resuscitation area within 10 minutes.
Category 3: Patients with pathology of potential to progress life-threatining conditions or urgent interventions. These patients were admitted to the ED in 30 minutes. Category 4: Patients who have pathologies that may take care in one and two hours. These patients admitted to ED within 60 minutes.
Category 5: Patients with non-urgent complaints and admitted to the ED in 120 minutes.
The details of ATS have been displayed in Supplement.
A chart de ning ATS was posted on the triage area for triage stuff. Patients waiting for ED admission retriaged with 30 minutes intervals for any change in triage category.
Two independent senior residents at the end of the study determined the ultimate triage category. An associated professor on emergency medicine evaluated the patients if there was any inconsistency between two observers. Statistical Analysis 'Decision trees' as the arti cial intelligence model was used in the present study. 'Mathlab in classregtree method' was used for decision tree analysis. This method generates decision trees from the upper root point to down. There is a separation parameter and a condition in every branching point. Separation criteria providing most information were used in the algorithm (Gini's diversity index). A branch was formed in the right and left side of every criteria and value. The values below the selected variable and value are usually localized at the left sight. The algorithm continues to construct the peripheral branches until there is little point. The smallest point number in branches for calculation was used, 1. The other analysis of the study was performed by MedCalc. Numerical data was presented as mean ± standard deviation or median (interquartile range/min-max) and categorical data as rates. The consistency between decision trees, paramedic and physicians were displayed by kappa value.

Results
Two thousand patients who gave inform consent were included into the study. One patient with lack of data in the study form excluded from the study and 1999 patients included into the nal analysis.
The median GCS score was 15 (IQR: 15-15, min-max: 3-15). The demographics of study patients were presented in Table 1. to the ED during the study period. The frequency of clinical criteria according to ATS was displayed in Fig. 2 and Table 2a and 2b.  There was an excellent consistency between two independent observers (kappa value: 0.997). Only ve patients were rated different between two observers (0.2%).
A new triage algorithm was constructed by decision trees (Fig. 3). The new algorithm by 'Decision Trees' rated wrong in only one patient. The accuracy rate was 99.9%. The consistency between ATS and 'Decision Trees was excellent (kappa value: 0.999).
The impact of parameters in determening triage categorization was displayed in Fig. 4.

Discussion
The term arti cial intelligence was rstly de ned by John McCarthy as a science and engineering of producing clever machines, particularly clever computer software (5). Arti cial intelligence is the performing of behaviors, called intelligence when conducted by humanity, by machines. It is composed of methods that model the imagination of human being and the work form of human brain.
The ability of arti cial intelligence models in revealing the relations in a data set is used for forecasting the diagnosis, treatments and prognosis in many clinical scenarios (6). There are many techniques of arti cial intelligence in order to be used for various clinical problems. The potency of these techniques in investigating the diseases and treatment modalities is promising.
'Decision Trees' is one of the arti cial intelligence models that used as a decision support system. Decision Trees constructs a prediction model between a patient and disease with the parameters that we determine. Information is presented as a tree composed of steps; decision step, probability step and conclusion step.
The fundamentals of arti cial intelligence are based on 1965 and de ning applicability of fuzzy clusters in medicine in 1969 leads so researches about it. Successful researches in cardiology in the rst years were followed by new investigations in radiology and other areas of medicine (7)(8)(9)(10)(11)(12). Arti cial intelligence models are used in various areas of medicine nowadays (13)(14)(15)(16).
Besides the usage of arti cial intelligence in many areas of medicine, models of arti cial intelligence have been constructed also for emergency medicine. Arti cial neural network, a kind of arti cial intelligence, was tested to predict the acute coronary syndromes in emergency department, which found to have a reasonable performance with AUC value of 0.97, 0.93 and 0.95 from different hospitals. (17).
Bektas et al. used arti cial neural network to predict the craniocervical junction injuries in trauma patients and reported the AUC value as 0.912 (18).
In a recent trial by Azeez et al., validity of arti cial intelligence in triage system was tested by producing prediction models with retrospective chart data by using arti cial neural network and neuro-fuzzy system (19). The authors reported the accuracy of ANN and neuro-fuzzy system in test-set as 96.7 and 94, respectively. Although the speci cities of two methods in prediction the triage category accurately over 90%, the sensitivities are lower particularly in neuro-fuzzy models except in one of the prediction models in both methods.
The present study showed that 'Decision Trees' may be used to construct highly predictive and practical algorithms for implanting triage. The model with decision trees in the present trial rated wrong only one among 1999 patients. These methods are so exible that you can change and produce new algorithms with different variables that the investigator chooses.
Actually, the results of this trial do not claim that arti cial intelligence should substitute the paramedics or triage nurses. However, they should be used as decision aid tools for triaging patients, particularly for beginner triagers and also in crowded EDs which needs rapid decision.

Conclusion
Decision trees as an arti cial intelligence model should be used for producing practical triage algorithms as a decision support tool in emergency departments. Figure 1 Ultimate triage scores of study patients.

Figure 2
The frequencies of clinical criteria in ATS.

Figure 3
The algorithm of 'Decision Trees'.

Figure 4
The impact of parameters in decision of triage categorization.

Figure 5
Consistency between physicians and paramedics.

Supplementary Files
This is a list of supplementary les associated with this preprint. Click to download.