Development of clinical prediction model to guide the use of CT head scans for non-traumatic Thai patient with seizure: A cross-sectional study

The aim of this study was to develop clinical predictor tools for guiding the use of computed tomography (CT) head scans in non-traumatic Thai patients presented with seizure. A prediction model using a retrospective cross-sectional design was conducted. We recruited adult patients (aged ≥ 18 years) who had been diagnosed with seizures by their physicians and had undergone CT head scans for further investigation. Positive CT head defined as the presence of any new lesion that related to the patient’s presented seizure officially reported by radiologist. A total of 9 candidate predictors were preselected. The prediction model was developed using a full multivariable logistic regression with backward stepwise elimination. We evaluated the model’s predictive performance in terms of its discriminative ability and calibration via AuROC and calibration plot. The application was then constructed based on final model. A total of 362 patients were included into the analysis which comprising of 71 patients with positive CT head findings and 291 patients with normal results. Six final predictors were identified including: Glasgow coma scale, the presence of focal neurological deficit, history of malignancy, history of CVA, Epilepsy, and the presence of alcohol withdrawal symptom. In terms of discriminative ability, the final model demonstrated excellent performance (AuROC of 0.82 (95% CI: 0.76–0.87)). The calibration plot illustrated a good agreement between observed and predicted risks. This prediction model offers a reliable tool for effectively reduce unnecessary use and instill confidence in supporting physicians in determining the need for CT head scans in non-traumatic patients with seizures.


Introduction
Patients experiencing seizures often present to the emergency department (ED), accounting for approximately one-third of neurological conditions and 1% of all ED visits [1,2].Seizures refer to the signs and/or symptoms that occur as a consequence of abnormal neuronal activity, which can be caused by various sources, including stroke, tumors, alcohol withdrawal, and infections.According to guidelines [3,4], the use of a Computed Tomography (CT) head scans as a diagnostic test should be reserved for patient with high risk for positive finding on CT such as old age, with focal neurological deficit, with history of malignancy.However, in routine clinical practice, the decision to perform a CT head scans is left to the discretion of the physician.
Due to increased availability and fast scanning speeds, almost all patients with seizures or a history of symptom related seizures (uncontrolled body stiffening, limb jerking, or sensory aura) usually undergo a CT head scans as part of their workup [5][6][7].Based on previous evidence, CT head scans have shown a low yield, ranging from 3.2% to 21.3% [8].The proportion of management changes after investigation was even lower [8].Requesting unnecessary CT head scans can lead to excessive costs, burdens on healthcare workers and unnecessary radiation exposure for patient.Additionally, CT head scans are one of the leading causes of radiation exposure, and their association with cancer development is well-documented [9].The benefits of CT must be carefully considered in relation to the risks posed to both the patient and the healthcare system.
According to surveys [10], decision support tools show promise in helping emergency physicians effectively utilize CT head scans.Many studies have demonstrated the potential factors that associated with positive finding on CT head scans such as Age, focal neurological deficit, history of malignancy, alcohol withdrawal symptoms (AWS) [5,[11][12][13].However, studies focusing on the utilization of these factors to develop clinical prediction tools for supporting the use of CT head scans in patients with seizures in resource-constrained settings are limited.Therefore, to assist physician decision-making, the aim of this study was to develop clinical predictor tools for guiding CT head scans in non-traumatic Thai patients presented with seizure.

Study design
This study was a diagnostic prediction research with a retrospective cross-sectional design at the Emergency Department of Lampang Hospital, a tertiary referral hospital in Thailand, during November 2019 to September 2021.The study received approval from the institutional review board and ethics committee of Lampang Hospital (No.133/65).Informed consent was waived due to the observational nature of the study.All tests and treatments were ordered as part of standard clinical or laboratory practice, and all were done in the past.Data were collected from December 2021 to March 2022.Patient identity was access only during the data collection and was not collected.This study was reported according to Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) reporting guideline [14].

Participant and data collection
Adult patients (� 18 years) who was diagnosed with seizures by their physician and underwent CT head scans for investigation were recruited into the study.Seizures refer to the uncontrolled, abnormal electrical activity of the brain that may cause changes in the level of consciousness, behavior, memory, or feelings [15].Demographic data, clinical characteristics, CT head scans official reports, and time parameters were retrospectively retrieved from the standardized electronic medical records.Patients without official reports from radiologists and those whose seizures were related to trauma were excluded.

Candidate predictor
A total of 9 potential predictors were preselected based on previously reported literature, expert consensus, simplicity, and their availability in general ED [5,[11][12][13]16].Demographic data: age, gender, clinical data: Glasgow coma scale (GCS) of 8�, 9-13, >13, new-onset focal neurological deficit (absence or presence) defined as: a set of symptoms or sign in which causation can be localized to an anatomic site in the central nervous system [17], previous CT head scans status (absence, normal or abnormal), history of malignancy (absence or presence), history of cerebrovascular accident (CVA) (absence or presence), Epilepsy (absence or presence) defined as: two unprovoked seizures occurring more than 24 h apart; a single unprovoked seizure if recurrence risk is high (ie, >60% over the next 10 years); or a diagnosis of an epilepsy syndrome [18], AWS (present or absence) defined as the present of autonomic hyperactivity (sweating, tachycardia); increased hand tremor; insomnia; nausea or vomiting; transient visual, tactile, auditory hallucinations or illusions; psychomotor agitation; anxiety; or tonicclonic seizures caused by abruptly discontinued of previous large consumed alcohol [19,20].All predictors can be accessed via hospital network database at the time of prediction.

Reference standard
Positive computed tomography (CT) head defined as the presence of any new lesion that related to the patient's presented seizure officially reported by radiologist.

Statistical analysis and sample size estimation
All statistical analyses were performed using Stata 17 (StataCorp, Lakeway, Texas, USA).Categorical variables were described using frequency and percentages.Numerical data were assessed for distribution using histograms, and described using means and standard deviations (SD) or medians and interquartile ranges (IQR) based on their distributions.Fisher's exact test was used for comparison of categorical variables, t-test and Mann Whitney U test were used for continuous variables comparison as appropriate.Statistical test results were considered significant if the p-values were less than 0.05.The sample size for developing a multivariable model with a binary outcome was calculated using the formula proposed by Richard D. Riley [21].We set the acceptable C-statistic and shrinkage factors at 0.8 and 0.9, respectively, considering 8 potential predictors.The incidence of positive CT head findings from pilot data at Lampang Hospital was 0.2.A minimum sample size of 352 patients and a minimum of 71 patients with positive CT head findings were required.

Model development
The final prediction model was developed using full multivariable logistic regression with backward stepwise elimination.Initially, all preselected predictors were included, irrespective of their statistical significance in the univariable analysis.Variables that did not show significance at a level of 0.05 were subsequently excluded, starting with the predictor that had the highest P value [22].All coefficients obtained from the final model were utilized in the generation of the prediction application.

Model performance and internal validation
The model's predictive performance was evaluated in terms of discriminative ability and calibration.Discriminative ability was assessed using the C-statistic and presented as the area under the receiver operating characteristic curve (AuROC).According to Hosmer and Lemeshow, an AuROC of 0.70-0.80,0.80-0.90, and above 0.90 were considered acceptable, excellent, and outstanding, respectively [23].For calibration, a calibration plot was used to visually inspect the agreement between observed risk and predictions.Internal validation was conducted through bootstrap resampling with 200 replicates to assess model optimism.
To evaluate the clinical utility of the prediction model, we performed decision curve analysis (DCA) [24,25].This straightforward method examines the net benefit (NB) derived from employing prediction models to guide clinical decisions.NB is computed by subtracting harms (false positives) from benefits (true positives).DCA were illustrated how the NB of the prediction models varied across different threshold probabilities for patients with positive findings on CT head scans.Considering previous research findings, we selected a range of 2% to 30% as reasonable threshold values.

Identifying cut point for clinical implication
The diagnostic indices, including sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), were calculated for each cut point based on estimated probability.Our aim is to establish cut-point values that can effectively assist clinicians in making decisions for performing CT head scans to achieve a definitive diagnosis, while also focusing on balance between the potential drawbacks of overutilization and the benefits of CT head scans.

Study patient
During November 2019 to September 2021, a total of 402 patients presented with seizure.Among these patients, 40 were excluded due to events suspected to be associated with traumatic brain injury (n = 39), and CT head scans were not performed.The remaining 362 patients were included into the analysis which comprising of 71 patients with positive CT head findings and 291 patients with normal results (Fig 1).The overall mean age of the study population was 55.2 ± 16.2 years, with three-quarters being male (73.8%).The prevalence of patients with focal neurological deficits was 17.1%.Approximately half of the total population previously had of CT head scans prior to the seizure event (47.5%).
Table 1 compares baseline characteristic between patient with positive CT head findings and those with normal findings.Statistically significant differences were observed in focal neurological deficits, previous CT head status, history of malignancy, history of CVA, epilepsy, and alcohol withdrawal symptoms.Patients with positive CT head findings exhibited a lower level of GCS compared to those with normal results (GCS median (IQR) 10 (7,15) vs. 15 (11,15)).Additional details of the predictors are presented in Table 1.

Model performance
After backward stepwise elimination, 6 final predictors were identified including: GCS, the presence of focal neurological deficit, history of malignancy, history of CVA, Epilepsy, and the presence of AWS.In terms of discriminative ability, the final model demonstrated excellent performance with AuROC of 0.82 (95% CI: 0.76-0.87)(Fig 2).Regarding model calibration, the calibration plot illustrated a good agreement between observed and predicted risks (S1 Fig in S1 File).The C-statistics from bootstrap resampling were 0.80 (95% 0.75-0.86),and the estimated optimism was 0.02 (95% CI: 0.01-0.03).The equation used for probability estimation in developing the application is provided below.X1 denoted a history of malignancy, X2 denoted the presence of focal neurological deficit, X3 denoted GCS (9-13), X4 denoted GCS �8, X5 demoted presence of AWS, X6 denoted the history of CVA, X7 denoted epilepsy.The clinical utility of prediction model was illustrated via DCA (Fig 3).The NB of newly derived clinical prediction model was higher than scan all strategies across the entire range of threshold probability for patient with positive finding on CT head scans.

Cut point threshold selection
Table 3 presents the detailed diagnostic indices at four potential threshold cut-points (15%, 20%, 25%, and 30%).At a probability threshold of 20%, the values of NPV and sensitivity were the highest, at 93.5% and 77.5%, respectively, along with a specificity of 79.4%.Conversely, at a 25% probability threshold, specificity was the highest, at with a sensitivity of 63.4% and NPV of 90.5%.Further details regarding the proportion of observed events and other diagnostic indices can be found in Table 3.

Discussion
In this study, we have developed a clinical prediction model to assist in guiding the utilizing of CT head scans in Thai patients presenting with seizures.The final model incorporates six predictors, including: GCS, the presence of neurological deficits, a history of malignancy, a history of CVA, epilepsy, and the presence of AWS.The model has demonstrated excellent discriminative ability, with AuROC of 0.82.Subsequently, an application was constructed to provide individualized recommendations for CT head scans based on each patient's risk profile.
The number of CT head scans ordered by ED physicians has increased substantially [26].Requesting unnecessary tests has been associated with overcrowding, patient severity, and low  physician experience [27,28].Extensive research has consistently demonstrated a low yield of CT as a seizure work up tools and suggests a more conservative approach to the use of CT head scans based on clinical factors [5,8,29].On the other hand, seizures represent clinically significant situations that require experienced assessment for accurate and timely patient guidance for further management.The triage strategies have been proposed to effectively reduced this type of problem [30].Therefore, incorporating clinical prediction tools as part of the diagnostic triage process may enhance the management process, reduce unnecessary CT head scans, consequently leading to cost savings and reduced patient stay time in the ED [5,8].
According to the result, the prevalence of observed positive finding on CT head scans was consistent with previous to the previous evidence [11].Factors such GCS of 9-13 and �8, the presence of neurological deficits, and a history of malignancy have consistently demonstrated a strong association with positive lesions on CT head scans, as these factors exhibiting signs or increasing the risk of new brain lesions [5,8].Conversely, three predictors, including: a history of CVA, epilepsy, and the presence of alcohol withdrawal symptoms, decreased the risk of having positive findings.Patients with a history of CVA without new neurological deficits were likely to have seizures caused by old lesions [16].Similarly, in the absence of either focal deficits on neurological examination or signs of acute head trauma, patients with alcohol withdrawal seizures or epileptic patient were likely to have normal findings on CT head scans [5,31].It is worth noting that CT head scans did not alter the acute management in these patients [8].
Our study represented the first attempt to construct a clinical decision support system for ordering CT head scans in non-traumatic patients presenting with seizures.The primary objective of this study was to balance between ensuring the standard care and the overuse of unnecessary CT head scans.We proposed a threshold set at a 20% probability.At this threshold, the model achieved a sensitivity of 77.5%, specificity of 79.5%, and a negative predictive value of 93.5%, indicating a well-balanced performance with the lowest false negative rate.With its excellent discriminative ability (AuROC of 0.82), the application offers a reliable tool for effectively reduce unnecessary use and instill confidence in supporting physicians in their decision-making process.

Limitation
There are several important limitations that should be taken into account.Firstly, our model was developed using retrospective data.Missing data, or misclassification error made by the nature of retrospective data may introduce certain biases.Secondly, the assessment of some predictors such as focal neurological deficits and AWS was primarily conducted by general practitioners.The inherently subjective nature of these assessments raises concerns about potential inaccuracies.However, they represent real-world accuracies and baseline physician performance.Since our aim is to implement the prediction model primarily for use by general practitioners, particularly to assist less experienced physicians in decision-making.Additionally, the potential subjectivity in diagnosing AWS is relatively low, as most cases can be easily identified due to a clear history of heavy alcohol drinking followed by immediate discontinuation.Therefore, the future predictive performance is likely to be less impacted.Lastly, the models were constructed based on data from a single center.To guarantee the reliability of the model performance, it is advisable to conduct multicenter externally validated studies before integrating the diagnostic score into a wider clinical setting.
To access this application, we have provided the information as follows link: https:// tharathipdevelop.com/pos-ct/form