Is the Sport Concussion Assessment Tool 5th Edition a useful concussion assessment tool in head-injured patients presenting to the emergency department?

Objective : Concussion is a common ED complaint, but diagnosis is chal-lenging as there are no validated objective measures. Use of concussion tools derived from sports medicine is common, but these tools are not well validated in ED settings. The aim of this study was to assess the ability of the Sport Concussion Assessment Tool 5th Edition (SCAT5) to identify concussion in ED patients presenting following head injury. Methods : We conducted a prospective observational study of head-injured adult patients presenting to ED between March and July 2021. ED diagnosis of concussion was used as the diagnostic standard, and we assessed the diagnostic performance of the SCAT5 test and its three subsections (Standardised Assessment of Concussion (SAC), Post-Concussion Symptom Scale (PCSS) and Modi ﬁ ed Balance Error Scoring System (mBESS)) against this. Results : Thirty-two head-injured participants were enrolled, 19 of whom had a discharge diagnosis of concussion, alongside 17 controls. Median


Introduction
Concussion is a common ED complaint, 1 but its diagnosis has several inherent challenges. Firstly, the pathophysiology of concussion has not been conclusively determined. 2,3 Secondly, symptoms are non-specific and numerous including features of somatic, cognitive, sleep-related and emotional disturbance. 4 Thirdly, routine radiological imaging is • The SCAT5 test had a low specificity of 20.0% for identifying concussion, had a median test time of 21 min, and 30.6% of participants were interrupted at least once during assessment suggesting that it is not an ideal assessment to use in ED. • The SAC section had a sensitivity of 48.1% and a specificity of 60.0% and was able to be completed by 95.9% of all participants. The mBESS had a sensitivitiy of 83.3% and a specificity of 58.8% and was able to be completed by 71.4% of all participants. • The PCSS section had a sensitivity of 89.7% and specificity of 85.0% and was able to be completed by all participants. It may be useful as a standalone tool to simplify ED concussion identification. normal in concussion by definition, making the diagnosis of concussion symptom based, with no pathognomonic characteristics, biomarkers or pathophysiologic profiles to aid diagnosis. 5 Currently, there is no ED-specific assessment tool for concussion, and ED concussion management is often adapted from those used in sports. 6,7 ED-specific guidelines focus on ruling out more serious injuries with CT head scans and often concussion assessment is brief and nondescript. 8 The Concussion in Sport Group 9 consensus statement recommends utilising the most recent version of the Sport Concussion Assessment Tool 5th Edition (SCAT5) as the standardised assessment tool for concussion. 4 However, there is a lack of evidence validating the use of the SCAT5 in the general ED population. 6,7,10 Compared to the typically young, healthy athletes included in sporting literature, the general ED population is considerably more diverse and concussion is often poorly managed. 6 Mechanisms resulting in mild traumatic brain injury (TBI) and/or concussion that are more likely to be seen in ED are injuries due to falls, motor vehicle crashes, and then in sport, with assaults and other mechanisms also contributing. 11

Sport Concussion Assessment Tool 5th Edition
The SCAT5 is designed for use by physicians and licenced healthcare professionals. It has three main sections, with subsections that produce a score. There is a Standardised Assessment of Concussion (SAC); a Post-Concussion Symptom Scale (PCSS); and a Modified Balance Error Scoring System (mBESS). The full SCAT5 test is included in Appendix S1. The SCAT5 test is designed to compare a baseline score to a post-head injury score. Therefore, the SCAT5 test does not have a pre-specified 'pass' or 'fail' score. To make this test accessible for those without a baseline assessment, some population normative baseline data has been reported in the literature and used to generate cut-off scores. 12,13 Following completion of  the test, clinicians are instructed to  use the test results and clinical judgement to determine if concussion is  present. There has been a validation study of the SCAT5 in professional male ice hockey 13 and one other pilot study looking at 30 day reattendance using the SCAT5 within the ED to the authors' knowledge. 6 There are few studies looking at the validity and feasibility of previous SCAT editions within the ED, 7,14 but more in sport. 15 There is limited evidence evaluating the feasibility and effectiveness of the SCAT5 test's performance in an ED setting. Therefore, the primary aim of the present study was to assess the discriminatory ability of the SCAT5 test to detect concussion in patients presenting to ED following a head injury. A secondary aim was to determine the feasibility of using the SCAT5 in an ED setting.

Study design
The present study was a prospective observational and feasibility study of patients presenting with mild head injuries to Wellington Regional Hospital ED between March 2021 and July 2021. Wellington ED is a publicly funded major referral centre. The National Institute for Health and Care Excellence head injury guidelines are used as a clinical decision aid in TBI and the SCAT5 is the recommended assessment tool for use in suspected concussion.
Recruitment was completed by authors AS, BK-G and JC who had instruction in completing the SCAT5 test by the author AR. Recruitment was initially completed in pairs to ensure consistency in testing and then individually. The investigators were independent of the clinical care team and all participants were treated in ED according to current clinical standard by their treating clinician. Once the SCAT5 was completed, the test scores were made available to the treating clinician via clinical notes if they so wished to use them for their care as recommended.

Patient selection
To explore the discriminatory ability of the SCAT5 test to identify concussion, three comparator groups were recruited: non-head-injured controls (C); head-injured but not meeting National Institute for Health and Care Excellence CT-head criteria (noCT) and head-injured with negative CT head scan performed (CTneg).
Convenience sampling methods were used to identify eligible patients for all three study groups, mainly during weekday office hours and occasionally weekends because of researcher availability. Patients in ED or in the waiting room were screened using clinical/triage notes to see if they met the study criteria. Health practitioners were consulted about the eligibility of their patient for the study if more information was required to determine eligibility.
Controls were recruited from the Department of Surgery and Anaesthesia, University of Otago and from the ED. Patients with a high triage number (3-5) (lower priority) and who potentially met the study criteria were approached and informed of the study. Inclusion criteria for controls were: 18 years or older; able to give informed consent; no pre-existing TBI or neurological condition diagnosed; no acute illness that prevented participation in the study; and no recent head injury within 7 days. All patients presenting to ED with head injury during researcher hours were screened for eligibility for the two head-injured groups. Those potentially meeting the study criteria were approached and informed of the study. Inclusion criteria were: 18 years or older; able to give informed consent; mild TBI (GCS 13-15); negative CT head (if performed); and confirmed head trauma within the previous 24 h. Exclusion criteria were: pre-existing TBI or concussion; positive CT head scan; pre-existing neurological condition such as dementia, acute psychosis, multiple sclerosis, motor neuron disease; unable to give informed consent.
Voluntary written informed consent was obtained from those agreeing to participate. All participants then completed the SCAT5 test conducted by a research investigator. Ethical approval was obtained from the Health Disability and Ethics committee (HDEC Reference 20/CEN/5/AM04. CCDHB Research Advisory Group for M aori [RAG-M] Endorsement: #722). Local authority approval was obtained for Wellington Regional Hospital Emergency Department and research was conducted in line with Capital and Coast District Health Board research policies.

Data collection
Study data were collected and managed using REDCap (Research Electronic Data Capture) electronic data capture tools hosted at University of Otago (version 10.1.2). 16,17 REDCap is a secure, web-based software platform designed to support data capture for research studies, providing (i) an intuitive interface for validated data capture; (ii) audit trails for tracking data manipulation and export procedures; (iii) automated export procedures for seemless data downloads to common statistical packages; and (iv) procedures for data integration and interoperability with external sources. Table 1 summarises scoring of the SCAT5 and the referenced cut offs used in the present study. Data were collected for participant demographics (age, sex, ethnicity), prior TBI history, clinical features including mechanism, alcohol use prior to presentation and injury time. Investigators also recorded the time taken to complete SCAT5, SCAT5 results, number of and reason for any interruptions during testing and other patient or environmental factors that interfered with test performance. It was recorded if the treating clinician included concussion as a discharge diagnosis; this was collected retrospectively from clinical records. All data were deidentified at the point of entry.

Statistical methods
Demographics, clinical features, feasibility of the SCAT5 and the discriminatory ability of the SCAT5 components were described statistically using SPSS software, version 27.0 (IBM Corp, Armonk, NY, USA) and Excel. Data were not normally distributed. Therefore, medians and interquartile ranges (IQRs) were used as appropriate to calculate the central tendency and spread of the data in the three comparator groups. SCAT5 scores were compared across the three groups using Kruskall-Wallis testing. Post-hoc testing in the form of pairwise comparisons was conducted on significant results of difference with significance values adjusted by the Bonferroni correction for multiple tests. The null hypothesis was that there would be no difference between SCAT5 scores between comparator groups.
To review the diagnostic performance of the SCAT5 test to identify patients with concussion, area under the receiver operating characteristic curve analysis was performed on subsection scores and is reported with 95% confidence intervals (CIs). A discharge diagnosis of concussion by an ED clinician was used as the diagnostic standard. The best-performing SCAT5 testing cut-offs were compared to existing literature (cut-offs values described in Table 1) and were within the 95% CIs identified in the present study. Therefore, participant results were coded into pass or fail cohorts. Any subsection score below a cut-off value was a 'fail' for that section. If any section was 'failed' this was then coded as a 'failed' SCAT5 score. This was used to calculate sensitivity, specificity, positive Tandem leg stance 0-10 ≤4 12 Single leg stance 0-20 ≤6 12 †The 22 symptoms cover six symptom categories: physical, vestibular-ocular, cognitive, sleep and emotional. ‡Each symptom is ranked on a Likert scale from 0 to 6, with 0 being no experience of the symptom to 6 being a severe experience of the symptom. The sum of these rankings is the symptom severity. §Participants assume the stances with their eyes closed for 20 s. The examiner records how many errors are made while holding these stances. Errors include breaking the stance, opening eyes or requiring significant correction. and negative predictive values for the sections using standard 2 Â 2 box plots. Missing data were not included in statistical analysis. Figure 1 illustrates the case selection process resulting in 49 participants (C = 17, noCT = 19 and CTneg = 13). Investigators screened 117 head injury cases and reported 68 ineligible cases.

Results
Demographics and clinical features for each comparator group are shown in Table 2. Of note, 18 of 49 (37%) participants reported a previous diagnosis of concussion. The most common mechanism of injury was fall from standing height.
In terms of diagnostic test performance, AUC values for the subsections of the SAC section were between 0.41 and 0.51. Regarding PCSS, total number of symptoms score had an AUC of 0.91 and the severity score had an AUC score of 0.92. Regarding mBESS, subsections AUC values were between 0.53 and 0.66 (Fig. 2).

Feasibility of the SCAT5
The median time taken to complete SCAT5 assessments was 21 min (IQR 16-27 min). Excluding interruptions, the median time was 19 min (IQR 16-24 min). Of the 49 participants, 15 (30.6%) were interrupted at least

Discussion
In the SAC, only the total SAC score and immediate memory score was lower in the CTneg group compared to the controls, with no difference between the noCT and the other two groups. Both PCSS scores were lower in controls when compared to both head-injured groups, however, there was no difference between the two head-injured groups. No score in the mBESS was significantly different between any of the groups. Both the mBESS and SAC lacked sufficient precision to identify the cohort with concussion, particularly the SAC score. Literature reviewing World Rugby head injury processes also demonstrated that immediate memory and tandem gait provided little diagnostic value. 18 The mBESS test performed better than the SAC score. However, many ED patients in the present study were unable to complete the full assessment. This is consistent with other international evidence noting that 33% of head-injured patients could not complete the test in ED. 7 Another ED-based study excluded the mBESS assessment entirely because participants were unwilling to perform the test. 14 Even literature examining the role of mBESS in sporting situations report limited utility as a diagnostic test 18 and high variability with baseline and repeated testing. 19 There is some promise that sensor-based balance measures conducted in patients during double leg stance (a component achievable by most patients in the present study) outperforms mBESS testing and can identify patients with concussion. 20 This warrants further assessment in an ED environment.
The PCSS score alone shows promise as a discriminatory tool for concussion assessment in ED. In the present study it identified almost all patients with concussion. The results from our study are similar to other research reviewing symptom scoring in headinjured ED patients. 7,14 However, there are some limitations to PCSS scoring particularly given it is a subjective self-reported questionnaire. 21 Its use as an isolated tool is advised against in sporting environments. 22 Athletes have a tendency to underreport concussion symptoms as this affects their return to play. 22 It is unlikely ED presenters would have the same motivations to underreport symptoms, but there are likely to be some attenders who want to return to sport or have pressure to continue working; this area requires further investigation.
The present study demonstrated that it took over 20-min to complete the SCAT5 test. Long test duration has also been reported in general practice, and was noted to be a deterrent for clinical use. 10 A long screening test is ill-suited for a timepressured clinician in ED who has to perform this test in addition to routine clinical assessment, examination and CT-head decision making. 11 Feasibility issues also influence the validity and accuracy of result. Interruptions were a major feature during SCAT5 assessment in the present study. Specific examples included concurrent unintentional participation by neighbouring patients during SAC testing and noises such as voices and beeping impeding the participants ability to conduct the test. ED interruptions are endemic and part of the fast-paced changeable environment. ED clinicians are frequently interrupted, even when performing clinical review of patients and it is unlikely these real-world ED interactions can be mitigated reliably. 23 Additionally, the SCAT5's original purpose was for use in sport. Sport and exercise physicians have specific expertise in this field compared to ED clinicians. 24 They perform the assessment faster, more efficiently and are more aware of its limitations. Also, elite athletes are usually known to the treating physician and are more likely to have had a baseline assessment. 18 This does not necessarily translate into an ED environment. In ED, clinicians' knowledge is variable with respect to current concussion guidelines 1,25 and are unlikely to know patients and their baseline functioning. Therefore, concussion diagnosis is more clinician dependent with a greater likelihood of error.
Furthermore, the roles that ED and sports clinicians have in concussion differ. Sport and exercise physicians likely require more sensitive screening tests as a key responsibility is to prevent further injury by returning to sport too soon. They will also monitor the athlete during recovery. Comparatively, a recent update from the American College of Emergency Physicians Sport Related Head Injury Prevention Task Force recommends that ED care should focus on mitigating the adverse effects of concussion by ensuring timely diagnosis and management. 5 This includes providing good consistent discharge advice and prompt referral to specialist concussion services for those at risk of prolonged symptoms.
The first step towards achieving this is simplifying and standardising concussion identification in ED in a way that is both effective and efficient. A standardised approach will allow for a better understanding of a patient's concussion progression as the assessment can be repeated during follow up. It will also provide better epidemiological data which would enable further research in other areas of concussion that are highly coveted; such as symptom treatment and the role of objective biomarkers. 26 We propose that symptom scoring alone in ED may be a good starting point to identify concussion.

Limitations
A major limitation of the present study is that we used clinical diagnosis of concussion as the standard. We acknowledge that this is highly variable and likely to be biased towards the PCSS given that clinicians rate concussion symptoms highly when deciding which patients have concussion. However, all concussion definitions include symptoms as one of the defining features. Therefore, it is the standard most applicable to this specific environment and it is likely our results still have value given that we included a range of participants. Participants were also recruited at varying times following their injury. Concussion symptoms can develop over time particularly within the first 72 h therefore if they were assessed prior to symptoms developing, they may not have correctly received a concussion diagnosis.
The SCAT5 specifically states that scoring on the test should not be used as a stand-alone method to diagnose concussion but instead should be used to aid assessment in conjunction with clinical judgement. The Concussion in Sport Group state that the SCAT is clinically useful in screening evaluations and making the diagnosis of sports-related concussion 4 and the present study evaluated whether that clinical utility also existed in the ED environment by assessing how the SCAT5 scores helped to differentiate a concussion patient. Most biomarkers and assessment tools should be used in conjunction with clinical judgement and clinically useful SCAT5 scores would be no exception.
Another limitation is the cut offs used from existing literature. Little baseline normative data exists for general populations, and normative data that is available can be quite variable. 26 Typically this data comes from young, healthy sporting populations which is not representative of the cohort of patients seen in ED. 12,27 This could significantly affect the results in the present study but also reveals another reason as to why the SCAT5 is less suited to an ED environment given individual baseline data and subsequent followup assessment are not available to ED attenders.
The present study also has a small sample size. Convenience sampling could have led to selection bias. Multiple research investigators performed the SCAT5 during the present study, which introduces interrater variability with regards to interpretation and timing. However, in the real ED environment multiple clinicians would need to use and interpret the test. Finally, we excluded those who could not consent due to language barriers and cognitive impairment. We acknowledge in practice, these groups would have difficulty completing assessments like the SCAT5. 27

Conclusion
The SCAT5 test had low specificity, was long, was frequently interrupted and has complex scoring methods. We suggest it is not an ideal assessment to use in ED. The PCSS score performed well and was easy to complete. It may be useful as a standalone tool to simplify ED concussion identification. Although this may not be completely perfect in isolation, it will enable better data monitoring, improve clinical consistency, and support further research into improved concussion care in ED.