Skip to main content

Advertisement

Log in

Neurocritical Care Performance Measures Derived from Electronic Health Record Data are Feasible and Reveal Site-Specific Variation: A CHoRUS Pilot Project

  • Big Data in Neurocritical Care
  • Published:
Neurocritical Care Aims and scope Submit manuscript

Abstract

Background

We evaluated the feasibility and discriminability of recently proposed Clinical Performance Measures for Neurocritical Care (Neurocritical Care Society) and Quality Indicators for Traumatic Brain Injury (Collaborative European NeuroTrauma Effectiveness Research in TBI; CENTER-TBI) extracted from electronic health record (EHR) flowsheet data.

Methods

At three centers within the Collaborative Hospital Repository Uniting Standards (CHoRUS) for Equitable AI consortium, we examined consecutive neurocritical care admissions exceeding 24 h (03/2015–02/2020) and evaluated the feasibility, discriminability, and site-specific variation of five clinical performance measures and quality indicators: (1) intracranial pressure (ICP) monitoring (ICPM) within 24 h when indicated, (2) ICPM latency when initiated within 24 h, (3) frequency of nurse-documented neurologic assessments, (4) intermittent pneumatic compression device (IPCd) initiation within 24 h, and (5) latency to IPCd application. We additionally explored associations between delayed IPCd initiation and codes for venous thromboembolism documented using the 10th revision of the International Statistical Classification of Diseases and Related Health Problems (ICD-10) system. Median (interquartile range) statistics are reported. Kruskal–Wallis tests were measured for differences across centers, and Dunn statistics were reported for between-center differences.

Results

A total of 14,985 admissions met inclusion criteria. ICPM was documented in 1514 (10.1%), neurologic assessments in 14,635 (91.1%), and IPCd application in 14,175 (88.5%). ICPM began within 24 h for 1267 (83.7%), with site-specific latency differences among sites 1–3, respectively, (0.54 h [2.82], 0.58 h [1.68], and 2.36 h [4.60]; p < 0.001). The frequency of nurse-documented neurologic assessments also varied by site (17.4 per day [5.97], 8.4 per day [3.12], and 15.3 per day [8.34]; p < 0.001) and diurnally (6.90 per day during daytime hours vs. 5.67 per day at night, p < 0.001). IPCds were applied within 24 h for 12,863 (90.7%) patients meeting clinical eligibility (excluding those with EHR documentation of limiting injuries, actively documented as ambulating, or refusing prophylaxis). In-hospital venous thromboembolism varied by site (1.23%, 1.55%, and 5.18%; p < 0.001) and was associated with increased IPCd latency (overall, 1.02 h [10.4] vs. 0.97 h [5.98], p = 0.479; site 1, 2.25 h [10.27] vs. 1.82 h [7.39], p = 0.713; site 2, 1.38 h [5.90] vs. 0.80 h [0.53], p = 0.216; site 3, 0.40 h [16.3] vs. 0.35 h [11.5], p = 0.036).

Conclusions

Electronic health record–derived reporting of neurocritical care performance measures is feasible and demonstrates site-specific variation. Future efforts should examine whether performance or documentation drives these measures, what outcomes are associated with performance, and whether EHR-derived measures of performance measures and quality indicators are modifiable.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Moheet AM, Livesay SL. Quality improvement in neurocritical care: current state and looking to the future. Curr Opin Crit Care. 2020;26:97–102.

    PubMed  Google Scholar 

  2. Huijben JA, Wiegers EJ, de Keizer NF, et al. Development of a quality indicator set to measure and improve quality of ICU care for patients with traumatic brain injury. Crit Care. 2019;23:1–13.

    Article  Google Scholar 

  3. Huijben JA, Wiegers EJ, Ercole A, et al. Quality indicators for patients with traumatic brain injury in European intensive care units: a CENTER-TBI study. Crit Care. 2020;24:1–12.

    Article  Google Scholar 

  4. Rhodes A, Moreno RP, Azoulay E, et al. Prospectively defined indicators to improve the safety and quality of care for critically ill patients: a report from the Task Force on Safety and Quality of the European Society of Intensive Care Medicine (ESICM). Intensive Care Med. 2012;38:598–605.

    Article  CAS  Google Scholar 

  5. Hong Y, LaBresh KA. Overview of the American Heart Association “Get with the Guidelines” programs: coronary heart disease, stroke, and heart failure. Crit Pathw Cardiol. 2006;5:179–86.

    Article  Google Scholar 

  6. Howard G, Schwamm LH, Donnelly JP, et al. Participation in get with the guidelines–stroke and its association with quality of care for stroke. JAMA Neurol. 2018;75:1331–7.

    Article  Google Scholar 

  7. Schwamm L, Fonarow G, Reeves M, Pan W, Frankel M, Smith E. Get with the guidelines-stroke is associated with sustained improvement in care for patients hospitalized with acute stroke or transient ischemic attack. Circulation. 2009;119:107–15.

    Article  Google Scholar 

  8. Borg J, Carroll L. Critical evaluation of the existing guidelines on mild traumatic brain injury. J Rehabil Med. 2004;43:106–12.

    Google Scholar 

  9. de Winkel J, van der Jagt M, Lingsma HF, et al. International practice variability in treatment of aneurysmal subarachnoid hemorrhage. J Clin Med. 2021;10:762.

    Article  Google Scholar 

  10. Livesay S, Fried H, Gagnon D, et al. Clinical performance measures for neurocritical care: a statement for healthcare professionals from the Neurocritical Care Society. Neurocrit Care. 2020;32:5–79.

    Article  Google Scholar 

  11. Sultan AA, West J, Stephansson O, et al. Defining venous thromboembolism and measuring its incidence using Swedish health registries: a nationwide pregnancy cohort study. BMJ Open. 2015;5:e008864.

    Article  Google Scholar 

  12. Messé SR, Khatri P, Reeves MJ, et al. Why are acute ischemic stroke patients not receiving IV tPA? Results from a national registry. Neurology. 2016;87:1565–74.

    Article  Google Scholar 

  13. Schwamm LH, Ali SF, Reeves MJ, et al. Temporal trends in patient characteristics and treatment with intravenous thrombolysis among acute ischemic stroke patients at get with the guidelines–stroke hospitals. Circ Cardiovasc Qual Outcomes. 2013;6:543–9.

    Article  Google Scholar 

  14. Bravata DM, Myers LJ, Cheng E, et al. Development and validation of electronic quality measures to assess care for patients with transient ischemic attack and minor ischemic stroke. Circ Cardiovasc Qual Outcomes. 2017;10:e003157.

    Article  Google Scholar 

  15. Chan KS, Fowles JB, Weiner JP. Electronic health records and the reliability and validity of quality measures: a review of the literature. Med Care Res Rev. 2010;67:503–27.

    Article  Google Scholar 

  16. East AT, Wakefield TW. What is the optimal duration of treatment for DVT? An update on evidence-based medicine of treatment for DVT. Semin Vasc Surg. 2010;23:182–91.

    Article  Google Scholar 

  17. Haut ER, Noll K, Efron DT, et al. Can increased incidence of deep vein thrombosis (DVT) be used as a marker of quality of care in the absence of standardized screening? The potential effect of surveillance bias on reported DVT rates after trauma. J Trauma Acute Care Surg. 2007;63:1132–7.

    Article  Google Scholar 

  18. Futoma J, Simons M, Doshi-Velez F, Kamaleswaran R. Generalization in clinical prediction models: the blessing and curse of measurement indicator variables. Crit Care Explor. 2021;3:e0453.

    Article  Google Scholar 

  19. Holder AL, Shashikumar SP, Wardi G, Buchman TG, Nemati S. A locally optimized data-driven tool to predict sepsis-associated vasopressor use in the ICU. Crit Care Med. 2021;49:e1196–205.

    PubMed  Google Scholar 

  20. Flinn WR, Sandager GP, Silva MB, Benjamin ME, Cerullo LJ, Taylor M. Prospective surveillance for perioperative venous thrombosis: experience in 2643 patients. Arch Surg. 1996;131:472–80.

    Article  CAS  Google Scholar 

Download references

Funding

This study was funded by the National Institutes of Health (5K23NS105950).

Author information

Authors and Affiliations

Authors

Contributions

SEA, SYL, EA, BPF, and ESR contributed to study conception and design. SEA, GS, EA, BPF, and ESR acquired, analyzed and interpreted data. SEA and ESR drafted the manuscript. SEA, SYL, GN, JNG, IAL, SMD, EA, PV, JRM, XH, GC, SP, RK, BPF, and ESR revised the manuscript critically for important intellectual content and gave final approval for this version to be published.

Corresponding author

Correspondence to Eric S. Rosenthal.

Ethics declarations

Conflict of interest

Dr. Amorim has received research Grants from National Institutes of Health (1K23NS119794), Hellman Fellows Fund, Regents of the University of California (Resource Allocation Program), Citizens United for Research in Epilepsy (CURE) Epilepsy Foundation (Taking Flight Award), Weil-Society of Critical Care Medicine Research Grant, Zoll Foundation Grant, and American Heart Association (20CDA35310297). Dr. Vespa has received consulting fees from Ceribell, Inc and UCB, Inc and has received research Grants from National Institutes of Health (3U54NS100064, 1R01NS113541). Dr. Moorman owns stock in Medical Predictive Science Corporation, has received consulting fees from Nihon Kohden Digital Health Solutions, and has received a research Grant from National Institutes of Health (5R01HD072071). Dr. Hu has received research Grants from National Institutes of Health (5R01NS106905, 1R01NS113541). Dr. Clermont owns stock in NOMA AI, Inc, receives royalties from UpToDate, Inc, and has received research Grants from Department of Defense (DoD) (W81XWH-17-R-BAA180061) and National Institutes of Health (R01NR013912, R21NS115174, R01DK131586). Dr. Park has received a research Grant from National Institutes of Health (1R21NS113055). Dr. Kamaleswaran has received a research Grant from National Institutes of Health (1R01GM139967). Dr. Foreman has received consulting fees from Union Chimique Belge (UCB), Inc and has received research Grants from DoD (W81XWH-18-DMRDP-PTCRA) and National Institutes of Health (5K23NS101123). Dr. Rosenthal has received consulting fees from UCB, Inc and Ceribell, Inc and has received research Grants from DoD (W81XWH-18-DMRDP-PTCRA) and National Institutes of Health (5R01NS117904, 5K23NS105950, 1R01NS113541, 3U54NS100064). The remaining authors declare that they do not have any potential conflicts of interest.

Ethical Approval/Informed Consent

The reported research adheres to ethical guidelines and was conducted under approved investigational review board protocols. This is a retrospective study with deidentified data and was exempt from informed consent.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix A: Selected Flowsheet Measure Names

Glasgow Coma Scale

  • R ED CLINICAL CALCULATOR—BEST MOTOR RESPONSE (PGCS)

  • G ED CLINICAL CALCULATOR—GLASGOW COMA SCALE

  • R ED CLINICAL CALCULATOR—GLASGOW COMA SCALE SCORE

  • R CPN GLASGOW COMA SCALE BEST MOTOR RESPONSE

  • G PHS THERAPY GLASGOW COMA SCALE

  • R PHS IP THERAPY GLASGOW COMA MOTOR RESPONSE

  • R PHS IP THERAPY GLASGOW COMA SCORE

  • R GLASGOW COMA SCALE

  • R CPN GLASGOW COMA SCALE SCORE

  • R CPN GLASGOW COMA SCALE BEST MOTOR RESPONSE

  • R CPN GLASGOW COMA SCALE SCORE

  • R GCS—MOTOR INITIAL HOSPITAL

  • R GCS—MOTOR PREHOSPITAL

  • R GCS—TOTAL PREHOSPITAL

  • R GCS—SCORE INITIAL HOSPITAL

  • R ED PRE_ARRIVAL GCS SCORE

Intracranial Pressure

  • R ICP MEAN 1

  • R AN ICP MEAN

  • R ICP MEAN 2

  • R ICP MEAN

  • R ICP DRAIN STATUS

  • R ICP MEAN #2

  • R ICP MONITORING

Mechanical Prophylaxis

  • R PHS IP PVS INTERVENTIONS

  • G PVS INTERVENTIONS

  • R PHS IP GRADUATED COMPRESSION STOCKINGS

  • G COMPRESSION STOCKINGS

  • R ACE WRAP

  • R PHS IP TUBULAR UNI GRIP

  • R PNEUMATIC COMPRESSORS INITIATED?

  • R PHS IP OT VASOPNEUMATIC DEVICES DETAILS

  • R PHS IP PT VASOPNEUMATIC DEVICES DETAILS

  • R PHS IP UNNA BOOTS

  • R PHS IP SCD

  • R INTERVENTION SCD

  • R PHS IP SCD APPLIED

  • R IP COMPRESSION BOOTS

  • R PVS (WDL)

  • R RLE DVT PROPHYLAXIS

Appendix B: ICD-10 Dictionary

INCLUDE in category if any instance during encounter unless otherwise stated

  1. 1.

    Cerebral ischemia

    • Cerebral ischemia I67.82

    • TIA G45. 9, Z86.73

    • Cerebral infarction I63.0, I63.1, I63.2, I63.3, I63.4, I63.5, I63.8, I63.9, I69.3, G46.3

    • Carotid occlusion I65.2

    • Stroke/other vascular occlusions (Retinal) H34.8392, I65.0, I65.1, I65.8, I65.9, I66

    • MELAS E88.41

    • Carotid dissection I77.71

    • Cerebral venous sinus thrombosis I82, I63.6, I67.6, Z86.718

    • Intracranial and intraspinal phlebitis and thrombophlebitis G08

    • Other transient cerebral ischemic attacks and related syndromes G45.8

    • Moyamoya disease I67.5

  1. 2.

    Nontraumatic subarachnoid hemorrhage

    • Nontraumatic Subarachnoid hemorrhage (aSAH) I60, I69.0

    • Cerebral aneurysm, nonruptured I67.1

    • Arteriovenous malformations Q28.1, Q28.2, Q28.3

    • Carotid aneurysms I72.0

    • Arteriovenous fistula, acquired I77.0

    • Stricture of artery I77.1

    • Compression of vein I87.1

    • Arteriovenous malformation, other site Q27.39

    • Arteriovenous malformation, other site Q27.39

    • Other specified congenital malformations of circulatory system Q28.8

  1. 3.

    Spontaneous Intracerebral hemorrhage

    • Nontraumatic intracerebral hemorrhage I61, I69.10, I69.159

    • Nontraumatic intracranial hemorrhage unspecified I62.9

  1. 4.

    Traumatic Brain Injury and Nontraumatic Subdural Hematoma

    • Epidural hemorrhage/hematoma S06.4X

    • Subdural hemorrhage I62.00, S06.5X0A

    • Traumatic subdural hemorrhage S06.5X

    • Traumatic brain injury S06.2, S06.2X, S06.3, S06.2X0A, S06.2X1A, S06.2X9A, S06.300A

    • Traumatic subarachnoid hemorrhage (tSAH) S06.6, S06.6X, S06.6X0A

    • Nontraumatic subdural hematoma I62.01

    • Maxillofacial injury S02.2XXA, S02.401A, S02.92XA, S05.10XA, S05.90XA, S06.9X0A, S06.9X0A, S09.90XA

    • Only if no other diagnosis 1–13:

    • Fall W19.

  1. 5.

    Structural or Dynamic CSF Disorders

    • Hydrocephalus G91.0, G91.1, G91.2, G91.3, G91.4, G91.5, G91.6, G91.7, G91.8, G91.9, Q07

    • Shunt malfunction T85.02, T85.09

    • Ventriculitis T85.02

    • Shunt infection T85.730

    • Congenital malformations, others Q01, Q04

    • Cerebrospinal fluid leak G96.0

    • Other acquired deformity of head M95.2

    • Arnold-Chiari syndrome without spina bifida or hydrocephalus Q07.00

  1. 6.

    Malignant Brain Tumors

    • Glioblastoma Multiforme C71.9

    • Primary Brain Tumor/Malignant neoplasm of brain C71

    • CNS lymphoma C85.89

    • Secondary malignant neoplasm of brain C79.31

    • Metastatic brain tumors C79.31

    • Hemangioma of other sites D18.09

  1. 7.

    Other head and neck tumors including benign brain/spine tumors

    • Benign neoplasm of brain, spinal cord D33.0, D33.1, D33.2, D33.4, D33.7, D33.9

    • Benign neoplasm of cranial nerves D33.3

    • Benign neoplasm of meninges, unspecified; Meningioma D32.0

    • Neoplasm of unspecified behavior of brain D49.6

    • Vertebral Tumors-CNS C41.2

    • Pituitary tumors and disorders C75.1, D35.2, E22.0, D44.3, E24.0, E23.6

    • Trigeminal Neuralgia G50.0

    • Cranial Nerve Disorders, including ophthalmoplegias G52.7, G52.9, H53.2, H54.7

    • Neoplasm of uncertain behavior of brain and central nervous system D43.0, D43.1, D43.2

    • Malignant neoplasm of frontal sinus C31.2

    • Malignant neoplasm of accessory sinus, unspecified C31.9

    • Unspecified malignant neoplasm of skin of scalp and neck C44.40

    • Squamous cell carcinoma of skin of scalp and neck C44.42

    • Malignant neoplasm of head, face and neck C76.0

    • Benign neoplasm of bones of skull and face D16.4

    • Benign neoplasm of meninges, unspecified D32.9

    • Benign neoplasm, unspecified site D36.9

    • Neoplasm of uncertain behavior of connective and other soft tissue D48.1

    • Neoplasm of unspecified behavior of respiratory system D49.1

    • Neoplasm of unspecified behavior of bone, soft tissue, and skin D49.2

    • Atypical facial pain G50.1

    • Clonic hemifacial spasm G51.3

    • Clonic hemifacial spasm, left G51.32

    • Clonic hemifacial spasm, unspecified G51.39

    • Other disorders of facial nerve G51.8

    • Disorders of glossopharyngeal nerve G52.1

    • Cerebral cysts G93.0

    • Epidermal cyst L72.0

    • Other disorders of meninges, not elsewhere classified G96.19

    • Right temporomandibular joint disorder, unspecified M26.601

    • Adhesions and ankylosis of right temporomandibular joint M26.611

    • Adhesions and ankylosis of temporomandibular joint, unspecified side M26.619

  1. 8.

    Seizure disorders (where no other 1–13)

    • Unspecified convulsions R56.9

    • Status epilepticus, Status disorder G40

  1. 9.

    Inflammatory, infectious and autoimmune cerebral disorders

    • Meningitis G00, G01, G02, G03, A87, A17.0, A27.81, A50.41, A51.41, A52.13, A54.81

    • Meningoencephalitis A17.82, A32.1, B58.2, B60.11, B57.42, B40.81, G04.2

    • Encephalitis A39.81, A42.82, A85, A86, A92.31, B01.1, B02.0, B05.0, B06.01, B10.0, B94.1, G04.0, G04.8, G04.9, G05.3

    • Paraneoplastic G13.0, L10.81

    • Cerebral abscess G06.0, G06.2

    • HSV encephalitis B00.4

    • Tb meningitis A17.0

    • HIV encephalopathy B20

    • Creutzfeldt-Jacob disease A81.0

    • Acute Disseminated Encephalomyelitis G04.0

    • Other specified demyelinating diseases of central nervous system G37.8

    • Multiple sclerosis G35

    • Other Encephalopathy G93.40, G93.49, G92, I67.4

    • Infection following a procedure, initial encounter T81.4XXA

    • Infection following a procedure, subsequent encounter T81.4XXD

  1. 10.

    Cardiac Arrest, anaphylaxis and risk of anaphylaxis

    • Cardiac arrest, hypoxic-ischemic encephalopathy, anoxic encephalopathy, anoxic brain injury P91.6, G93.1, I46.2

    • Cardiac arrest, cause unspecified I46.9

    • Encounter for desensitization to allergens Z51.6

    • Urticaria, unspecified L50.9

    • Adverse effect of other drugs, medicaments and biological substances, initial encounter T50.995A

  1. 11.

    Neuromuscular Disorders

    • Guillain–Barre syndrome G61.0

    • Acute demyelinating polyneuropathy G37.9

    • Chronic Inflammatory demyelinating polyneuropathy G61.81

    • Amyotrophic Lateral Sclerosis, Primary Lateral Sclerosis G12.21

    • Myasthenia gravis G70.0, P94.0, G73.3

    • Other Neuromuscular dysfunction N31.9, G12.20

    • Botulism A48, A05.1

    • Chronic inflammatory demyelinating polyneuritis G61.81

  1. 12.

    Spinal cord injury, tumor, and infections

    • Spinal epidural abscess G06.2

    • Spinal tumor C72

    • Cauda equina syndrome G83.4

    • Spinal fracture S32.0, S12.9XXA

    • Spinal cord injury S14

    • Intraspinal abscess G06.1

    • Disc infection M50.80, M50.90, M46.20, m46.45, m51,84, m51.85,

    • Neck pain M54.2

    • Disc disease, compression fracture M50.30, M50.20, M50.00, M48.50XA, M80.08XA, M84.48XA, M84.68XA, M96.1, m48.9

    • Vertebral dislocation S13.101A

    • Cervical fracture S12.000A, S12.001A, S12.100A, S12.101A, S12.200A, S12.201A, S12.300A, S12.301A, S14.101A, S14.102A, S14.103A, S14.104A, S12.400A, S12.401A, S12.500A, S12.501A, S12.600A, S12.601A, S14.105A, S14.106A, S14.107A

    • Sprain of ligaments, Other joint derangement unspecified M24.8, S13.4XXA, S13.8XXA

    • Spinal stenosis and cervical radiculopathy M54.12, M54.13, M48.02, M47.12

    • Thoracic disc herniation, dislocation, and radiculopathy, joint pain M54.14, M54.15, M54.16, M54.17, M54.6, M51.34, M51,35, M51.24, M51.25, M94.0, M54.6, S23.101A

    • Thoracic fracture S22.009A

    • Thoracic myelopathy M51.04, M51.05

    • Spondylolisthesis, lumbar region M43.16

    • Spondylosis without myelopathy or radiculopathy, cervical region M47.812

    • Spinal stenosis, lumbar region M48.06

    • Lumbago with sciatica, unspecified side M54.40

    • Benign neoplasm of spinal meninges D32.1

    • Benign neoplasm of peripheral nerves and autonomic nervous system, unspecified D36.10

    • Quadriplegia, unspecified G82.50

    • Quadriplegia, C1-C4 complete G82.51

    • Unspecified injury of neck, initial encounter S19.9XXA

  1. 13.

    Vasculopathy

    • PRES/RCVS I67.83, I67.841

    • Hypertensive encephalopathy I67.4

    • Vasculitis L95.0, I73.1, M31.7

    • Angiitis M31.0

    • Cerebral venous sinus thrombosis I67.6

    • Eclampsia O14.95

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ack, S.E., Loiseau, S.Y., Sharma, G. et al. Neurocritical Care Performance Measures Derived from Electronic Health Record Data are Feasible and Reveal Site-Specific Variation: A CHoRUS Pilot Project. Neurocrit Care 37 (Suppl 2), 276–290 (2022). https://doi.org/10.1007/s12028-022-01497-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12028-022-01497-0

Keywords

Navigation