Accepted for/Published in: JMIR Research Protocols
Date Submitted: Sep 30, 2022
Date Accepted: Dec 5, 2022
Evaluation of a clinical decision support tool using natural language processing to screen hospitalized adults for unhealthy substance use: A protocol for a quasi-experimental design
ABSTRACT
Background:
Automated and data-driven methods for screening using natural language processing (NLP) and machine learning may replace resource-intensive manual approaches in usual care of patients hospitalized with conditions related to unhealthy substance use. Rigorous evaluation of tools that use artificial intelligence (AI) is necessary to demonstrate effectiveness before system-wide implementation.
Objective:
To provide a study protocol to evaluate health outcomes and cost-benefit of an AI-driven automated screener compared to manual human screening for unhealthy substance use.
Methods:
A pre-post design to evaluate 12 months of manual screening followed by 12 months of automated screening across surgical and medical wards in a single medical center. Effectiveness in terms of patient outcomes will be determined by non-inferior rates of interventions (brief intervention/motivational interviewing, medication assisted treatment, naloxone dispensing, referral to outpatient care) in the post-period by a substance use intervention team compared to pre-period. A separate analysis will be performed to assess the cost-benefit to the health system of using automated screening.
Results:
A natural language processing tool to use routinely collected data in the electronic health record was previously validated for diagnostic accuracy in a retrospective study for screening unhealthy substance use. Our next step is a non-inferiority design incorporated into a research protocol for clinical implementation with prospective evaluation of clinical effectiveness in a large health system. The study is approved by the Institutional Review Board and registered at clinicaltrials.gov with a plan for implementation beginning in September 2022.
Conclusions:
The use of augmented intelligence for clinical decision support is growing with an increasing number of artificial intelligence tools. We provide a research protocol for prospective evaluation of an automated NLP system for screening unhealthy substance use using a non-inferiority design to demonstrate comprehensive screening that may be as effective as manual screening but less costly via automated solutions. Clinical Trial: ClinicalTrials.gov NCT03833804
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.