Can a Hybrid Decision Support System Effectively Rule Out Prescriptions from Medication Review in Daily Practice? A Randomized Case-Control Study.

Background Medication review is time-consuming and not exhaustive in most French hospitals. We routinely use an innovative hybrid decision support system using Articial Intelligence to prioritize medication review by scoring prescriptions by their risk of containing at least one medication error. Aim We aimed to demonstrate the digital tool’s ability to improve prescription safety by ruling out prescriptions that are effectively risk-free in daily practice. Methods We conducted a case-control study to compare the rate of pharmaceutical interventions (PI) between low and high-risk prescriptions dened by the tool’s calculated score. Medication orders were reviewed daily by a clinical pharmacist. Proportion of prescriptions with at least one severe medication error was calculated in both groups. Severe medication errors were characterized through a multidisciplinary approach. Results Four hundred and twenty (107 low score and 313 high score) prescriptions were analyzed. A signicant difference in the percentage of PI was found between the “low score” (29%) and “high score” (51%) prescriptions (p < 0.001). The percentage of prescriptions with severe medication errors was dramatically decreased in low score prescriptions (2.8% vs. 15,3% respectively; p < 0.05). During the study period, the use of this tool allowed to rule out 55% of all prescriptions in our hospital. Conclusion This new decision support tool is an accurate method to rule out “low score” prescriptions, with an acceptable risk of missing medication errors and can be improved by the integration of future features. It offers a solution to focus pharmaceutical expertise on the most at-risk prescriptions and considerably improve the safety of patients’ care.


Introduction
In France, 4.5% of hospitalizations are caused by healthcare-related adverse events, and 1.6% by preventable medication adverse events In addition, 20% of serious adverse events during hospitalization are related to medications errors for which 40% of those errors are preventable [1]. Medication errors, de ned as any preventable event that may lead to an inappropriate medication use or patient harm, still remain a clinical and economic burden. The annual cost of non-optimized medication therapy resulting in treatment failure or a new medical problem was estimated to reach between $495 and $672 billion US dollars, representing 16% of total US health care expenses in 2016 [2]. Secondary care (hospital and specialists) adverse drug events (ADE) were found to lead to longer hospital stay, costing £14.8 million sterling pounds ($19.1 million US dollars) and causing or contributing to 1081 deaths annually in England [3].
In hospital setting, medication orders and administrations were found to be the most common steps responsible for preventable adverse events. McCarter et al. offered solutions to reduce medication errors by improving education, implementing new safety systems and technology, and perform clinical pharmacist interventions [4]. In this respect, computerized physician orders and medication reviews by pharmacists proved to be key elements in patients' hospital care and have shown to be effective and essential to reduce medication errors [5,6]. In France, the exhaustive medication review of all hospitalized patients' medication orders is recommended and more and more encouraged as a quality and safety requirement. Every hospital is liable to provide a su cient quality management of patient care including medication review of orders, and the number of medication review achieved is a criterion among others to evaluate the quality and safety of hospital care [7]. Unfortunately, exhaustive medication review by pharmacists for all hospitalized patients remains a challenging goal to reach in French acute care hospitals. The high frequency of medication orders changes and the lack of time or trained pharmacists dedicated to this activity are the most common obstacles.
One way of improving the e ciency of clinical pharmacists is to speci cally target and review prescriptions with a potential risk of serious medication error and rule out prescriptions with no or low risk of error. To improve this approach, a study recently identi ed variables (age, renal function, number of prescribed drugs) which could help prioritize prescriptions in need of a pharmacist-led medication review [12].
With this in mind, a digital tool was recently developed, combining machine learning with Arti cial Intelligence (AI) and rule-based expert system to help prioritize medication reviews [9]. This hybrid model using knowledge-driven (expert system) and date-driven approaches (machine learning) has shown to be an accurate tool at intercepting potential prescription errors. The accuracy of this hybrid decision support algorithm showed a sensitivity of 0.81 (95% CI, 0.78-0.84) and a precision 0.75 (95% CI, 0.70-0.80), that outperformed classic prescription order analysis tools [9]. Currently used in practice, this tool helps pharmacists to prioritize their medication review activity by distinguishing low and high risk prescriptions.
Nevertheless, it seems essential to be able to test the effectiveness of the tool in real life conditions.

Aim
Our aim was to attest that the prescriptions with low risk of medication errors ruled out by the tool in everyday practice, were effectively free of any medication error with potentially severe clinical impact.

Ethics approval
Considering the type of study, international review board approval was not required as no patient data were collected at any point during the study. The research was conducted according to the principles of the Declaration of Helsinki.

Methods
A hybrid decision support system A validated hybrid decision support system combining machine learning AI and rule-based expert system was used for this study [9]. The tool calculated, for each prescription, an inferred score from 0 to 10. The score was related to the probability for the prescription to require a pharmaceutical intervention (PI), 10 being the higher probability of containing a medication error leading to a PI. The cut-off score of 2.4 was used to classify the prescriptions as "low risk" if strictly inferior to 2.4, or "high risk" if higher or equal to 2.4, as the result of the original research on the tool development, which showed that this cut-off score maximized the harmonic mean of precision and recall of the tool [9].

Study setting
The study was conducted in a private non-pro t adult hospital (592 beds) in Paris, providing both surgical and medical activities. Computerized hospitalized patients' prescriptions from all the medical and surgical wards of the hospital were included in the study, with the exception of the Intensive Care Unit (ICU), and Neonatology ICU which were using a different prescription software.

Blinded medication review by a trained clinical pharmacist
During this case-control study, a sample of 30 prescriptions was daily randomly selected by a clinical pharmacist with access to the prescriptions' scores, with a ratio of one "low score" prescription (score < 2.4) for three "high score" prescriptions (score ≥ 2.4). One other trained clinical pharmacist, blinded to the prescriptions' scores, reviewed the 30 daily prescriptions, for a duration of 14 days.
Medication review was performed in accordance with the French Society of Clinical Pharmacy (SFPC) guidelines [14]. DxCare® medical software (Dedalus, le Plessis-Robinson, France) was used to access the patient's prescription, medical le, laboratory results and vital signs and to validate the prescription or alert the prescriber or nursing team of a medication error (pharmaceutical intervention). The clinical pharmacist review was therefore considered as the gold standard.
For each medication error detected, the clinical pharmacist formulated a PI.

Primary outcomes
The primary outcome of the study was the percentage of prescriptions with at least one PI for the "low score" prescriptions compared to the percentage of prescriptions requiring a PI for the "high score" prescriptions in daily practice.

Comparison of severe medication errors in both groups
To assess the risk of missing a severe medication error by using this tool in daily practice, we evaluated the percentage of prescriptions with at least one severe medication error for both groups.
The clinical impact of the medication errors identi ed by the pharmacist was assessed by both a pharmaceutical and medical point of view. One general physician and one clinical pharmacist, independent from the study and blinded from the prescriptions scores, reviewed all the Pharmaceutical Interventions (PI) and rated the clinical impact according to a simpli ed version of the Clinical, Economic and Organizational (CLEO) scale developed by the SFPC [11]. The 4 points-scale ranged from "1-Minor" to "4-Vital".
A medication error was considered severe if it was rated as "3-Major" or "4-Vital" by both the physician and the pharmacist (and non-severe if rated as "1-Minor" or "2-Moderate" by both parties).
A conciliation meeting was organized to discuss the discrepancies between the physician and pharmacist that occurred between severe and non-severe errors, in order to reach to a nal agreement on the clinical impact.
Bene t of a hybrid approach compared to usual computerized clinical decision support systems.
To better document the added value of this innovative tool, we then compared this hybrid approach to a classic computerized clinical decision support system (CDSS) by analyzing the ability of both systems to detect each PI formulated by the clinical pharmacist during the study period. For all medication errors identi ed by the clinical pharmacist performing medication review, the ability of both systems to detect the error was calculated. For our hybrid system, the medication error was considered as detected if the associated prescription score was over 2.4. For CDSS, the medication error was considered as detected if it related to a drug interaction, or a drug's under/overdosage as de ned by the minimum/maximum drug dosage listed in the drug's label.

Variables
Daily number of prescriptions for both the "low score" and "high score" prescriptions for all inpatients in our hospital, number of Pharmaceutical Interventions (PIs) for each reviewed prescription, the inpatient ward (medical or surgical), the type of medication errors and PIs and the time spent on medication review for each prescription were retrieved.
The types of medication errors and PIs were classi ed according to the codi cation tool "ACT'IP" developed by the French Society of Clinical Pharmacy (SFPC) [12].

Statistical test
Continuous and categorical variables were expressed as mean (standard deviation) and numbers (percentage) where appropriate. Categorical variables were compared using Chi-square tests or Fisher tests where appropriate. A two tailed p value of < 0.05 was considered signi cant. All analyses were performed through scripts developed in the R software (version 4.0.3 (2020-10-10), R Foundation for Statistical Computing, Vienna, Austria).

Blinded medication review by a trained clinical pharmacist
During the study period, a total of 6794 prescriptions were written in our hospital, and 3759 had a calculated score < 2.4 between September 30th and October 23rd, 2020.
A sample of 420 prescriptions was reviewed during the 14 days period, 107 "low score" prescriptions and 313 "high score". The prescriptions were randomly distributed among 7 surgical wards and 14 medical wards of the hospital.
For those 420 prescriptions, 282 PIs were formulated, representing 191 prescriptions with at least one PI.
Primary outcome A signi cant difference in the percentage of prescriptions needing at least one PI (p < 0.001) was found between the "low score" and "high score" prescriptions, with 31 (29%) "low score" prescriptions requiring at least a PI versus 160 (51%) "high score" prescriptions ( Fig. 1).
There was no signi cant difference in the typology of medication error or PIs between the low and high score prescriptions as shown in Table 1. The main drug-related problems requiring PIs were overdosage, non-treated medical indication (medical indication present without associated treatment prescribed) and unavailable drug. The most common solution advised by the pharmacists were a dose adjustment, a drug addition or a drug replacement.

Comparison of severe medication errors in both groups
Two hundred and eighty-two medication errors were rated. During the conciliation meeting, 54 (19%) of them were discussed between the general physician and the clinical pharmacist and nally 38 (70%) were classi ed as severe and 16 (30%) as non-severe.
As expected, there was signi cantly more prescriptions containing at least one severe medication errors in the 'high score' group (n = 48, 15.3%) when compared to the 'low risk' group (n = 3, 2.8%) (p < 0.05).
Regarding the three severe medication errors in the 'low risk' group, the rst error was a possible overdosage of an injectable blood thinner drug for which the pharmacist suggested a dosage modi cation or a therapeutic monitoring (measurement of anti-Xa activity). The second error was an antibiotic under-dosage (only a third of the total daily planned dose was prescribed) with a pharmacist's suggestion for dose adjustment. The last medication error involved an untreated hyperkalemia for which a PI recommended to monitor the ECG and prescribe sodium polystyrene sulfonate (Kayexalate®).
Bene t of a hybrid approach compared to usual computerized clinical decision support systems As described before, a total of 51 prescriptions with at least one severe medication error were identi ed by the clinical pharmacist during medication review. A signi cant difference was found (p < 0.001) in the number of severe medication errors detected by the hybrid approach compared to a CDSS. When using the hybrid approach, 48/51 (94%) prescriptions with at least one severe error were detected (associated prescription score > 2.4). When using a classic CDSS, only 10 (20%) prescriptions with at least one severe medication error were detected (Fig. 2). Differences were mainly related to over or under-dosage not adapted to the disposition of the patient or drug omissions undetected by the usual computerized clinical decision support system (data not shown).

Discussion
As previously published, we con rm that our hybrid clinical decision support system can improve pharmacists e ciency in everyday practice by prioritizing high risk prescriptions [9]. In this study, we assessed the ability of this innovative tool to rule out low score prescriptions in daily practice without taking the risk of excluding prescriptions containing serious prescription errors and therefore allowing clinical pharmacists to focus their medication review activity on a reduced number of inpatients prescriptions. As expected, we found a dramatic decrease of pharmaceutical interventions needed for the low score prescriptions when compared to the high score prescriptions.
It is commonly known that pharmaceutical medication review remains complex, as it requires prede ned rules that can also be adaptable to associated prescribed drugs and patient's biological and physiological characteristics. Classic CDS systems, which are widely implemented in electronic patient records, are only intended to improve medication review by detecting potential prescription errors such as drug-drug interactions and dosage outliers.
However, designing a tool that could replicate human performance in pharmaceutical expertise remains challenging. An algorithm using con gurable rule-based engines is a rational approach but remains di cult to create due to the many rules that would have to be implemented in the tool. The use of Arti cial Intelligence (AI) could be a solution to that issue. Nevertheless, this approach cannot be exclusively relaying on AI, as many severe medication errors remains exceptional ones, therefore harder for an AI to integrate in their engines. To address this issue, we chose to create a hybrid tool using AI and a rule-based system to be able to detect occasional but severe medication errors. This tool can therefore classify prescriptions according to the risk of medication errors, which allows the pharmacist to prioritize the most at-risk prescriptions to review. We could argue that this prioritization could lead to incomplete medication review of the hospital prescriptions. However, it is important to note that this prioritization allows the selection of the most at-risk prescriptions and offer the possibility to better target the pharmaceutical expertise on the most at-risk prescriptions, which is particularly important when the medication review cannot be exhaustive.
This type of tools must constantly evolve and keep up to date with the recommended practice but also in regard to clinical feedbacks about the errors detected or not by the tool. Our study falls within that perspective by ensuring that low score prescriptions, removed from the daily and systematic medication review process, are risk-free of severe medication errors.
In our study, 36 prescriptions, including 6 low score prescriptions, had a PI concerning missing or modi ed treatments during hospitalization compared to the patient's usual treatments or untreated indications related to the patient's medical history that could not have been detected by the digital tool.
When taking into account their clinical impact, only 3 severe medication errors were found in the low score prescriptions, including 2 medication error that could not have been detected by the tool in its current settings, versus 50 severe medication errors in the high score group (p < 0.05). These nonintercepted errors allowed us to identify missing data in the score calculation, such as anti-Xa activity and natural language processing which could have allowed the tool to intercept unadapted antibiotic dosage.
In that regard and despite showing satisfactory sensitivity and precision, some improvements of the digital hybrid tool are yet to be integrated, such as the addition of biological values in the score calculation (anti-Xa activity for example) and the integration of natural language processing. When adding this last feature, the tool will also be able to detect an untreated indication for example.
In addition to the ability of this hybrid digital tool to detect potentially at-risk prescriptions, another important advantage is its capacity to better alert pharmacists about prescriptions with potential severe medication errors compared to classic CDS system. When comparing this hybrid tool to a CDS system, we found that more prescriptions with at least one severe medication errors were detected by the hybrid tool. Indeed, only drug-drug interactions and over/under dosage within the range of drug labels are detected by a CDSS. The hybrid tool has therefore the capacity to detect other type of medication errors, as for example medication errors related to patients' clinical and biological characteristics. This major difference between these tools con rms that CDSS are not able to prioritize medication review the way the hybrid tool does.
Our goal was to improve pharmaceutical e ciency to provide a more secure approach for hospitalized patients' care. Indeed, in France, a healthcare facility employs 2.8 pharmacists on average, and 1.7 pharmacists per 100 hospital beds [13,14]. As pharmacists are in charge of many activities, only a low percentage is dedicated to clinical pharmacy activities.
By using this tool in daily clinical pharmacy practice, we found that 55% of all medication orders for hospitalized patients could be ruled out from medication review by clinical pharmacists with an acceptable risk of missing prescriptions with severe medication errors, as exhaustive medication review could not be performed in this hospital anyway. These ndings could allow clinical pharmacists to reinforce other activities, such as medication reconciliation or patient education.

Conclusion
This new decision support tool using arti cial intelligence have proven to have a su cient accuracy and safety in order to rule out "low score" prescriptions. The future integration of other features would allow the tool to be even safer. When used in practice, it can offer a solution to focus pharmaceutical expertise on the most at-risk prescriptions and thus considerably improve the safety of our patients' care. It also demonstrates the contribution of arti cial intelligence in the improvement of very time-consuming processes.

Declarations Funding
No funding was received for conducting this study.

Con icts of interest/Competing interests
The authors declare that they have no competing interest.