Keywords

1 Introduction

Diversion of controlled substances (CSs) in healthcare poses a risk of harm for patients by exposing them to insufficient pain management, incorrect medications and their documentation, possible risk of infection from contaminated needles, and healthcare personnel incapacitated by narcotics misuse [1]. In the USA, it has been estimated that 1% of healthcare workers divert drugs [2], which underscores the necessity to strictly monitor and control the use of these substances.

In Finnish healthcare, the distribution and handling of CSs has been strictly regulated by national legislation since 1971. Both narcotics and psychotropic medicines are considered CSs; despite their therapeutic value for medical purposes, such as pain relief, they present a risk for drug abuse and diversion [3]. Hospital pharmacies are required to keep records of the distribution and consumption of CSs in hospital settings [4]. Traditionally, the CS documentation process has relied on paper-based practices. CSs are delivered with a package-specific consumption monitoring form from a hospital pharmacy to a healthcare unit [4]. When a CS package is empty, the filled consumption form (i.e., the consumption record), accompanied by the physician's verification signature, is returned to the hospital pharmacy for examination and approval. CS consumption records are required to be archived for at least six years after the end of the year in which the document was prepared [5].

Modern electronic health record (EHR) systems, however, allow the paper-based CS surveillance process to be digitalized. In Finland, HUS Helsinki University Hospital and four municipalities implemented a new EHR system between 2018 and 2021 [6]. The Epic-based Apotti system brought about closed-loop electronic medication management within the same EHR: ordering, pharmacy verification, reconstitution, patient identification, barcoded medication administration, and monitoring are executed electronically without manual copying [7]. It also enables automated data transfer between the EHR and smart infusion pumps or automated dispensing cabinets. All of these features decrease the need for error-prone manual work and reduce the risk of medication documentation errors [8,9,10]. These functionalities also enable automatic entries into an electronic narcotic consumption card (eNCC), which was built into the EHR and thereafter implemented gradually at HUS Helsinki University Hospital. The first pilot took place at three inpatient wards in November 2021 to examine whether the digitalized process is applicable for broader implementation. Thereafter, three implementations were completed successfully from 2022 to 2023. By the end of 2023, more than 80 units had begun using the eNCC.

There are many anticipated benefits of the eNCC for hospital pharmacies related to the distribution and handling of CSs. However, this implementation of new digital processes may create new challenges for the healthcare professionals (HCPs) using the EHR, including nurses, physicians, and clinical pharmacists. Research into the digital solutions related to NCC workflows and eNCC solutions remains scarce [11]. Putri et al. [12] evaluated the usability of the narcotics and psychotropic reporting system and found that pharmacies were quite satisfied with the system.

From end users’ perspectives, the usability of the solution impacts work efficiency, error-free task completion, and user experience (UX) [13,14,15]. The objective of this study was to explore the usability and UX of the eNCC solution built into the EHR system, and the related end-user workflows of nurses, pharmacists, and physicians in the wards by comparing the new digitalized process to a conventional paper-based narcotic consumption card (pNCC). The research questions were as follows:

  • How do the HCPs’ workflows using eNCC differ from the conventional paper-based process?

  • What kinds of experiences do HCPs have with the recently implemented eNCC?

2 Methods

A qualitative approach was used for this research because the eNCC was only in the pilot phase in spring 2022, and very few wards had begun using it. This field study was conducted using semi-structured interviews, observations with usability measurements, and surveys for HCPs. Semi-structured interviews were selected as the primary method to research workflows and UX, because interviews can be used to gather in-depth and detailed information while giving interviewees a chance to bring up themes absent from the interview script [16] Through observations, data can be collected that end-users may not be able to express verbally [17]. In field studies, observations can be conducted without interrupting the observed individuals [17, 18], which is particularly important in clinical environments. During observations, measurements related to tasks and usability can also be carried out, and information about the work environment can be gathered to support interviews [17]. The usability of interactive systems can be measured in terms of effectiveness, efficiency, and satisfaction [19]. Suitable metrics for measuring these variables include task completion time and error rates during task completion as well as standard usability questionnaires [19]. Finally, the study included a validated usability survey, System Usability Scale (SUS) [20], with complementing open-ended questions about development ideas related to the evaluated solution.

The research data were collected at a large university hospital (HUS Helsinki University Hospital) from the hospital pharmacy, one inpatient ward that used the eNCC, and one ward that used the pNCC in spring 2022. We selected these two wards because they were surgical units that cared for patients who frequently needed opioid-based postoperative analgesics. All the nurses and physicians working in these wards were invited to participate in the study, though the final participants for the study were selected through convenience sampling. The research data were collected during the daytime, which limited the potential participants. Moreover, the study was affected by labor actions, which affected the number of nurses available for the study. The number of personnel varies at the wards that involved in the study, but is typically 20–50 nurses and 10–20 physicians.

The study was conducted in two parts. The first part focused on the workflows related to the NCC (both paper-based and electronic). The aim was to describe and validate workflow descriptions. The initial versions of the descriptions were created before the interviews based on documentation available from the hospital pharmacy, the EHR system administrators, and discussions with domain experts. Two hospital pharmacists and a nurse participated in the interviews, which were conducted in their real working environments. The interviews included questions related to workflows, document handling, and the challenges and advantages of the eNCC. As part of the interviews, the participants were also asked to demonstrate the workflows in practice. Experiences from the first part were utilized when planning the data gathering and practical arrangements for the second part.

The second part focused on the usability and UX of the eNCC. The procedure included observations with discussion of the processes, observations with measurements, and a semi-structured interview, including a survey. All observations were conducted in medication rooms and focused on time measurements related to handling and documentation of CSs. The researcher utilized a predefined template to document observations and time measurements. Interview questions were designed based on the expertise of the research group. The data gathering was conducted in nine sessions, which lasted from three to four hours and included one to three participants. Each participant was interviewed and observed one at a time. The duration of an individual time measurement event was only a few minutes within the observation sessions. The interviews lasted about 15–30 min with each participant. After the interview the participants were asked to answer to the SUS questionnaire on a paper format. Field notes were written during the observations, and both interviews and observations were audio recorded.

In total, the second part included 13 participants: ten nurses, two pharmacists, and one physician. Table 1 provides the number of participants and their categorization between NCC types and professions. The physician participated only in an interview and survey; observations were not conducted due to the small sample size and the different work tasks.

Table 1. Number of participants in the second part of the study and their categorization between the use of paper-based narcotic consumption card (pNCC) and electronic narcotic consumption card (eNCC).
Full size table

The study had a research permit from HUS Helsinki University Hospital, and the participants signed a consent form before the interviews and observations. Audio recordings were used to ensure that no patient data would be unintentionally recorded, which might have occurred with photos or video recordings. The research data were pseudonymized for analysis. The study data consisted of participants’ background information, audio recordings from the interviews and observations, measurements from the observations, and responses to the survey.

In the first part, the analysis was conducted forming process charts of the workflows with verbal descriptions. The audio recordings of the interviews and the participants’ comments were transcribed and coded thematically by one researcher. The thematic coding [21] utilized predefined codes that are commonly used in usability and UX research (such as, problems, positive and negative experiences and technical challenges) and in addition to these new codes were added (such as, development suggestions). Affinity diagrams [22] were utilized in the analysis process. The affinity diagram of usability and UX findings consisted of approximately 350 observations that were grouped under 26 different thematic groups and further combined under three main themes and five sub-themes.

From the observation data, we recorded elapsed time, usability findings, and irregular instances. From the elapsed time, the averages, standard deviations and interquartile ranges were calculated. Data from the observations were mostly analyzed with a qualitative approach. Responses to the SUS were analyzed with the standard analysis method [20], which results in a score between 0 and 100 points. The open-ended questions also were analyzed thematically and combined with the interview results.

3 Results

3.1 Workflows

At the hospital pharmacy, there were three differences in the workflows between the conventional paper-based pNCC process and the digitalized eNCC process. First, paper prints were handled in the conventional process. Second, the digitalized process required the use of 2D codes to create the eNCCs in the EHR. Third, eNCCs were created in the EHR system though all other pharmacy-related workflows were documented in the pharmacy enterprise resource planning (ERP) system. The process of CSs delivery at the hospital pharmacy is presented in Fig. 1.

Fig. 1.
figure 1

The processes of paper-based and digitalized CS delivery at the hospital pharmacy

Full size image
Fig. 2.
figure 2

The processes for paper-based and digitalized NCC examination, approval, and archiving at the hospital pharmacy

Full size image

CSs were ordered, and packages were identified similarly regardless of the NCC process. Each package was marked with a unique identification number label that connects the CS package and the NCC although the type of barcode varies depending on the process type. After the CS package was used, an empty package was discarded and the NCC with the physician verification signature (hand-written for pNCC and digital for eNCC) was returned to the pharmacy for examination, approval, and archiving (Fig. 2). The most remarkable change in pharmacy activities was to render manual paper archives unnecessary with eNCC. Digital archiving does not require a separate room, and the NCC records are accessible in the EHR system.

For administration, medications were prepared in the EHR system via two different workflows depending on the administration route or dosage form (Fig. 3). The barcodes or 2D codes of tablet and capsule packages were scanned in the medication room, and the CSs were placed in a container with the patient’s identification label. Each injection required a printed label with specific patient information; then, the label and a package were scanned in the medication room, and the label was scanned again in the patient room. Workflows were the same regardless of the NCC process, except for the last step of the dispense preparation workflow. Double verification for the waste of partial injection vials by another HCP was not required in the pNCC process but was mandatory in the eNCC. In the eNCC, all unused CSs were considered waste requiring double verification. By contrast, separate documentation for medication administrations in the pNCC was omitted in the digitalized process with the eNCC.

Fig. 3.
figure 3

Medication administration process for controlled substances (CSs) with paper-based (p) and electronic (e) narcotic consumption card (NCC) at hospital units in the EHR system

Full size image

The medication administration process remained almost unchanged in the wards that had implemented the eNCC (Fig. 3). In the paper-based process, all required information had to be copied manually into the separate pNCC although most information was already recorded in the EHR system. By comparison, the eNCC automatically utilized all documented information and displayed it in the patients’ charts. There were no more hand-written notes in the wards, which made reading the NCCs easier.

Barcode scanners were used in medication administrations regardless of the NCC process although scanning was a prerequisite for fluent documentation in the eNCC. If the scanning was skipped, automatic entries would not appear in the eNCC. Because hand-written entries in the pNCC were separate from the medication administration documentation in the EHR, skipped scanning did not influence entries in the pNCC.

Handling of the pNCC required physical access, whereas the eNCC could be accessed through the reporting tools in the EHR system. The consumption balances in the pNCC had to be counted manually in contrast to the eNCC where the balances were counted automatically in the EHR system based on earlier entries.

3.2 Usability and UX

Based on the interviews, user experiences of the digitalized process with eNCC were divided into three main themes (efficiency, reliability, and technical challenges) and six sub-themes (Table 2).

Table 2. UXs of the eNCC based on HCPs (inpatient nurse, physician, and pharmacist) experiences of paper-based and digital workflows, n = 14.
Full size table

According to the interviews, the workflow for administering tablets and capsules with eNCC was especially efficient because all documentation was done in the EHR system, and entries appeared automatically in the eNCC based on scanning 2D codes in the medication room. The participants reported that the eNCC workflow for tablets “is as easy as can be,” and the eNCC “has been easy to learn.”

Interviews also indicated that eNCC has increased reliability for CS documentation. One participant commented that “[the documentation is] faster and more reliable, and not many erroneous entries presumably.” Another stated that “everything is more visible, even though the right workflow wasn't followed.” The eNCC was considered to be “easy to read; everything seems to be unambiguous,” and “everything is visible because of the automatic workflow, and there is no unclear handwriting.”

Most of the experienced challenges were related to injections. All unused amounts of CSs from injection vials had to be verified by another HCP’s signature, which was quite easily forgotten if not done immediately after the administration. As one of the participants reported, “There are multiple steps for injections; one must also remember to document an entry for the waste and ask for a double verification from a colleague, which is complicated.”

When comparing the paper-based and digitalized workflows, the main advantages and challenges experienced were as follows:

  • The digitalized process is easier and faster for tablets and capsules preparation, mainly because of the automatic workflow and the use of barcode scanners. With the digitalized process, possible errors in workflows are more visible; it is easier to find errors and make corrections.

  • Perceived challenges are related to injections because they are prepared using a different workflow. Due to multiple steps, the process of injection preparation is prone to errors in the eNCC. In addition, injections usually require double verification for waste entries.

Based on the observations and measurements, the digitalized workflow appeared to be faster than the paper-based one (Table 3). For the paper-based process, the average time taken for pending administrations of tablets and capsules was 107 s, and for the digitalized process, it was 80 s. Some interruptions in the workflows were recorded that affected task completion times. Most exceptions were part of the normal workflow, such as a rush in a medication room, opening new CS packages, inventory of the tablets and capsules from used packages. If interruptions were irrelevant to the task, they were not included in the study measurements. Administration workflow in the patient room was identical irrespective of the NCC process, so no measurements were conducted related to this task.

Table 3. Time consumed for pending administrations of tablets or capsules in paper-based and digitalized NCC processes, in seconds (s)
Full size table

Inventory of CSs appeared faster with eNCC than pNCC at wards. Workflow was observed for both processes, but completion times were not comparable as only two cases were observed. However, the most remarkable differences in the inventory workflow were the use of the EHR system and handling of papers. The EHR system was required in eNCC, whereas all of the consumption information is found on paper in the pNCC.

As a part of the interviews, the participants responded to a SUS questionnaire. The questionnaire was completed by four nurses, one ward pharmacist, and one physician. The average score was 58 and the median was 65. One participant gave an extremely low score (8), which lowered the overall score. In the SUS scoring scale, the score 58 can be considered as marginal, between not acceptable and acceptable [23]. A SUS score below 50 is considered not acceptable, above 70 acceptable, and a good score is 73.

4 Discussion

The distribution and handling of CSs, such as narcotics, is strictly monitored in healthcare. The full digital integration of CS surveillance and medication administration processes offered by the eNCC reduces the opportunities for drug diversion. HCPs often criticize EHRs for not supporting and facilitating routine tasks [24]. Therefore, while studying the usability and UX of a recently implemented eNCC in inpatient wards participating in the pilot, we wanted to ensure that the increased monitoring of quality and hospital pharmacy digital workflow efficiency would not negatively impact the HCPs’ end-user experiences. To our knowledge, this was the first implementation of fully digital CS surveillance process in Finland..

The main result of our field study was that the digital process may even improve UX by reducing the need for manual steps. The end users believed that the eNCC enables faster and easier workflows for tablets and capsules, more reliable entries, and higher traceability for all CSs in comparison to the pNCC. In addition, the eNCC provided better opportunities to investigate and fix errors.

4.1 Comparisons of the Paper-Based and Digital Processes

From a CS surveillance perspective, the main difference between the paper-based and digital processes was that the entries from the EHR were automatically transferred to the eNCC. In practice, this automation may reduce the possibility of diversion, as entries to the eNCC could only be falsified by falsifying the medication administration documentation in the EHR. Indeed, most medication administration documentations in modern EHR systems are based on identifying the patient and the medication by barcode scanning [6]. Moreover, a physician’s order in the EHR is technically required to document the administration of all medications, including CSs, which increases the reliability of documentation and may further reduce the risk of diversion [9]. In the paper-based process, systematic verification of all pNCC entries from the EHR would be practically impossible as it would require a laborious manual process. In addition, in the paper-based process, hand-written entries may result in interpretation errors; the digital process allows the hospital pharmacy personnel to concentrate on genuine discrepancies; in essence, the possibility of detecting actual diversion attempts is likely to be higher. Although interpretation errors are irrelevant in the eNCC, omitting steps in the medication administration process may cause new types of errors compared to the pNCC. Fully integrated eNCC in EHR provides easier and more frequent access to CS consumption records because a separate login is not required, as Witry et al. [25] suggested. Moreover, there is always a risk that pNCCs can disappear intentionally or accidentally.

4.2 Benefits of a Digital Process from the Hospital Pharmacy Perspective

In Finland, hospital pharmacies are mandated to keep records of CS consumption in hospital settings, which was already identified as an advantage of the digitalized NCC process before implementation of the eNCC. Archiving of the eNCCs can be done without extra effort, and all necessary information is found in the EHR system. Some problems related to drug diversion are completely precluded; for instance, it is impossible to steal eNCCs, and entries are hard to falsify due to automation. The generally identified [26] benefits of paperless processes could be seen with the digitalized eNCC process: 1) it can be remotely accessed; 2) it does not require archiving space; 3) it can be used simultaneously by several end users; and 4) it cannot be easily destroyed.

4.3 Perceived Benefits for Digitalized Processes from the HCP Perspective

The participants reported noticing a reduction in the potential for documentation errors for tablets and capsules. In general, the pharmacists’ routine CS-related tasks at wards, such as local monitoring, inventory, and investigation of errors, were considered easier with the digitalized process. Because the pharmacists may work at several wards, the reporting tools in the EHR provide them with access to CSs’ consumption data independently of their location, which was not possible with the pNCC. This benefit of remote access has been identified in other paperless processes [26].

In Finland, nurses primarily dispense and administer medications, including CSs. All interviewed nurses believed that the eNCC accelerated and eased NCC documentation for tablets and capsules. Indeed, the manual documentation phase in pNCC can be totally omitted with the eNCC by scanning the 2D code from the packages during the medication preparation and administration. If the predefined workflows were followed, all necessary information would be automatically documented in the eNCC. However, the preparation of injections was considered more time-consuming, complicated, and prone to errors. While the workflow for administering injections in the EHR did not change due to the implementation of the eNCC, it includes multiple steps, and skipping any of those steps was likely to cause errors in the eNCC. Injections are prepared differently in the EHR system than tablets and capsules because of a distinct workflow in the medication room. Tablets and capsules must be identified by scanning the package once in the medication room. By comparison, injections and other intravenous medications have to be identified twice; a label and components of the injection are scanned first in the medication room after which the label of the prepared injection is scanned in the patient room. The use of labels and barcode scanners can be considered as technology-based systemic defenses that are required for closed-loop electronic medication management to mitigate medication safety risks [27, 28]. Thus, workflows in the EHR aim for closed-loop medication administration for injection but require barcode scanning in multiple phases of the process. Although the EHR system requires end users to administer medications with medication safety, standardized workflows are partially based on instructions, and end users are not forced to use the standardized workflow for the specific dosage form by the EHR system.

We only interviewed one physician, who felt it was too complicated to use reports for signing the eNCCs. Moreover, the physician apparently had not recognized the benefit of the eNCC of not needing to manually copy from the EHR; this removes the possibility of human error, whether intentional or unintentional. To our knowledge, physicians hardly ever actually verify that the entries in the pNCC also appear in the EHR. Interviewed pharmacists also used reporting tools in the EHR, but they did not report the same usability problems. According to an earlier Finnish study related to the usability of EHR systems, physicians experienced the ease of use of their EHR systems worse than nurses, for instance [24]. After the pilot, more attention was given to training the physicians to use the reports. Additionally, after the interview, the usability of the physicians’ reports was improved.

4.4 The eNCC Revealed Existing Workarounds in Medication Processes

Regardless of the different workflows for tablets and capsules or injections, our study identified that documentation- and workflow-related errors became more visible with the eNCC than with the pNCC. Most of the increased transparency was due to scanning and automatic entries. In fact, the documentation processes for the pNCC and eNCC were almost identical except for documenting waste for partly used ampoules with the eNCC. The introduction of the eNCC revealed that the end users had used workarounds instead of the correct and more medication safety-oriented workflows. The correct, predefined workflow required by the closed-loop electronic medication management process is suggested to increase safety [7, 9, 29] but is different from the workflows used before the implementation of the new EHR. Indeed, digitalization often reveals workarounds that may contribute to the dissatisfaction of end users and compromise patient safety [30]. HCPs may adopt workarounds to avoid new additional steps in workflows when transitioning from a paper-based systems to EHRs [31]. Barcoded medication administration systems are known to improve medication safety, but workarounds can nullify the effects of error prevention; for instance, documenting the medication administration in the EHR after giving the medication to the patient poses a risk to medication safety [32]. Although the new EHR system was implemented several years ago [6], the eNCC seemed to disclose some workarounds in the medication administration workflow.

4.5 Opportunities for Improved CS Surveillance

The closed-loop electronic medication management process has improved medication safety by ensuring that the right patient receives the right medication and dose at the right time with seamless information sharing and documentation [7]. However, the surveillance of CSs often includes manual steps leaving an opportunity for drug diversion [1, 9]. We implemented a digital eNCC process to close the surveillance loop from the hospital pharmacy to dispensing and administering the CSs at wards and back to the hospital pharmacy that utilizes the same entries as in the actual medication process. The digitalized eNCC process is also likely to prevent incorrect interpretation of narcotics consumption records because the eNCC is not based on handwriting like the pNCC. Compared to the pNCC, it is more difficult to falsify entries in the eNCC, mainly due to the automated process and the requirement of a physician’s order. Although digitalizing the CS surveillance process is a great step forward in preventing drug diversion, it cannot be the only action to limit CS theft and illegal nonmedical use at healthcare organizations [1]. Other actions include, for instance, regular inventory of CSs, physical access controls for HCPs, and auditing.

Although the digitalized process relies on scanning workflow and automatic entries, waste entries always required double verification by another HCP regardless of the amount of waste. With the pNCC, double verification for the waste of partial injection vials was not needed. The tightened requirements related to double verification could improve CS surveillance, but the results also suggested that participants experienced double verification as hard to remember if not done immediately after administration.

4.6 Strengths and Limitations of the Study and Future Research

The process described in this paper was aimed at further reducing the risk of diversion of CSs and to streamline the legislation-driven process in hospital settings. However, a remarkable share of narcotics, especially weak opioids, are dispensed from community pharmacies [33] and, thus, eNCC is only one measure to manage and control the handling of CSs. Moreover, we have not deployed any automated algorithms to identify potential drug diversion situations; however, there is inconclusive evidence on the efficacy of these algorithms [34].

There are some other limitations that deserve to be discussed. As this study was related to the piloting of the eNCC, there were few HCPs at the hospital using the eNCC at the time of the study. The study focused on wards; therefore, the pharmacy side of the processes was not addressed. Consequently, the number of participants in the study was rather small.

In the study, we primarily applied a qualitative approach on researching user experiences, and the numeric usability results could not be analyzed quantitatively. The use of three complementary field research methods in the study can be seen as a clear strength. Even though the number of participants in the study was limited, the study managed to research the UX of three HCP groups with the eNCC. It is important to explore multiple perspectives of end-user groups instead of focusing on only one when the aim is to understand the practical benefits of recently implemented digital solutions and to provide information to support development work and wider implementation.

Our study provided initial results about usability measurements related to eNCC benefits. However, the small sample size needs to be considered. It was difficult to research and observe usability and user experiences related to unexpected events and exceptions because they occur rarely in workflows and would require extensive observation times over long periods in the wards. In the future, it would be beneficial to utilize log-data together with observations to gather rich data about errors and exceptions related to eNCC use and digitalized workflows. In addition, it should be noted that the practices and timing of medication distribution were not entirely consistent among the two wards in the study. Due to these differences, the measurement results should be considered very preliminary, and further research is needed to conduct additional measurements and strengthen the generalizability and validity of the findings. For future research, we are planning a larger-scale study once the eNCC is being used in more wards.

Digital medication processes are intended to improve medication safety and surveillance. However, after the first implementations of the new EHR system, ordering errors increased [6], and several usability problems were identified, particularly after the first go-live [35]. Therefore, we wanted to analyze possible usability problems before the large-scale implementation of the eNCC solution. Indeed, several usability problems were identified after the first pilot and this research project and fixed before the broader implementations. Moreover, as new workflows need to be followed, communication and trainings were improved for later phases of the implementation. Our findings underscore the importance of piloting new EHR features with a smaller group of end users whenever possible.

5 Conclusions

While the benefits of a digitalized process (i.e., the eNCC) were obvious from the hospital pharmacy point of view already before the implementation, changes in the workflows and usability from the clinicians’ perspectives deserved deeper attention. Compared to paper-based practices and processes, the main advantages were related to improved efficiency and reliability. Based on HCPs’ experiences, the eNCC enabled faster and easier workflows for tablets and capsules and higher traceability for documentation regardless of the workflow. Additionally, the eNCC with automated entries might reduce the number of errors in the documentation. However, a successful eNCC process requires following precise and predefined workflows, which emphasizes the role of training and communication before and during implementation.