Physicians’ and nurses’ experiences on EHR usability: Comparison between the professional groups by employment sector and system brand

Background: Usability associates with patient safety and quality of care. This article reports results from nation- wide usability-focused survey studies for physicians and nurses in Finland. Earlier research has shown dis-satisfaction and serious deficiencies, which hamper the efficient use of health information systems (HIS); however, evaluation studies covering the viewpoints of both user groups are practically lacking. Our study aimed at comparing end-users’ experiences on the usability of electronic health record (EHR) systems by employment sector and EHR brand. Methods: To measure usability, we used the validated National Usability-focused HIS Scale (NuHISS). For this study, we selected 11 usability statements that relate to technical quality (n=3), ease of use (n=6), benefits (n=1) and collaboration (n=1), and were identical in both surveys. We report the responses from 3013 physicians and 2560 nurses working in public sector hospitals or primary care health centers in 2017. Results: Results in total and by healthcare sector showed notable differences between nurses’ and physicians’ experiences on usability of their EHR systems. Physicians were more satisfied than nurses on technical quality and learnability of the EHR-systems, while nurses experienced the ease of use better and were more satisfied with collaboration aspects than physicians. Two EHR brands used in hospitals appeared to have succeeded in supporting physician workflows, while two others used in health centers were more suitable for nurses’ needs. Conclusions: Nurses’ and physicians’ experiences on EHR usability appear to vary more by EHR brand and employment sector rather than either professional group being generally more satisfied. Development of EHR systems should consider the perspectives of these two main user groups and their working contexts.


Introduction
The relationship between usability, patient safety and quality of care is evident. Poor usability associates with user fatigue, increased error rates and low user satisfaction [1][2][3][4][5][6]. Errors in electronic health record (EHR) use or medication documentation may endanger patient safety [7,8]. Moreover, EHR downtime events may pose patient safety hazards [9]. Indeed, improved usability -defined as the extent to which technology can be used efficiently, effectively and satisfactorily [10] associates with increased patient safety [11].
EHRs are complex information systems by nature, which makes their design and evaluation challenging. Various end-user groups such as physicians, nurses, and increasingly patients as well, all have diverse needs and requirements relating to EHR functionality and usability. Healthcare professionals process complex data to support a range of activities such as diagnostics, health management, care planning and treatment documentation [5]. Professionals may perceive the usefulness of the EHR systems differently depending on their objectives of usage, context of use and personal background including competency and experience of use [12].
This article presents results from two nation-wide, cross-sectional usability-focused surveys on physicians' and nurses' experiences with currently used EHR systems in Finland. The national monitoring studies were first conducted among physicians in 2010 and 2014 [13,14]. In 2017, responses were collected from both physicians and nurses [15,16]. The aim of this paper is to compare end-users' experiences on T EHR usability between physicians and nurses a) working within the same healthcare sector (public sector hospitals and public primary care health centers) and b) using the same EHR brand.

Related research
Usability contributes substantially to physicians' satisfaction on their EHRs [17]. Several studies have evaluated usability from physicians' perspective [13][14][15][17][18][19]. Hudson et al. [17] found speed (system response time) and layout as major usability factors contributing to physician dissatisfaction. Finnish surveys conducted between 2010 and 2017 have shown constant discontent with EHR usability; the main concerns have related to efficiency of use including lack of support for collaboration and intuitiveness of user interfaces [13][14][15]18].
In the recent years, a few investigations have also covered nursingcentric technologies (e.g. [16,[20][21][22]). These studies report usability challenges with the use of medication administration record (MAR) and documentation workload (e.g [6,16,23]). Nurses are also dissatisfied with the documentation tools [24]. Suitability for individualization, self-descriptiveness and error tolerance have been regarded as the main areas needing improvement [25].
Only a few survey studies have focused on monitoring usability of health information systems (HIS) from the viewpoints of various user groups. A German national HIS usability survey targeted several professional groups working in the hospital (e.g. clinicians including physicians and nurses, radiologists, laboratory personnel and administrative personnel) [26]. It covered a wide array of HIS from EHRs to laboratory (LIS), radiology (RIS), pharmacy and operating room information systems, and even a staff rostering system. However, the results were reported by information system type (e.g. EHR, LIS, RIS), not by user group or system brand. Indeed, to our knowledge, the study by Pereira et al. [27] is one of the few reporting results by user group. Their questionnaire was identical for physicians (n = 100) and nurses (n = 100) and it included ten multiple choice questions about the EHR, (e.g. design, structure, language, and system's support for data management). Physicians experienced entering patient data time-consuming and were critical about the success of system design [27]. In contrast, nurses were not as dissatisfied with time spent on documenting patient data. However, half of the nurses were not aware of speed-up tools like copying and pasting. These findings correspond with a recently published KLAS-report [28] in which physicians experienced lower satisfaction than nurses especially concerning ease of use.

Context of the study
In Finland, municipalities (n = 311 in 2019 [29]) organize social welfare and primary healthcare (health centers). Twenty hospital districts, jointly owned by the municipalities of the region, organize specialized medical care. Private sector provides a smaller proportion and variety of services [30].
In hospital wards, nurses typically work around the clock in close collaboration with other nurses, ancillary staff, and physicians. Physicians typically work in several environments -for example, a surgeon works (even during the same day) in outpatient polyclinics, inpatient wards, emergency department and in the operating roomwhereas nurses usually work in only one of these locations. In health center outpatient services, nurses and physicians work together in selfsteered multiprofessional teams.
In Finnish public healthcare, EHR coverage reached 100% already in 2010 [31,32]. However, decision support tools have not been fully integrated as yet, thus none of the EHRs can be considered "comprehensive" [33,34]. Currently, there are five large EHR brands used in public hospitals and three in public health centers. The national centralized patient data repository ('Kanta') and electronic prescription have been in use since 2014 [35,36]. EHRs are integrated with the Kanta services.

Material and methods
To measure usability, we used the validated National Usability-focused HIS Scale (NuHISS) [37]. The questionnaire was originally developed to explore physicians' experiences on HIS use and usability, particularly EHR systems [13,14,37], however, all statements were reviewed and, if appropriate, revised to be used for nurses [16]. The original survey consists of 16 background questions (including a question about the respondent's principally used EHR system), 32 usability-focused statements, a question for the overall rating of the EHR system in use, as well as other HIS questions about advantages and challenges in utilizing systems in daily work, participation in system development, and work welbeing [13,38]. Both the nurse and the physician surveys were conducted in spring 2017 by using a web-based questionnaire. The used questionnaires are available online [39,40]. For this study, we selected 11 usability statements (see Table 1), which concerned both professional groups' key EHR use tasks and belonged to the core set of usability statements [13,14], were identified as key items Table 1 Domains studied and measures used.

Domain Measure
Technical quality Q1) The system is stable in terms of technical functionality (does not crash, no downtime).
Q2) The system responds quickly to inputs.
Q3) Information entered / documented occasionally disappears from the system.
Ease of use Q4) The arrangement of the fields and functions is logical on computer screen.

Q5
) Terminology on the screen is clear and understandable (for example titles and labels).
Q6) Routine tasks can be performed in a straight forward manner without the need for extra steps using the systems.
Q7) Learning to use the EHR system does not require a lot of training.
Q8) It is easy to obtain necessary patient information using the EHR system. Q9) Entering and documenting patient data is quick, easy and smooth.
Benefits Q10) IT systems help in preventing errors and mistakes associated with medications.
Collaboration Q11) IT systems support collaboration and information exchange between physicians and nurses*.
*For nurses the statement was formatted: '…between nurses and physicians'.
in the scale validation [37], had been reported in our earlier articles [13,14] and were identical in both the nurse and the physician surveys. The physician survey was sent to all working-aged physicians in Finland who had provided their email addresses to the Finnish Medical Association (ca. 90% of all physicians in Finland; Table 2). The nurse survey was distributed by the Finnish Nursing Association and the Union of Health and Social Care Professionals in Finland to their members who had provided email addresses (Table 2). Notably, the Finnish Medical Association registry comprises all physicians in Finland, but the nurses' associations' registries only include their members.
We selected respondents working in public hospitals and healthcare centers; private sector physician and nurse respondents mainly used different EHR system brands, which would have rendered comparisons between the professional groups unreliable. We report results by the largest EHR brands (N > 30) -five brands used in public hospitals and three brands used in health centers. For the purposes of this study, of the five-point Likert scale assessments, 'Fully agree' and 'Somewhat agree' were combined to form the category 'Agree'. Similarly, 'Fully disagree' and 'Somewhat disagree' formed the category 'Disagree'. The third category remained "Neither agree nor disagree". Of the 3999 physicians with information on working sector available, 1943 (51.1%) worked in public hospitals and 1070 (26.8%) in health centers. The respective figures for nurses (n = 3607) were 1833 (50.8%) and 727 (20.2%). The respondent characteristics are presented in Table 2.
Statistical analyses were carried out with SPSS 22 (IBM Corp, Armonk, NY). The Chi-square test or Fisher's exact test were used to compare categorical variables when applicable. Statistical significance was determined as p < 0.05.

Results: comparison between physicians' and nurses' experiences
The results are presented by a) healthcare sector (hospital vs. health center; Table 3), and b) EHR brands (EHRs used in hospitals (Table 4) and in health centers (Table 5)).
Physicians in health centers were more satisfied than nurses with systems' ability to help in preventing errors with medication (49% vs. 37% (p < 0.001) agreeing) whereas the proportions were more similar among physicians and nurses working in hospitals (37% physicians vs. 40% of nurses (p = 0.032) agreeing).

Experiences from public health centers by EHR brands
While nurses' and physicians' responses on EHR brand C ho (used in hospitals) differed considerably, the same was not seen among those working in health centers (C hc ) ( Table 5). Concerning the three health center EHR brands (A hc , C hc and E hc ), the results showed similar overall trend as the analysis by healthcare sector (presented in Table 3): Physicians' responses to statements about technical quality (Q1-Q3) were more generally positive than nurses', whereas nurses' responses on ease of use (Q4-Q6, Q8, Q9) were more positive. Compared to brands A hc and C hc , the proportion of agreeing responses concerning brand E hc from both groups were lower, particularly on statements Q4, Q5, Q7 and Q10. For EHR brand A hc , nurses were more positive, especially on statements Q6 (44% vs. 25% (p < 0.001) agreeing) and Q8 (54% vs. 42% (p < 0.001) agreeing).

Main contribution
Our study is one of the first to compare nurses' and physicians' experiences on EHR usability on a national level by utilizing the same validated measurement scale and questionnaire items with a large number of respondents. Results in total and by healthcare sector showed notable differences between nurses and physicians. Our findings concur with earlier studies with reports on low level of satisfaction [3,4,6] and usability problems related to data entry, poor system support for workflow and visual display (e.g. [3]). Moreover, our study  (continued on next page) suggests that the differences between the professional groups vary more by the EHR system brand in use rather than either group being generally more positive about the usability of the EHRs. Our results contradict earlier studies, which have reported nurses being more satisfied with their systems than physicians [27,28]. Previous usability surveys have mainly focused on the viewpoint of one user group (e.g. [14,24]) or analyzed the results per clinical IT systems by compiling the responses of all professional groups (e.g. [26]). Furthermore, in several usability evaluation studies the sample size has been small, and the findings have limited generalizability (e.g. [27]). Importantly, in our data, the differences between the two professional groups varied also by usability domain.

Comparisons between nurses and physicians 6.2.1. Technical quality
As compared with nurses, physicians were more satisfied with the technical stability of their EHRs (Q1). This was especially evident with hospital brands B, C and E. Results on the responsiveness of the EHR (Q2) were similar. One explanation for these differences may be that, especially in inpatient wards, nurses are in constant interaction with the EHR and even minor technical problems interfere their work. Moreover, most planned downtime is at night when double documentation (paper and EHR) is often needed especially from nurses. Similarly, differences between the two groups in experiences on entered or documented information disappearing from the system (Q3) seemed to vary greatly by EHR brand. Experience of disappearing information may be explained either by poor usability i.e. user does not find documented data or instability of the information systems.

Ease of use
Nurses experienced some aspects of ease of use better than physicians -support for routine tasks (Q6), ease of obtaining information (Q8) and terminology (Q5)-whereas physicians were more positive on learnability (Q7) and ease of documenting data (Q9). Differences in experiences about entering and documenting patient data (Q9) appeared to depend on EHR brand instead of healthcare sector. As compared with physicians, nurses using brand C in health centers and in hospitals were less satisfied with their EHRs. The national nursing documentation model, Finnish Care Classification (FinCC) is implemented into brand C. However, it has been implemented also into brands A ho and E ho [21] but here the results do not show significant differences. Interestingly, physicians perceived the learnability of almost all EHR brands better than nurses (Q7), however, nurses' responses differed by EHR brand in use; a more detailed study would be needed to analyze whether this depends on the nursing documentation model used.

Benefits
One of the expected benefits of EHRs is support for improving medication safety. Physicians are responsible for medication-related decisions and ordering while, in Finland, nurses are responsible not only for the medication administration process but also dispensing as the few hospital pharmacists are typically only available during office hours. At the time of our survey, none of the healthcare organizations had implemented closed-loop medication administration process or integrated alerts for inappropriate dosing. Therefore, the differences between EHRs can be considered to reflect the impact of computerized physician order entry (CPOE) tools, drug-drug interaction alerts, eprescribing and the usability of the MAR that are all fully implemented in all hospitals. By contrast, among health center physicians the proportion of physicians agreeing (statement Q10) was higher compared with nurses. It is possible that the available tools are more suitable for primary healthcare physicians who usually treat less acutely ill patients.

Collaboration
As compared with physicians, nurses were generally more satisfied with the EHRs' abilities to support collaboration and information exchange (Q11). Especially in hospitals, most EHRs have separate documentation modules for nurses and the quality of the user interfaces of nursing documentation modules may vary (as the results by EHR brands concerning the statement Q9 suggest) as well as the solutions for noticing the nurses on orders placed by physicians.

Differences between EHR brands
Interestingly, some EHR brands (e.g. B ho and C ho used in hospitals) appeared to have succeeded in supporting physician workflows, while others (e.g. A hc and E hc used in health centers) were more suitable for nurses' needs. The common denominator for the two systems with greatest physician satisfaction (B and C) is that, to our knowledge, they have both been originally developed primarily by physicians.

Limitations
A comprehensive usability evaluation of EHRs is a time-consuming and resource-intensive process due to complexities associated with work processes, range of user groups and healthcare contexts [26]. Despite survey having several restrictions as a usability assessment method, it is often used in evaluation studies [19] and considered suitable for national cross-sectional monitoring studies [36]. However, self-reported measures may lead to an inflation of the strengths of relationships and common method variance. To minimize problems associated with self-reports, we used measures with good reliability used in previous studies, including our recently validated measure of  [36]. However, in order to support the user-centered development of EHR functionalities, we recommend also using qualitative research approach and more detailed questions for a more in-depth view of the phenomenon. For example, our questionnaire did not include detailed statements about medication ordering and administration documentation tools either from physicians' or nurses' perspective. Moreover, EHR-supported collaboration between physicians and nurses should be assessed with more detailed questions. Furthermore, our data does not allow scrutinizing results by working context within the hospital since, in Finland, clinicians use two HIS (specialized HIS and the "main" EHR) in operating rooms, emergency departments, or intensive care units and our survey covered only the "main" EHR brand. Moreover, most physicians work in both outpatient polyclinics and inpatient wards. While the total number of respondents in both surveys can be considered rather high, the response rates remained relatively low particularly among nurses. However, for physicians, we carried out a nonresponse bias assessment and the comparison with the target population showed good representativeness of the sample [15]. The representativeness of the nurse data has been discussed previously [21] and it has been concluded that the respondents represented well nurses employed in hospital services and in health centers. This survey has been conducted already twice among physicians, whereas this was the first time among nurses which may partly explain the higher physician response rates. Finnish Medical Association also advertised this survey extensively. Findings of physicians being highly stressed with their information systems [2,42], may further motivate to answer surveys related to them.

Further research
Our findings highlight that development of EHRs should consider the perspectives of the two main user groups and their working contexts. Future surveys should cover the various aspects of usability since within the same EHR and context, physicians and nurses found different areas of usability challenging. Even though the workflows of physicians and nurses differ, EHRs should support fluent multidisciplinary information exchange and patient care.
Recent research suggests that the success of the implementation process might strongly influence the usability and safety of an EHR system. Ratwani et al. [8] found wide variability in task duration, clicks and accuracy when physicians were completing basic functions across EHR products based on given scenarios. Depending on the quality and success of the implementation process, even the implemented EHR products from the same vendor may differ in terms of usability and safety. Further research should analyze the survey results from nurses and physicians per the implemented EHR product. This information would be useful for the vendors and implementing organizations to support successful implementation.
In Finland, the results from the national monitoring studies are used to follow the development and implementation of the EHR systems and this work is supported by the national e-Health strategy by the Ministry of Ministry of Social Affairs and Health [43]. The usability-focused questionnaire studies will be repeated in 2020 among physicians, nurses and, for the first time, the survey will cover also social workers' perspectives. This will enable comparison of the results between the years and differences between the experiences of the groups reported in this article.

Conclusions
Usability ratings overall were low which is consistent with earlier studies. The observed differences between nurses' and physicians' experiences on EHR usability appear to vary more by the measured aspect of usability, EHR brand and employment sector rather than either professional group being more positive than other. While nurses were more critical about learnability and stability of the systems, physicians criticized ease of use and support for collaboration. Therefore, the perspectives of these main user groups and their working contexts should be considered in the development of EHR systems as well as the requirements for usability that arise from multidisciplinary healthcare teams.
Authors' contribution D.Sc.(Tech.) Johanna Kaipio had the main responsibility for the manuscript. Tinja Lääveri (MD) had main responsibility for the statistical analysis. All the authors (Kaipio and Lääveri together with Anne Kuusisto PhD, RN; Hannele Hyppönen, PhD; and Tarja Heponiemi, PhD) contributed actively to all parts of the article, including data interpretation, revision and approval of the manuscript. In addition, authors Kaipio, Lääveri and Hyppönen contributed to the design of both surveys studies as well as to the development of the used nation-wide usability-focused survey instrument.

Summary points
What was already known on the topic? -Usability associates with patient safety and quality of care.
-Usability problems of health information systems (HIS) hamper the efficient use and clinical work. -Both physicians and nurses are dissatisfied with their current EHR systems.

What this study added to our knowledge?
-This study is one of the first to compare nurses' and physicians' experiences on EHR usability on a national level. -Results showed that experiences vary by usability aspects measured, EHR brand and employment sector rather than either professional group being generally more satisfied than the other. -The findings contradict earlier studies reporting nurses being more satisfied with their EHR systems compared to physicians. -Development of EHR systems should consider the perspectives of the main user groups and the contexts of healthcare work.

Declaration of Competing Interest
We wish to confirm that there are no known conflicts of interest associated with this publication and there has been no significant financial support for this work that could have influenced its outcome.