Evaluation of Electronic Medical Record Implementation from User’s Perspectives in Ayder Referral Hospital Ethiopia

An Electronic Medical Record (EMR) is the legal and longitudinal electronic record of patient health information that is created in digital format in Hospitals. The EMR system may include demographics, medical history, medical and allergies, laboratory test results, radiology images and vital signs [1]. It is a computer based systems for input, storage, display, retrieval, and printing of information contained in a patient’s medical record and have become necessary for proper management of patients [2,3].


Introduction
An Electronic Medical Record (EMR) is the legal and longitudinal electronic record of patient health information that is created in digital format in Hospitals. The EMR system may include demographics, medical history, medical and allergies, laboratory test results, radiology images and vital signs [1]. It is a computer based systems for input, storage, display, retrieval, and printing of information contained in a patient's medical record and have become necessary for proper management of patients [2,3].
While the healthcare industry as a whole of the world wide has made significant investments in technology for the purpose of diagnosis, clinical and treatment, it has been comparatively slow to adopt Health Information Technology (HIT) and health information exchange as a health management tool. It has been in the last decade that significant investments have been made for acceptance and use of EMR system [4]. Implementation of an EMR system increases efficiency of health services, quality of care and user satisfaction. Successful implementation depends on multiple factors; one of the factors is users' response to utilization of EMR system [2]. Health Management Information system (HMIS) was strengthening in Ethiopia. As a result HMIS documentation package was implemented to standardize the patient documentation and reporting systems in all health facilities of the country. The implementation of this standardized documentation system was mandatory and a prerequisite to implement the EMR system to make sure that the paper workflow is in line with the EMR system. It leads to improvement in quality of healthcare, decreased time spent on paper work, increased patient satisfaction and financial saving [5,6].
Computerization of health care is an on-going measure. Even though considerable achievements have been made in the spread of information technology in the healthcare service, general clinical use of EMR system has not yet succeeded. The reasons for this lack of use are not clear but some possible explanations happen. These include computer skills, the implementation process, immediate benefit and the impact on time efficiency [3]. With an increasing fiscal limitation and a great demand by all stakeholders regarding return on investment, it is important to measure the success of EMR implementation [7]. However, in many developing countries the EMR system is not widely to the requesting clinic), eHMIS (to generate monthly, quarterly, and annual reports), and finance. Currently all but the financial module are implemented and used in the hospitals of this study. Installation of the network, server infrastructure, and the EMR system at all hospital sites was conducted by TUTAPE. After implementation, 5 day-long onsite user training sessions were provided to all health professionals of each hospital. Additionally, TUTAPE computer and network experts are accountable to provide continuous on-call service for technical assistance during system failure. On average, the Smart-Care system has been in use in the ARH of this study since 2012. In parallel, the paper-based medical record system is also still in use which means that the health professionals are expected to document both on paper and within the EMR system. The plan of the government is to expand the system to the other 127 existing hospitals in the country after the pilot testing. Additionally, the government is training health informatics professionals to support the health management information system and [10] implementation of EMR in the country (Table 1 and Figure 1).

Stakeholder analysis and description
Stakeholder analysis is a process of systematically gathering and analysing qualitative information to determine whose interests should be taken into account when developing and/or implementing a policy or program (Table 2).

Evaluation Methodology Study area
Ayder Referral Hospital (ARH) is found in Mekelle town capital city of Tigray, which is 783 km away from the capital city Addis Ababa at 2084 meters elevation above sea level. ARH started rendering its referral and specialized medical services in 2008 to the 8 million populations in its catchment areas of the Tigray, Afar and Southeastern parts of the Amhara Regional States. With the day-by-day rising patient flow that has more than 100,000 per year, and ARH has made outstanding successes in equipping itself with advanced medical equipment, supplies and facilities over the last three years. The study was conducted from March to May 2016 including all healthcare providers in ARH using the EMR system ( Figure 2).

Evaluation design
We used cross-sectional design with a formative evaluation approach to address the objective. For the evaluation we used 271 study participant healthcare providers and three focus group discussions. We have used self-administered questionnaire, FGD and observation methods of data collection (Table 3).

Availability
Majority of nurses 156 (95.7%) and all laboratory technicians had a computer with EMR installed in their office. From all type of respondents (91.8%) had computer with installed of EMR system in their working area ( Table 5).
Most of the respondents (82.7%) can input patient data easily in to computers using EMR system. From the type of the respondents spread or implemented. Most of published literature shows low acceptance and high failure rate for successful EMR implementations. The success of EMR can be evaluated along several axes including the dimension of quality, utilization and satisfaction. However, regardless of the outcome of the system satisfaction quality evaluations, if users are not satisfied with a system they may be unwilling to use it. Then user resistance has been suspected as primary factors in the failure of EMR implementation [8].
In Ethiopian hospitals and health centres implemented an EMR system was called Smart Care 5. Since the implementation of EMR system in ARH, There were so many problems hindering use of the system and successful of the system implementation. Such as training of users, utilization, and performance of usability and user satisfaction with EMR Implementation at the study area. In order to effectively scale up the EMR in future implementations, evidence on the main factors of EMR implementation success or failures must be studied. Therefore, to get the solution for the reasons of failure, not success and not use of the EMR effectively developed an evaluation to evaluate EMR implementation with the dimension of availability, utilization, performance of usability and user satisfaction with EMR implementation in ARH, Tigray Region, Ethiopia.

Stage of EMR development
In the 1960s, a physician named Lawrence L. Weed first described the concept of computerized or EMR which is a system to automate and rearrange patient medical records to improve their utilization and there by lead to improve patient care. Weeds work was a collaborative effort between physicians and information technology experts started in 1967 to develop an automated EMR system, and the objectives were to develop a system that would deliver timely and successive patient data to the physicians, and enable the quick collection of data for epidemiological studies, medical audits and business audits [9].
In 2009, the ministry of Health, with support of the Tulane University Technical Assistance project in Ethiopia (TUTAPE), in partnership with CDC and the Federal ministry of health Ethiopia (FMOH) started the development and implementation of Full EMR system for hospitals called Smart Care. Therefore, for the first time, the system was developed in 5 hospitals in Addis Ababa and other hospitals in regional cities 5. In the study area the system was deployed since 2012. In 2013, the Ministry of Health adapted the system as a national EMR for all hospitals, and planned to scale it up to further hospitals and regions [10,11].

Context of EMR systems
Smart-Care is a portable, integrated EMR system that is currently used by three African countries (Zambia, Ethiopia, and South Africa) [6]. Installation of the network, server infrastructure, of the EMR system at all hospital sites was conducted by TUTAPE. After implementation, onsite users training sessions were provided to all health professionals of each hospital. TUTAPE computer and network experts are accountable to provide continuous on-call service for technical assistance during system failure [10].
The main modules of Smart-Care include registration, outpatient department, inpatient (to admit, follow, and discharge patients in wards), tuberculosis, paediatrics, HIV/AIDS (to manage patients in antiretroviral therapy clinics), antenatal care, postpartum, pharmacy, drug stock control, laboratory (to store and send laboratory results      80% of physician, 100% of laboratory, 84% of Nurses and 69.2% of Radiology were input the patient data using the system easily. Only 9.8% of respondents had poor typing ability and the remaining (90.2%) were medium, good and excellent typing ability) ( Figure 3).

RESOURCES/INPUTs
The highest score of training received was laboratory technicians (66.7%). And the lowest was nurses (31%). Generally from the total Respondents only 37% had received training. 46% of the total respondents reported no support after training (Table 6).

Judgment for the dimension of availability
According the judgment criteria, the total score of the dimension of availability was 71.2%. The overall result would be judged as good ( Table 7).
The qualitative information was collected from individual suggestions, observations and focus group discussions. The goal of the analysis was to identify themes regarding Benefits, challenges or barriers hindering on-going EMR use and users satisfaction that were common among health professionals and the practice sites. These practices had moved beyond the implementation stage and were focused on the use of the EMR and users satisfaction.

Availability of computer
Nurse, said "there is no enough computers in each ward. There are 2 computers in the ward but the classes are 6 and more so there is a problem to use the system because of lack of computer" "Network problem and some time it stuck. Radiology reporting by smart-Care locked within a minute and we should log in again and again for every patient" this was because of lack of ability or skill of computer."

Typing ability
The typing ability was one of the components (factors) that hindered the EMR use. Therefore, the users give suggestions as follows: Physicians said "It needs writing skill those most of our staff have poor capability to writing. Not flexible especially in writing diagnosis. It does not include all diagnosis. Some Diagnosis is difficult to put on system". Proportion of satisfied due to Easy to use of EMR system (5) -  Themes of learning to use the EMR: If the learning or training was not sufficient, there is no effective on-going use of the system. Therefore the factors influencing the on-going use of the system due to learning was suggesting by the users as follows: Physicians said "Shortage of awareness how to use computer and Shortage of training. All diagnosis not include and we are not trained the computer system if the college arrange the training it will be ok".
Nurse said "There is no any training about how what the use of smart care for the new comers and there is no continuous assessment on EMR system needs strictly follow up".
The idea raised from one group that one nurse said "has sufficient computers in our unit but Have not good computer skill to typing and maintained if the computer does not work. Therefore, we need training about computer skill".
The other nurse said "there was no sufficient computer and training. Therefore, this hindered to use the system continuously and properly". Another challenges hindered to not use properly was electricity interruption and connection problems".

Use/utilization of EMR system
From total respondents (94.9%), used the EMR systems. More than 80% of them use the system all days of the week. All respondents of Physician, laboratory technician and radiologist had used the system 100%. But 150 (92%) of nurses were using the EMR system ( Table 8).

Usefulness of EMR
Overall result of usefulness of the system was shows 80% of the respondents were use the system to different medical tasks for more than half of the time. From table 10, 87.5% of the respondents used EMR systems to obtain the result from clinical laboratory, 86.3% used it to follow the result of particular test, 84.4% use the system to order clinical laboratory investigations, 84.2% used it to obtain the result from new test, 79.3% used it to obtain the information on investigation and 76.9% use it to seek out specific information (Table 9).
From one group one presenter said "the system was used for different purpose like save time, secured patient history, to manage resource (e.g., paper), easily communicate with health professionals on      line, minimized patient waiting time and other" The other respondents from ward said "the benefit of the system was mange bed management by following the admission and discharge patient, controlling our patient, follow the patient history".
Factors that hindered use of EMR system included: 1) Information technology challenges such as learning to use the EMR, electronic connectivity, and electric interruption. Two factors motivated ongoing use: 1) Improved efficiency in providing patient care, and 2) Confidence with computer typing ability and the EMR. As in many literature described there were so many benefits of the system. In this study the respondents gave their opinion or suggestion about the benefits of the system as follows: Nurses said "Patient problem solved. We get all investigation like CT, MRI and laboratory; it is important system if there is an electronic data record as some of patients chart is lost at triage so we can retrieve the lost document easily. To save time, receive cleared information, to keep any data for long time and to provide clear information".
Physicians said "it will save time if properly used and also it helps to patient to be satisfied, Comprehensive save time. If patient card lost it can save patients information, Patient problem solved. We get all investigation like CT, MRI and laboratory". "It is important to know the previous disease, treatment, job investigation and the action that was done. It is important to improve the work that you do".
Judgment for the dimension of usefulness: According the judgment criteria, the total score of utilization/usefulness was 87.6%. The overall result would be judged as v. good (Table 10).

Performance of usability of the EMR system to list of tasks list below
50.7% of the respondents were reported as the EMR systems use to review patient history ease and more ease. The rest respondents were reported as the system was not easy to use for this task. The majority of the respondents (79.2%) were reported that the EMR system was easy and more easy when use it to follow the result of particular test, 76.9% of the respondents use the system easy and more easy to order clinical laboratory, 81.5% of the respondents were reported as the system was easy and more easy when use it to obtain result of laboratory analysis.
The average performance of usability result shows 67.5% of the total respondents were use the system more easy and easier (Figure 4).  Performance of usability of the EMR system to list of tasks list below  Judgment for the dimension of performance of usability: According the judgment criteria, the total score of performance usability was 67.5%. The overall result would be judged as good (Table 11).

Users Satisfaction with context of the system
The satisfaction score of the system provide prcise, sufficient, clear information and with accuracy of the system of nurses was 76.7%, 69.80%, 71.80% and 69.20% respectively. Among physicians, 72% of the respondents agreed and strongly agreed with the system provide prsice information, 66.1% of the respondents agreed/strongly agreed with the system provide sufficient and clear information and lastly 63.2% of the respondents satisfied with the accuracy of the systyem. Generrally, 66.85% of the respondents were satisfied with the context of the system (Table 12).

Judgment for the dimension user's satisfaction with the context of the system:
According the judgment criteria, the total score of user's satisfaction with the context of the system was 66.8%. The overall result would be judged as good (Table 13).

Users Satisfaction with usefulness of the system
Most of the respondents on the study opinion were agree/strongly agree with usefulness of the system. 65% of respondents help them the system for more activities of their daily work, 63% of the respondents the system help them for more productive, 79.9% of respondents agree with the system was useful, 61.8% of the respondents were agreeing with save their time due to use of the system. Generally, 65.6% of the respondents were agreeing with satisfied by usefulness of the system. The detailed of user satisfaction with type respondents were present in Table 14.

Judgment for the dimension user's satisfaction with the usefulness of the system:
According the judgment criteria, the total score of the dimension of user's satisfaction with usefulness of the system was 65.6%. The overall result would be judged as good (Table 15).

Users Satisfaction with easy of learn
Cross tabulation of type respondents with user Satisfaction with ease of learn of EMR system, in Ayder referral hospital, Ethiopia, 2016 (Table 16).
Judgments for the dimension of user's satisfaction with ease of learn of the system: According the judgment criteria, the total score of user's satisfaction with ease to learn of the system was 62.6%. The overall result would be judged as good (Table 17).
Users satisfaction with ease of use of the EMR system 191(75.8%) of the respondents were satisfied or agree the system was ease to use, 181(71.9%) out of total respondents were satisfied or agree/strongly agree with the system is user friendly, 155(61.5%) agree and strongly agree with the system is flexible and 167(66.3%) were agree the system was successfully every time. Then the total satisfaction score with ease of use was 67.4% (Table 18). Judgment for the dimension user's satisfaction with ease of use the system: According the judgment criteria, the total score of user's satisfaction with ease to use of the system was 67.4%. The overall result would be judged as good (Table 19).
Overall user's satisfaction with EMR implementation: From all respondents 66% were satisfied with different indicators of the user's satisfaction dimension. But 37% of the respondents were not satisfied with the ease to learn of the system ( Figure 5).

Judgment for the dimension of overall user's satisfaction with EMR implementation
According the judgment criteria, the total score of the dimension of users satisfaction was 65.6%= 66%. The overall result would be judged as good (Table 20).

Overall EMR implementation effectiveness (Summary of dimensions):
The overall results of the EMR implementation effectiveness was 73.5%. From all dimension users satisfaction was low ( Figure 6).
According the judgment criteria the overall result of all dimensions means the effectiveness of the EMR implementation were 73.5%. Depend on the judging criteria this result was good. But as we see before some dimensions were good and others were poor (training, support and follow up after training). Therefore, it needs improvement on poor side (Table 21).

Discussion
From the total of respondents, 91.8% (234/255) have computer with installed EMR system in their office. In comparison with another study, only 20.4% of respondents had computers with EMR installed 5. The previous study in ARH only 57.5% of respondents had computer access 11. Therefore as mentioned above the availability of computer in the current study was better than previous studies. According to the judgment criteria this situation is very good ( Table 6).
In contrast, the current result shows (9.8%) of respondents have poor typing ability and (82.7%) of respondents inputting patient data easily but the result of other study shows (4.6%) of respondents have poor typing ability and (81.5%) of respondents inputting patient data easily. Then, current study was better in inputting patient data easily but lowers in typing ability 13. The availability of computer was good and some of users typing ability were good and some of them poor. Then if the typing ability of users were not good the usability and waiting time of patients may affect.
From all respondents, only (37%) of respondents were got training on EMR. As compare with previous study (35.5% of respondents) in the same area, the current results about training were better than the previous 11. In contrary, with the result of previous study on different study areas (64%, 49.6%) then current result was lower 5, 13. The availability of computers, inputting patient data, typing ability was very good. But as mentioned above, training is poor followed by fair support. And the overall judgment on availability dimension is good.

Indicators/variables Number of survey Observation (%) Weight (%) Result (%) Judgment
To review patient history 255 50. 7     The most probable reason for this variation may be that Ayder referral hospital is one of the pilot sites for EMR system for the Federal Minister of Health (FMOH) and TUTAPE. It thus the service started early and the users may also be informed well earlier than in the others studies area. The presence of higher number of computer represents in this study may be another reason to have better EMR system utilization. Participants' daily Usage frequency on EMR was (87.9%) and the remaining 12.1% used the system less than three days of the week. As compare with previous study conducted in the same area the usage frequency was 39.3% 11. As the current study described the usage frequency was better than previous study. The reason was due to improvement of user's attitude.
The usefulness of EMR system to handle patient health care via different modules and service providing units was 80%. Obtaining results from clinical laboratory which has the working process, ordering laboratory requests by privileged health professionals then the laboratory technologists took appropriate sample for ordered investigations then send results via the system to be viewed and support for medical decision makers in definitive diagnosis (87.5%).
Form investigation places or sites the health professionals like lab technologists can suggest new tests and investigations in order to strengthen findings (84.2%). Health professionals can follow the result of particular test and investigation which need to attend the progresses of changes made by interventions any time (86.3%). Patients could have lots of medical information recorded within different date and time in the database, so in this case doctors or any care providers could seek or retrieve specific medical information 76.9%. As compare with other study the usage of EMR system was (77.7%) of respondents use the system for seek out specific information and (80%) of the respondents was used the system for obtain result of test and investigations [13].
According the judgment criteria, the overall results of the dimension of use and usefulness of EMR system were 87.6%. And this was judged as V. good. This result was due to best infrastructure, enough manpower and good attitude of the users. The overall result of dimension of performance of Usability the EMR system was 67.5%. Then according the judgment criteria indicated, this result was judged as good. This result was with difficulties may due to lack of deep knowledge of the system and training.

Indicator of dimension
Type of respondents Strongly Disagree/ Disagree/Neither Strongly Agree/Agree The system provide precise information  One things that affected the use and users satisfaction was challenges and barriers during use of the system. Then would be observed the challenges like connection problem, electricity interruption, delayed for correcting of connection problem. In some area there was problem of typing ability and lack computer.
In this study, the respondents were satisfied with several aspects of the system such as the context of the system (provide precise, sufficient and clear information and its accuracy); usefulness of the system (help for more activities and daily work, useful and in terms of save time) and ease to use (ease to use, user friendly and flexible) are satisfied.
But users are not satisfied with ease to learn. As the result show most of the respondents were not satisfied with the training and even from the trained respondents 45.8% of them said the training was not sufficient. The overall satisfaction score was 66.9% of the respondents satisfied with the context of the system, 65.6% of respondents satisfied with the usefulness of the system, 62.6% respondents satisfied with ease to learn and 67.4% of respondents satisfied with ease to use. As the result of other study, 64% of physicians were considering ease to use the system [4,14]. Another study of our country described 64.4% of respondents dissatisfied with use of the system this means 35.6% of respondents were satisfied 5. Therefore as compared with that study the current study result shows (67.4%) of respondents were reported as satisfied with ease to use. Even score of overall satisfaction with the system overall was 65.6% but the score of other study was 40% [14]. The Overall satisfaction of the respondents with context of the system, usefulness, with ease to learn and with ease to use was 65.6%. Then result of current study was better as compare with other previous studies in our country. This was judged as good.
A study from Pakistan reported user satisfaction with EMR was 83% 14. Therefore, the current result was small than other studies in china and Pakistan. But it was better than the result of other study in our country. The reason for the current result was lower than the previous study in china and Pakistan were may be due to the users in current study has lack of full knowledge about the system and due to infrastructural development. The reason for users of the system more satisfied in the previous studies (china and Pakistan) was not similar with our country by adoption of new technology, infrastructure, and availability (accessibility) and they are developed country.
Overall results of all dimensions would get the final outcome of or objectives of the study. Then, the overall score of EMR implementation effectiveness with the main four dimensions was 73.5%. Therefore the final judgement of the overall study means the EMR implementation effectiveness was good. But as recommended below there was some an indicator or dimensions need improvement. User's satisfaction was lowest result than the other dimensions. Because the user satisfaction with ease to learn was low. The satisfaction with the other indicators    like with context of the system, with usefulness and with ease to use was good. This result indicates to work on building the users with good knowledge of the system.

Conclusion
The majority of the respondents have computer with EMR installed and they used the system daily. The training was low and most of the trained reported as the training and support after training was not sufficient. This could be due to short period of training and there was no refreshment training continuously. The majority of respondents have good typing ability and easily inputting patient data to the system. The overall EMR implementation effectiveness was measured by four dimensions of the evaluation which is rated as good. There is a need for training and resource provision for new staffs.