Evaluation of the electronic health record system in maternal and child health centers of Marie Stopes International Ethiopia [version 1; peer review: 2 not approved]

Background: The Health Information Management Systems Society (HIMSS) defines an Electronic Health Record (EHR) as a longitudinal electronic record of patient health information generated by one or more encounters in any care delivery setting”. EHR systems improve quality and adaptability of care, contributing to patient safety. The system is also efficient and provides real-time administration to patients. Driven by the need to facilitate clinical and administrative process, and to reduce medical errors, Marie Stopes international Ethiopia (MSIE) implemented EHR system in its maternal and child health (MCH) centers. This study sought to evaluate the EHR system in six departments (inpatient, outpatient, pharmacy, laboratory, reception cashiers and MCH managers) of these MCH centers in terms of ease of performing tasks and satisfaction with the system. Methods: A cross-sectional method with formative evaluation was used to address the objective. A sample size of 54 were used for the investigation. The participants were from six departments (inpatient, outpatient, pharmacy, laboratory, reception cashiers and MCH managers) within three MCH centers. Data was collected using structured, self-administered questionnaires and interviews and analyzed using SPSS 20. Results: In total, 83% of respondents found that performing tasks using EHR made tasks easier; the lowest score was from Pharmacy department (48%). The overall satisfaction with EHR was 87%. The lowest


Introduction
The Health Information Management Systems Society's (HIMSS) define EHR as: "The Electronic Health Record (EHR) is a longitudinal electronic record of patient health information generated by one or more encounters in any care delivery setting.Included in this information, are patient demographics, progress notes, problems, medications, vital signs, past medical history, immunizations, laboratory data and radiology reports" (Ahmed et al., 2013).
Recording of patient data on paper obstructs the coherence and nature of medical care offered to patients.Furthermore, paperbased frameworks have restricted usefulness, limiting reviewing of medical records by care givers simultaneously.Storage of paper records takes up more space and hinders easy access to patient information (Carayon et al., 2009).By contrast, EHR can create a complete patient record of the entire clinical experience.The system automates the clinical work process and improves quality of care (Ahmed et al., 2013).Patient records can also be easily retrieved using the patient's name, phone number or registry number.
Currently, paper-based recordings are being migrated to electronic recording systems because electronic health recording (EHR) systems improves quality and adaptability of care recording, contributing to patient safety.The system is also efficient and provides real-time administration to patients (Rezaeibagha et al., 2015).Therefore, using HERs is an essential part in health care delivery to improve quality and safety (Ellsworth et al., 2017).The US Institute of Medicine has characterized EHRs as an essential technology for improving the safety, quality and efficiency of health care (Chiang et al., 2013).
An assessment of nine hospitals that implemented a comprehensive electronic health record (EHR) system discovered an improvement on the speed and accuracy of communication leading to less duplicative tests, quicker responses to client's needs, and a complete capture of charges (Silow-Carroll et al., 2012).
The averted costs associated with efficiencies created by maintaining and availing patient data electronically contributes to the financial benefits of the EHR system.Some of the averted costs are related to reduced staff number required in patient management, reduced stationary and printing costs required to maintain paper records (Menachemi & Collum, 2011).
On the other hand, a study done in a residency's family medicine center in Birmingham, AL, USA, revealed that Physicians and residents are very dissatisfied with the amount of time required for documentation using the EHR system (Bloom & Huntington, 2010).Following the implementation of EHR, both clinical staff and office staff spent more time on transcribing data to computer to maintain medical information system (Carayon et al., 2009).
In this study we will evaluate the EHR system implemented in three of the MCH centers in Marie stopes international Ethiopia (MSIE).The study will evaluate ease of performing tasks and satisfaction with the system in six departments (inpatient, outpatient, pharmacy, laboratory, reception cashiers and MCH managers).Each department performs a unique task that contributes to the overall performance of the facility.Thus, we hypothesized that the ease of performing tasks and satisfaction with the EHR system varies by department and by specific task within a department.

Study approach
We used a cross-sectional study with formative evaluation method to address the objective.At the time of the investigation there were 134 staff members from three of the MCH centers using the EHR system in their daily activities.Of the 134 staff members, only 68 fulfilled the selection criteria of working 6 months in the MCH center prior to the implementation of the system.From the selected 68 staff members, only those trained by MISE about the system were selected as final study participants.Therefore, a total sample size of 54 participants were used for the investigation.The participants were from six departments (inpatient, outpatient, pharmacy, laboratory, reception cashiers and MCH managers) within three MCH centers.Data was collected using structured, self-administered questionnaires and interviews (available as Extended data (Sharew, 2019)).The interviews were conducted with two people from each department to find out their daily tasks.Then a unique set of task list was prepared for each department.Using these task lists, a different questionnaire for each of the six departments were developed.The questionnaires involved a section where the respondents answer by ticking and a section where respondents are asked to answer by writing.

Data analysis
Data were then checked for completeness, consistency and analyzed using SPSS for Windows v20.The usefulness dimension was measured by how the participants found the system affecting their day to day activities and operations.The user satisfactiondimension was measured for relevant aspects.Each dimension and their indicators were judged using a judgement criterion: >95% excellent, 80-94% very good, 65-79% good, 50-64% fair, <50% poor.Descriptive statistics tables and graphs will be used to describe the findings.

Ethical approval
The study was conducted after ethical clearance was obtained from the Institutional Ethical Review Committee of Addis Ababa University School of Commerce.During the data collection period, the participants were well informed about the purpose of the study.Written informed consent of the respondents was obtained before the commencement of the study.Involvement was fully decided by the participants.The response was confidential.Monitoring and supervision of data collectors was done throughout the whole data collection period.To assess the completeness, clarity and consistency the collected data were checked daily after data collection.

Results and discussion
Demographic data of respondents Of the total 54 respondents, 17 (31%) were from the inpatient department and 18 (33%) were from the outpatient department.Of all the participants, 31 (57%) worked between 4 and 5 years with the organization.Of the respondents, 14 (26%) were nurses and 11 (20%) were midwives (Table 1).Results from all surveys and questionnaires are available as Underlying data (Sharew, 2019).
Ease of performing tasks using EHR Inpatient department Most participants from the inpatient department agree that the EHR system made their daily tasks easier.Admitting patients, following admitted patients' medical usage and monitoring consumables during procedure were indicated by all the participants from the department as activities made easier following EHR implantation (Table 2).Similarly, a study done in a small family practice clinic in USA also indicated an increase in time spent on patient care and decrease in time spent in meeting and performing lab works (Carayon et al., 2009).

Outpatient department
In total, 94% of those in the outpatient department felt EHR made performing tasks easier (Table 3).All respondents agree that the system made it easier to seek out specific information from the client record, to produce data reviews for specific patient groups, to order clinical laboratory, to order and receive ultrasound results and to write prescription.However, only 61% of the respondents agree that the system made it easier to complete sick-leave forms.

Pharmacy department
The result showed that the system made it easier to know available drugs in the dispensing area and billing for medications.However, the respondents indicated that the system made the it difficult to rule out drug interactions and maintain allergy and active medication lists (Table 4).The study revealed that the need and the requirements of pharmacy department had not been reflected in the system design.The agency of healthcare research and quality to support research policy information in the area of EHR indicated that, if effectively implemented, EHR system reduces the need to rely on memory alone for information required to complete a task such as medical history, allergies and formulations (Armijo et al., 2009).

Laboratory department
The overall ease of performing tasks using EHR in laboratory department was 82%.From the list of activities performed in laboratory department, only half of the agree that the system make processing lab packages by category of services easier (Table 5).

Reception cashiers
All the respondents, reception cashiers, showed that the system made their daily task easier (Table 6).

MCH managers
MCH managers found the system easy to use when performing most of their day to day activities.However, 67% of the respondents  To follow admitted clients Medication usage 0% 100% To monitor consumables during procedure 0% 100% Billing for consumables 6% 94% To follow 1 st stage labor care 12% 88% To follow 2 nd stage labor care 12% 88% To monitor labor using partograph 12% 88% To follow post-delivery status 6% 94% To monitor OR outcome 6% 94% To monitor Neonate outcome 24% 76% Overall ease of performing tasks using EHR/Inpatient 92% Table 3. Ease of performing tasks using EHR in the outpatient department.
How has the EHR system changed the performance of the following task?
More difficult/slightly more difficult/No change

Slightly easier/easier/ significantly easier
To review the client's problems has become 6% 95% To seek out specific information from the client record 0% 100% To follow the results of a test or investigation has become 6% 94% To enter daily notes has become 6% 94% To answer questions concerning general medical knowledge (e.g.Concerning treatment, symptoms, complications etc.) has become 11% 89% To produce data reviews for specific patient groups (EG. Complication rate) has become 0% 100% To order clinical laboratory analysis has become 0% 100% To obtain the results from clinical laboratory analysis has become 6% 94% To order ultrasound investigation has become 0% 100% To obtain results from ultrasound has become 0% 100% To refer the patients to other departments or specialists has become 11% 89% To order treatments directly (E.g.medicines, operations etc.,) has become 6% 94% To write prescription has become 0% 100% To complete sick-leave forms has become 39% 61% To give written individual information to patients (E.g.About medications, disease status) 6% 94% Over all ease of performing tasks using EHR/Outpatient 94% indicated that the system makes it difficult to do income to cost ration (Table 7).Similarly, a survey done in Australian hospitals revealed that a majority of managers (82%) stated that EHR improved data quality by readily availing information and improving the legibility of records (Mckenzie, 2003).

User satisfaction
When participants asked to rank their satisfaction with various aspects of use, total reported satisfaction with the EHR system (agree/strongly agree) was 87% (Figure 1).With the work they do on EHR system, inpatient staff were least satisfied Table 4. Ease of performing tasks using EHR in the pharmacy department.
How has the EHR system changed the performance of the following tasks?
More difficult/slightly more difficult/no change

Slightly easier/easier/ significantly easier
To clearly read prescription 40% 60% To dispense medications 40% 60% To dispense multiple drug from the prescription 60% 40% To know available drugs in dispensing area 0% 100% Monitor medications by expiry date 80% 20% Serving clients in order 20% 80% Billing for medication 0% 100% Implement drug-drug and drug-allergy interaction checks 100% 0% To maintain an active medications allergy list 100% 0% To maintain an active medications list 100% 0% To detect duplication of therapies 20% 80% To monitor controlled drug prescription 60% 40% Over all ease of performing tasks using EHR/Pharmacy 48% Table 6.Ease of performing tasks using EHR for reception cashiers.
How has the EHR system changed the performance of the following tasks?
More difficult/slightly difficult/no change

Significantly easier/ easier/slightly easier
To register patient/client information -100% Finding mistakes and missing information -100% Billing and change capture -100% Searching for patient file -100% Finding the exact client document searching for -100% Over all ease of performing tasks using EHR/Reception cashiers 100% Table 5. Ease of performing tasks using EHR in the laboratory department.
How has the EHR system changed the performance of the following tasks?Over all ease of performing tasks using EHR/Laboratory 82% Table 7. Ease of performing tasks using EHR system for MCH managers.
How has the EHR system changed the performance of the following tasks?
More difficult/Slightly difficult/no change

Significantly easier/ Easier/Slightly easier
Getting total number of clients as needed 0% 100% Getting provider level performance 0% 100% Getting number of clients/services 0% 100% Producing data reviews for specific patient groups (e.g.complication rate, diagnosis) 0% 100% Total cash collected daily/weekly and monthly 0% 100% To calculate income to cost ration 67% 33% To show service trend 0% 100% Providing remote access to data 33% 67% Providing live data for presentations 33% 67% Controlling fraud 0% 100% Over all ease of performing tasks using EHR/MCH Manager 81% Figure 1.Satisfaction reported by staff in each departments.

Limitations
Limited literatures were available to compare and discuss the findings with, and social desirability bias.

Conclusion and recommendations
Staff in all department reported the EHR system ease of use as 'very good' for ease of performing their daily tasks using EHR system.The lowest score for ease of performing tasks was from pharmacy department and highest score was from reception cashiers.The reported satisfaction with the system was also high.In addition, the study revealed that ease of performing tasks using EHR and satisfaction with the system varied by department and by task within a department.It is essential to know the needs and requirements of each department before implementation of the system and getting user feedback for long lasting uptake and impact.It is also imperative that the workflow and information needs of each unit are met to optimize system utilization resulting quality of care.
This project contains the following underlying data:

Salla Atkins
New Social Research and Faculty of Social Sciences, Tampere University, Tampere, Finland Gizaw et al.'s study focused on an evaluation of the electronic health record system in maternal and child health centers of Marie Stopes International Ethiopia.Overall, electronic health records have several benefits over paper-based records, and are an essential part of safe, quality healthcare.I found this paper to be highly descriptive and would recommend further analysis and consideration of their findings.My more detailed comments are below: The literature used in the article is rather dated, with fewer recent papers.The authors would benefit from adding more recent literature.For example there are several systematic reviews that the authors could include, and consider in their presentation of the paper. 1.
The study design is a cross-sectional evaluation study, with interviews and surveys.Despite using mixed methods, the paper does not refer to the implications or methods of using mixed methods.I suggest the authors use SURGE -The Survey reporting guideline, to report their findings, and add detail of the interviews conducted 1 .

2.
Sufficient detail is given of the analysis of the survey responses, but no information is given of the analysis of the 6 interviews that were part of this study.

3.
In terms of statistical analysis and interpretation, again this refers to the descriptive nature of the study.Having three different centres is not considered in the analysis or results at all.I suggest the authors re-examine their data for patterns across the different MCH, and 4.
patterns across the interviews and self-administered questionnaires.
The limitations section is extremely limited, and does not discuss e.g. the potential limitation of self report, response rate of the survey (was it 100%)?and the limited sample size.

References
I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above.

Taiwan
The authors attempted to evaluate the EHR system in six departments of the MCH centres in terms of ease of performing tasks and satisfaction with the system.I have a few concerns with the methodology and results presented.What new information this study is contributing to the research society?Already there are tons of research done on exactly the same topic. 1.
Inclusion criteria are not clear, why authors need 6-month experience prior to the implementation of the system?I think they should include a participant with 6-months experience with EHR. 2.
English has to be revised by the native speaker, many sentences are not clear.3.
The percentage of the respondents shown in table 1 is more than 100.4.
Citations are too few and very old, I strongly recommend authors to update the citations with the latest studies.I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above.

5 .
Is the work clearly and accurately presented and does it cite the current literature?NoIs the study design appropriate and is the work technically sound?PartlyAre sufficient details of methods and analysis provided to allow replication by others?PartlyIf applicable, is the statistical analysis and its interpretation appropriate?I cannot comment.A qualified statistician is required.Are all the source data underlying the results available to ensure full reproducibility?YesAre the conclusions drawn adequately supported by the results?PartlyCompeting Interests: No competing interests were disclosed.Reviewer Expertise: AI, Big data analytics, EHR.

Open Peer Review Current Peer Review Status: Version 1
This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Another challenge of the paper is the highly descriptive nature of the paper, where results are reported by percentages in six departments in three different MCH centres.This, with 68 participants means that we don't get a picture of what might have differed between different MCH centres, and we don't know what data came from where.Overall, the results are presented by different departments, not taking into account different MCH centres in the study, and with no comparison across the settings.

Is the work clearly and accurately presented and does it cite the current literature? Partly Is the study design appropriate and is the work technically sound? Partly Are sufficient details of methods and analysis provided to allow replication by others? Partly If applicable, is the statistical analysis and its interpretation appropriate? Partly Are all the source data underlying the results available to ensure full reproducibility? Yes Are the conclusions drawn adequately supported by the results? Yes Competing Interests
1. Grimshaw J: SURGE (The SUrvey Reporting GuidelinE).2014.206-213 Publisher Full Text : No competing interests were disclosed.