Participant characteristics
A total of 42 individuals from health systems and universities were interviewed for the study. Out of these participants, 9 were female, and their ages ranged from 22 to 45 years old. The majority of the participants, 35 out of 42 (83.3%), had more than 5 years of work experience. Almost 50% of the participants (21 out of 42) were from health centers, followed by 33.3% (14 out of 42) from the Woreda Health Office (WoHo) (Table 1).
Best practices and lessons learned from the model facilities
The study explored the best practices and lessons learned for enhancing the quality of data and its use in the CBMP implementation model woredas. The findings are presented using the Performance Routine Information System Management (PRISM) framework, which encompasses HIS interventions (technical, behavioral, and organizational) and HIS performance (data quality and data use) (26). Furthermore, during the presentation, a specific topic was discussed concerning sustainability and the factors related to challenges (Figure 1).
HIS interventions
This study examines how local CBMP universities designed and implemented interventions tailored to the unique needs and gaps identified in the baseline assessment. It also explores the collaboration between the Federal Ministry of Health (FMOH), Regional Health Bureaus (RHB), Zonal Health Department (ZHD), Woreda Health Office (WoHo), and local health facilities in planning and executing these interventions. Each intervention was customized based on the context and needs of the specific Woreda. The report categorizes the best practices and lessons learned from the interventions into three areas: Technical, Behavioral, and Organizational, using the PRISM framework.
Technical interventions
This study investigated the various technical support measures that the universities under the CBMP program provided to district health offices and healthcare facilities. Among the selected interventions were the supply of up-to-date HIS materials and manuals, the construction and renovation of HIS infrastructure, and the implementation of digitalization in the system.
According to the respondents, the CBMP universities have provided helpful manuals and guidelines to support the proper implementation of Health Information Systems (HIS). The participants also observed that the manuals covered topics such as data quality and information use, computer troubleshooting and maintenance, HMIS indicator definition, HMIS data recording and processing, National Classification of Disease (NCoD), and Information Revolution (IR) HIS self-assessment checklist. Both soft and hard copies of these manuals and guidelines were made available, and they have assisted facilities and health professionals in their daily HIS-related activities. HMIS focal person from health center stated: "Different guidelines and reference manuals were provided by the universities that support improving the technical skills and practice of health professionals." (KII, HC HMIS focal).
Respondents reported implementing various HIS infrastructure interventions through domestic resource mobilization, such as the renovation of MRUs and the establishment of separate HMIS offices. Hospital quality head stated: "… We prepared new standard shelves and expanded the number of services providing windows in the MRU. Currently, the number of windows increased from one to four and the clients are now served and hosted at different windows based on their service character." (KII, hospital quality head).
Digitalizing the services at the district and health facility levels was among the technical interventions to enhance HIS implementation. The provision of internet modems, Wi-Fi, and CDMA to all health facilities was identified by the study participants as a critical intervention to support HIS digitalization. Besides, CBMP implementation allowed the facilities to use smart care and DHIS 2. A director at CBMP Participant University described: "After providing the computer, we have provided them with internet access; It is needed to send the data online. For those facilities that didn't access internet service in that area, we provided them with CDMA." (KII, CBMP director).
Behavioural Interventions
Interventions targeting the behavior of health professionals included mentorship, peer-to-peer learning, and need-based capacity-building trainings were provided.
Based on the feedback received from the study participants, the CBMP institutions provided various short-term trainings that were relevant and helpful in enhancing the capacity and skills of healthcare professionals to carry out HIS operations with efficiency and effectiveness. This, in turn, improved their overall behavior. Furthermore, the participants noted that the training programs were mostly need-based, offered hands-on experience, and were conducted in well-organized training labs. A CBMP training and mentorship coordinator stated: “Training needs were based on need assessment during facility mentorship and recommendations from the woreda health office and facilities. Besides, health facilities are supposed to conduct HIS needs assessments every six month and submit to the woreda office”. (KII, CBMP mentorship and training coordinator).
The universities that are part of CBMP are creating and executing training programs to improve the skills of Healthcare Information System (HIS) professionals. These programs include both short-term and long-term trainings. As part of their efforts, the universities conducted a national study in 2018 to forecast the HIS human resource needs for the next 10 years. The report indicates that the country requires more than 50,000 HIS professionals(27). Therefore, universities recommended national HIS curricula harmonization. CBMP university DDCF coordinator mentioned: "...With the national HIS HR forecasting result, national HIS program curriculum harmonization and eight HIS program supporting modules were prepared. Besides, with the support of DUP, more than 4000 copies of HIS supporting modules were distributed to the ten universities and colleges that provide the HI program…." (KII, University DDCF coordinator).
Onsite peer-to-peer learning and orientation is one of the ways to improve the implementation of HIS activities. Professionals who received training at CBMP universities implement this method by informing all staff members about the training they received upon returning to their facility. This way, they can effectively transfer the knowledge and skills they have acquired. A hospital quality head stated: “… the hospital established a system that every trained staff should provide training orientation about the training they obtained to staffs who didn’t get through a peer-to-peer orientation.” (KII, Hospital quality head).
Peer-to-peer mentorship was implemented to improve HIS activity implementation. Almost all participants responded that mentorship has been provided. Both the University of Gondar and Hawassa University stated that they used well-trained mentors. They engaged participants from the RHB and woreda health offices during the mentorship, which was conducted quarterly. A respondent stated: "… We are conducting mentorship with the lead of university mentors in collaboration with participants from the RHB and woreda health office. “(KII, CBMP director).
Despite universities declare mentorship have a significant impact, their approaches differed. Hawassa university participant mentioned that, they follow an approach Focus, Gap analysis and develop an action plan – Execute, Cascade and Reward (FGD-ECR). FGD-ECR approach focused and invested to a selected one facility to make it learning and demonstration site for other. A respondent from Hawassa University explained:
"…We have observed that the checklist-based mentorship didn't bring any changes, so we followed an approach for our mentors to take time with the health professionals and coach them on how it is done while on the job. This approach significantly improves HIS at the health facilities within two weeks." (KII, CBMP training coordinator).
The university of Gondar respondent stated that the beginning of the mentorship intervention, there were challenges related to consistency and a shared understanding of the mentorship process. After two rounds of the mentorship, a workshop was organized to help mentors develop a common understanding of the process. During the workshop, inconsistencies in the understanding of the mentorship checklist and process were identified. A university DDCF coordinator stated:
"… After two rounds of mentorship, we prepare a workshop to assess the practice of mentorship, and we have identified inconsistencies in understanding the mentorship checklist and mentorship approach. We discussed the mentorship checklist and approach with the mentors, and after that, we had good mentorship practice…." (KII, University DDCF coordinator).
A participant from the University of Gondar has shared that they use four different models for providing mentorship. These models are University lead, Woreda lead, cluster, and remote/virtual mentorship. The university takes the lead in providing mentorship, and RHB, ZHD, and WoHo work together to implement the university lead mentorship. Similarly, the woreda lead mentorship approach is where the woreda health office conducts mentorship independently. The cluster mentorship approach involves selecting staff from the top-performing health facility, who then mentor the other health facilities assigned in their cluster. Additionally, virtual/remote mentorship is a useful approach to help solve problems facing the health facilities and woreda health offices remotely. This type of mentorship includes attending PMT meetings through conference calls using the mobile phones of university mentors. CBMP training and mentorship coordinator stated:
"During the university lead mentorship, members are from the RHB, ZHD, woreda health office, and University. But, the woreda lead mentorship is led by the woreda health office independently. In addition, virtual mentorship is conducted by the university mentors by telegram and phone call. The cluster mentorship is by the facilities in the woreda and managed by the woreda health office" (KII, CBMP training and mentorship coordinator).
Organizational interventions
According to the study, various interventions related to the organization such as collaboration, leadership engagement, local resource mobilization, HIS governance, HMIS task audit, evidence generation, experience sharing, and reward and recognition were implemented to improve the activities related to Health Information System (HIS). The participants in the study highlighted that the district health offices, RHB, and CBMP universities collaborated closely to implement most of the HIS activities.
Supportive supervision, joint planning, joint curricula development, mentorship, review meetings and training are part of the collaborative implementation strategy. A study participant from regional health bureau acknowledged the collaboration as:
"As RHB, we collaborated with the University of Gondar in several activities that include baseline assessment, mentorship, and supportive supervision with prior communication.” (KII, RHB PPD).
One of the interventions to improve health information systems (HIS) is to provide computers for local use. Universities have been successful in mobilizing computers and supplying them to Regional Health Bureaus (RHB), district health offices, and local health facilities. This organizational intervention has helped enhance HIS activities at the local level. A study participant from a regional health bureau indicated:
"… The availability of computers was assessed at the health facilities. And provided for those facilities that don’t have it as one intervention package considering it as a basic requirement to generate a report and use data." (KII, RHB PPD).
A study participant from a health facility is also witnessed the recipient of computer and printer based on mentorship gap identification:
"We received two computers from Hawassa University that we did not have before. Currently, we are using DHIS2, which we previously needed to go to submit reports with a hard copy." (KII, HC HMIS focal).
This study examines how leadership engagement can improve Health Information System (HIS) activities within an organization. The research found that involving leaders in the early stages of project implementation is not easy, but leadership engagement increases over time with collaborative implementation science research, regular mentorship, and targeted meetings. Participants in the study reported that leadership engagement has a positive impact on HIS activities. A study participant from Hawassa University stated:
"At the early stage of project implementation communications were mainly with only the facilities which lacks leadership engagement. Later on, we realized that we need leadership decisions. You cannot simply convince them allocate HIS budget for the HC without engagement..." (KII, RCSD).
According to respondents, there is no established HIS governance and accountability framework that is implemented at the national, regional, and local levels. As a result, they created their own accountability frameworks for facility management at the local level to address this issue. The framework enabled them to monitor and manage HIS activities closely and also helped them to tackle challenges related to behavior and attitude. A respondent from a health facility stated:
"Our HIS governance strategy was developed by the facility management and disseminated to the staff. All staff are oriented and informed to implement the HIS activities as their routine activity and accountability will be ensured based on the stated framework." (KII, HC head).
Experience sharing, recognition and reward, and joint evidence generation and utilization were also organizational factors that enhanced HIS activity, taken as best practices and lessons in the study areas.
A participant explained how experience sharing, making things easier, clearing doubts, and inspiring health professionals to move for action:
"…We provided three days of training to the staff at MRU in the woreda. However, when we visited the facilities after the training, we noticed that some of them were not performing as expected. To address this issue, we decided to assign vehicles and per diem to take them to Wojel Health Center. There, they could observe how their peers were doing at their MRU, which had been renovated. After a full day of experience sharing, they returned with a radical change in their approach to their work. This experience sharing played a significant role in improving their performance." (KII, WrHo head)
CBMP universities and implementers used rewards and recognition as motivation strategies. Some of the rewards and recognition include certificates, flash discs, computers, mobile phones, scholarships, and tea making materials for best-performer individuals and case teams. A Woreda Health Office respondent stated:
"… The top performer is awarded a cup, while others receive certificates. If a health extension worker has been better at connecting TB suspects than others, she will receive 100 birr mobile airtime." (KII, WrHO head).
Both universities involved in the CBMP program collaborated with stakeholders to conduct an implementation science research aimed at improving Health Information Systems (HIS). According to respondents from the University of Gondar, the implementation research was based on the findings of operational research conducted at the local health facility, and the results of the research were published (28).
His performance improvement
The performance improvement of the model woredas was measured by comparing the quality of data and information use before and after the implementation of CBMP. This study revealed that prior to CBMP implementation, many facilities did not conduct Lot Quality Assurance Sampling (LQAS), and the quality of data was not checked before it was sent to the next level. A study participant from CBMP University stated:
"… None of the health facilities used data quality assurance techniques, and none were conducting LQAS for data quality check-ups." (KII, CBMP Training and Mentorship coordinator)
This study found that prior to the implementation of the CBMP, patient data was incomplete, there was no central MRU, and smart care was not available. However, these issues were addressed and improved after the implementation of the interventions. Improving the quality of data was a challenging task that required a gradual and collaborative effort. Study participant from CBMP University mentioned that:
"….We follows the learning by doing principle. We don't order them to do it; rather, we work the LQAS together." (KII, RCSD).
In addition to producing quality health data, the assessment includes the use of PMT functionality, data dissemination, visualization, strategic problem-solving approach, and implementation of quality improvement projects.
Before the CBMP implementation, most study participants reported that the PMTs were not fully functional. However, the functionality gradually evolved after the implementation. A respondent from health centre stated, currently they are even having PMT at case team level because of the intervention.
"….Now, they even evaluate at their case team level. I now get evaluated case team reports; for example, the MCH case team evaluates before submitting to HMIS; they even do LQAS and submit it to me. That is a new experience in PMT activity" (KI HC HMIS focal).
CBMP universities have helped health facilities and woreda health offices in disseminating core data elements to the stakeholders to enhance evidence-based decision-making. They have provided training on how to prepare data dissemination tools such as brochures, banners, leaflets, and bulletin, and how to use the DHIS2 dashboard. Additionally, they have advised the facilities to use local and national languages to disseminate information to create awareness about the health services and available data. A study participant from health centre stated:
"By the brochure we prepare, we created awareness in the community about our facility services and performances. Especially during our CBHI promotion campaign, discussing the information we disseminate with the community helps us a lot….." (KII, HC HMIS_Focal).
A strategic problem-solving approach was implemented in the model woredas to enhance the use of data for decision-making. Health Centre Head study participant stated:
"….we went to the community and cleaned stagnant water and reduced the case. That is how we are using the data; it is not just politics; rather, we are identifying problems and taking actions" (KII, HC_Head).
In addition, the model woredas provided valuable insights and best practices for quality improvement projects (QIP). The QIPs are implemented collaboratively by the facility's quality improvement team and the PMT. The PMT is responsible for identifying gaps and sharing them with the quality team. Together, they identify the root causes and design appropriate strategies to address them. Participant from Quality team stated:
"We designed this intervention after we prioritized the problems and selected the most important one with the quality team" (KII, Hospital Quality Head).
The current study also identified factors that require strengthening to make the HIS change sustainable and other factors considered as a challenge for the existing best practices and lessons learned stated below.
Sustainability
The study participants stated that some of the above findings, namely institutional capacity building, HIS governance framework, leadership engagement, and creating data ownership, can be used as a sustainability strategy. One of the study participants stated that they implemented HIS governance frameworks to monitor and manage behavioral-related challenges. Moreover, participants mentioned that they adapted this monitoring system, which helped them to sustain the HIS performance.
"…We have prepared the HIS governance accountability framework by the management to follow the HIS activities, and as a facility, we have a weekly internal supervision with HMIS task audit platform which checklist based." (KII, PHCU Director).
Improved leadership engagement and data ownership can enhance the sustainability of HIS activities. Additionally, participants found CBMP interventions to be highly effective and recommended their implementation in other areas. Participant from Health Centre mentioned:
"…. The CBMP program interventions helped us improve our staff's technical skill and increase our value and attitude towards health data. It is good to reach these interventions to other areas even." (KII, HC head).
In addition, the study identified some challenges that should be taken into account when expanding to other regions. These challenges include frequent unplanned campaigns, limited involvement from consortium universities, insufficient computer resources, high staff turnover, dissatisfaction among HIS staff regarding career development and salary. These factors may pose challenges to the long-term sustainability of the current HIS improvement efforts and its ability to be scaled up to similar settings. Participant from RHB stated:
"…as you might have heard, HIT staffs felt discriminated and disappointed after JEG was implemented…It is a paradox that how can you take this facility to digitalization …without treating HIT equally with other staffs…He added that HIT professionals are changing their field of study because of lack of vertical growth. They are changing their profession to clinical field streams" (KII, RHB plan & program officer)
Respondents also described how frequent and unplanned campaigns affect their HIS activity. A respondent from WoHo stated:
"There are also different campaigns and emerging priorities that they (PMT members) have to be engaged in…becomes difficult for them to cascade other activities in parallel." (KII, WoHo Head).
According to the respondents, one of the major challenges in implementing Health Information System (HIS) activities, such as Electronic Medical Records (EMR), is the poor competency of MRU staff. They noted that most staff members of the MRU are working based on their experience or after being promoted from their previous roles, such as security guards or housekeepers. Additionally, the respondents pointed out that the attitude towards the role of MRU staff as doable by anyone assigned is a significant issue that affects the implementation of HIS activities.
"…. the staffs working in the unit were promoted from their previous housekeeping and security roles, and since they have been working from experience, it was challenging and energy-consuming (KII, WrHo M&E officer).
A mixed study design identified low-level ownership as the main challenge. Poor facility responsiveness to feedback and misinterpreting CBMP as a research agenda rather than a partnership were also mentioned as challenges (29).