Evaluation of the adverse events following immunization surveillance system in Guruve district, Mashonaland Central 2017

Introduction An adverse event following immunisation is any untoward medical occurrence which follows vaccination. Frequency of adverse events ranges from 13% to 34% and they should be reported regardless of severity. From the beginning of 2016 to mid-2017, Guruve district in Zimbabwe did not report any AEFIs. This suggests the surveillance system may be failing to detect adverse events. We therefore evaluated the AEFI surveillance system in Guruve district. Methods We conducted a surveillance system evaluation using the updated Centers for Disease Control guidelines for evaluating public health surveillance systems. We interviewed health workers and caregivers of babies under 2 years in Guruve district. We also reviewed all records on AEFI surveillance for the period of January 2016 to November 2017. Results We recruited 31 health workers and 33 caregivers into the study. Between January 2016 and mid-2017, 39% of the caregivers had children who had suffered AEFIs and 45% of the health workers had encountered AEFIs but none had been notified. The main reasons for failure to report AEFIs included health workers' fear of personal consequences and caregivers thinking that an adverse event was not serious enough to report. Knowledge of the surveillance system was good amongst the majority of health workers. All the resources needed by the surveillance system were available. Conclusion We concluded that health workers in Guruve district were afraid to report adverse events following immunization and caregivers were reluctant to report mild adverse events hence the surveillance system was performing poorly and was not useful. However, the stability of the system and the good knowledge gives a good foundation for improving the surveillance system.


Introduction
An adverse event following immunisation (AEFI) is any untoward medical occurrence which follows vaccination and does not necessarily have a causal relationship with the usage of the vaccine [1]. Adverse events range from mild to severe, and the mild events include fever, pain at injection site and local swelling. Severe reactions can include convulsions, coma and even death [2]. The majority of adverse events following immunisation are mild and resolve quickly, but one cannot predict individuals who might have a mild or serious reaction to a vaccine [3]. AEFIs may be true adverse reactions that are intrinsic to the vaccine, caused by the way it is administered, related to an underlying condition in the recipient, or coincidental. The World Health Organization (WHO) classifies AEFIs into four main categories which are programme related, vaccineinduced, coincidental and unknown [1]. A clinical trial conducted by Decker et al. in the USA showed that mild adverse events such as fever occurred in more than 35% of children who were vaccinated [4]. In 2011, WHO came up with the global vaccine safety initiative (GVSI) whose primary objective is AEFI detection. The GVSI aims to ensure that monitoring of vaccine safety takes place even in low resource settings [1]. The Global Advisory Committee on Vaccine Safety (GACVS) proposed a reporting rate of at least 10 severe AEFIs per 100,000 surviving infants and this is the vaccine safety indicator which was adopted by WHO [5]. There is no indicator for minor AEFIs.
In the WHO African region, only half of the countries (i.e. 51%) reported AEFIs in 2015 and Africa had the second lowest case reporting rate of 74 per 100,000 surviving infants while the Eastern Mediterranean Region had the highest rate of 2,740 per 100,000 surviving infants [6]. A study conducted in Mali and Gambia showed that in children who received the meningococcal A vaccine 13.4%-34.3% developed minor adverse events [7]. According to the WHO, all adverse events that are of concern to the caregiver should be reported regardless of severity [8]. Allegations that vaccination causes adverse events must be dealt with rapidly and effectively.
Failure to do so can undermine confidence in a vaccine and ultimately have dramatic consequences for immunisation coverage [1]. Zimbabwe introduced an AEFI surveillance system in 2000 which is based on the WHO guidelines [9]. The system is meant to identify and correct programme errors and distinguish coincidental events from true AEFIs. Mashonaland Central was expected to report at least six severe AEFIs from the beginning of 2016 to mid-2017 based on its infant survival rate; they only managed two. From the beginning of 2016 to mid-2017, Guruve district in Mashonaland Central did not report any AEFIs. This lack of reports suggests that the AEFI surveillance system may be failing to detect adverse events. Failure to detect adverse events that occur in the community may result in loss of confidence in the safety of vaccines [10,11]. There is no published evidence of previous evaluation of this surveillance system; it is for this reason that we evaluated the AEFI surveillance system in Guruve district to identify reasons for its poor performance and make recommendations to improve it.

Flow of information:
Guruve district uses a passive surveillance system where caregivers detect adverse events in their children in the community. The caregivers then report to the clinic where the nurse confirms the AEFI, manages and reports the AEFI to the district level within 24 hours. When an AEFI is reported, five notification forms are completed at the clinic, four are sent to the district, and one is filed at the clinic. When the district is notified they facilitate case management and investigation of the AEFI. They also notify the provincial level within 24 hours and send three of the notification forms to them. The fourth form is filed at the district level. The province supports investigations of serious AEFIs, and they notify the national level and Medicines Control Authority of Zimbabwe (MCAZ). The national level reviews AEFI case reports and gives regular feedback to lower levels. The MCAZ reviews all AEFIs at the national committee meeting and provides feedback to the reporter through the same channels ( Figure 1).

Study design:
We conducted a surveillance system evaluation using the updated Centers for Disease Control (CDC) guidelines for evaluating public health surveillance systems in November 2017.  Sampling: Guruve District has 21 health facilities, one district hospital and 20 clinics. We purposively selected the district hospital into the study since it is the largest health facility in the district serving the largest population and 10 out of the 20 clinics were randomly selected into the study using a random number generator (Random lite TM version 1.2). The clinics were assigned numbers 1 to 20 in alphabetical order, and a random number was generated with replacement until the sample size was satisfied. At each clinic nurses who were found to be on duty on the day of data collection were recruited into the study, at the hospital six nurses (i.e., three from the outpatient's department and children's ward respectively) were randomly selected from those found on duty on the day of data collection. Caregivers were recruited from the same facilities, a maximum of three from each facility were randomly selected amongst those who had visited the facility for immunisation services on the day of data collection. All available records were reviewed.
Data collection: Separate interviewer-administered questionnaires were used to collect information from the healthcare workers and caregivers. The health workers were interviewed to assess their knowledge of the AEFI surveillance system as well to establish its perceived simplicity, acceptability, stability and usefulness. The caregivers' knowledge of the surveillance system was also assessed as well as their reasons for not reporting AEFIs. Key informant interviews were conducted with the provincial EPI manager, the District Medical Officer (DMO), District Nursing Officer (DNO) and community nurse in charge of immunisation for Guruve. A checklist was used to objectively assess all available AEFI reporting forms regarding completeness, simplicity, data quality and timeliness.

Data management analysis: We created a questionnaire in Epi
Info 7 TM and captured all the collected data. We used the software for data analysis to generate means, proportions and frequencies. Acceptability: Acceptability refers to the willingness of persons and organisations to participate in the surveillance system. It was determined by looking at the completeness of reporting forms and timeliness of reporting. We also compared the number of adverse events that each health worker and caregiver identified during the period under study and the number of times that they reported.

Measurement of variables
Sensitivity: Sensitivity refers to the proportion of AEFIs that occurred that were picked up by the surveillance system; it can also refer to the ability of the system to monitor changes in the number of cases over time. It was measured by comparing the number of cases that were detected to those reported to have occurred in the Page number not for citation purposes 4 community by the caregivers. It was also compared to the recommended WHO case detection rate of 10 per 100,000 surviving infants.
Timeliness: Refers to the speed at which data is transmitted between different levels in the surveillance system. To answer the research question, the time taken for caregivers to report an adverse event or a health facility and the time taken for health workers to report to the district were evaluated.
Stability: Stability refers to the reliability (i.e. the ability to collect, manage, and provide data adequately without failure) and availability (the ability to be operational when it is needed) of the surveillance system. This was assessed by looking at the availability of reporting forms and the availability of a health worker trained to identify and report AEFIs at each facility every time.
Knowledge: Health care workers' knowledge of the AEFI surveillance system was assessed by asking them the meaning of AEFI, to name five examples of common adverse events, to describe the reporting channels and the timelines for reporting. Caregiver knowledge was also assessed since they are the ones who are supposed to report any AEFIs to the health facility. They were asked to name any three things that could be of concern to them after immunisation, what they would do if they were worried about anything and how soon they should take this action. A Likert type scale was used to rate knowledge. Getting 0 to 2 questions correct was scored as poor knowledge, 3-4 was fair, and 5 was good knowledge.

Permission and ethical considerations:
Permission to carry out the study was sought from the Provincial Medical Director for Mashonaland Central province, the district medical officer for Guruve district and Health Studies Office. We obtained informed written consent from all study participants. Study participants were not coerced or offered rewards for participating in the study. All interviews were held in privacy. Names of participants were not recorded and any information gathered will remain confidential.

Results
Demographic characteristics of study participants: We successfully recruited 31 health workers and 33 caregivers into the study. The majority of health workers who participated in the study were female 22/31 (71%) and primary care nurses 21/31 (68%) with median years in service of 6 years ( Table 1). All the caregivers that were recruited into the study were mothers. Their median age was 28 (Q1= 22; Q3 =32) and their median number of children was 2 (Q1 = 2; Q3 = 4).
Reasons for not notifying AEFIs: All the health workers were asked what they thought were the reasons why AEFIs were not being notified. The majority of health workers 14/31 (45%) said that adverse events were not being reported because they were afraid of the personal consequences of reporting. These included fear of being blamed for causing adverse events and being investigated. Thirty-two per cent (10/31) of the health workers said that while adverse events may be occurring in the communities, they were not being notified because caregivers were not reporting them to health facilities (Table 1).
Sensitivity: Compared to the recommended WHO case detection rate of 10 serious cases per 100 000 surviving infants, Guruve district identified zero cases from January 2016 to November 2017.
Of the interviewed caregivers, 13/33 (39%) had children who had experienced AEFIs while 14/31 (45%) of the health workers had encountered AEFIs during the period under study. The surveillance system did not pick all these cases. The actual burden of AEFIs could not be determined.
Acceptability: All of the health workers reported that it was their duty to notify AEFIs and they were willing to participate in the surveillance system. Although 14/31 (45%) of health workers said they had encountered AEFIs during the period under study, there was no evidence of completed forms at all the health facilities, and there were no line lists of the AEFIs they had encountered ( figure   2). All the caregivers said they were willing to report severe adverse events to their health facilities. However, 31/33 (94%) of them said they would not report if they did not perceive the adverse event to be severe. Of the 33 caregivers who were interviewed, 13 (39%) of them had children who had experienced AEFIs, and 8 (65%) of them did not report them to health facilities ( Figure 2). Simplicity: Fourteen (45%) of the interviewed health workers had ever encountered an AEFI and three (21%) of them found the AEFI notification form challenging to complete. Twenty-six (85%) healthcare workers said they needed the training to complete the notification forms (Table 2). Three caregivers who had experienced Page number not for citation purposes 5 adverse events with their children were asked to report the events to health workers to assess the time it took to complete the notification forms. All three health workers took between 10 and 20 minutes without any difficulties.
Knowledge of the AEFI surveillance system: All the healthcare workers knew where to send the completed notification forms (Table 3). The least known variable was the number of notifying forms that must be completed (55%). Two-thirds (22) of the interviewed caregivers had ever heard of adverse events following immunisation, and all of the caregivers knew that they should report to a health facility if anything untoward happened after vaccination.

Usefulness of the system:
The majority of the health workers 29/31 (94%) perceived the AEFI surveillance system in Guruve district to be useful. However, only 8 (26%) said they had ever discussed AEFIs in regular meetings, and 3 (10%) said they had implemented new practices as a result of AEFI surveillance data. On records review, it was found that no health facility had any record of having discussed AEFIs in meetings or using AEFI surveillance data to influence policy or clinical practice.

Discussion
We found that the health workers were not reporting adverse events following immunisation despite having good knowledge of the surveillance system. Most of them were afraid to report AEFIs because they feared they would be investigated and face personal consequences if the adverse event was found to be a result of an error on their part. This fear of reporting was also reported in studies in South Africa and Kenya [12,13]. This persistent fear could be a result of insufficient training and lack of supportive supervision [14]. It can also be understood in an environment where health workers feel their job security is not guaranteed. We also found that caregivers of children were not reporting AEFIs to health facilities especially when they perceived them to be mild. This was despite most of them knowing that they should report adverse events to the clinic. Our finding is contrary to what Muchekeza et al. published in Kwekwe district of Zimbabwe where caregivers had a low reporting rate because they had inadequate knowledge of the surveillance system [15]. This difference shows that caregiver knowledge is not sufficient to improve their reporting rate and there is need to address the inconvenience of travelling back to the clinic.
There is need to consider complementing the passive surveillance system with a participant-centred active surveillance system where caregivers can send an SMS or use other communication technologies such as WhatsApp. Such innovations have been tried in low resource countries such as Cameroon and Cambodia with notable improvement in AEFI reporting [16,17]. The fear of reporting for health workers, coupled with the inconvenience of travelling back to the clinic for caregivers, made the surveillance system not acceptable. All the health workers who encountered AEFIs during the period under study did not report them, and the majority of caregivers said they would not report an adverse event if they thought it was mild. Mild adverse events have the potential to become severe if they are not reported, which may result in the community losing confidence in vaccines [1].
The sensitivity of the surveillance system was poor due to attitudes of caregivers and health workers towards reporting which suggests inadequacies in community sensitisation and training of health workers. We found knowledge among the health workers and caregivers to be good. At every health facility in Guruve district, there was at least one nurse formally trained in AEFI surveillance, and this gives the other nurses an opportunity to learn from the trained cadre, this is supported by results from a study in Kenya Page number not for citation purposes 6 where peer mentorship resulted in improved knowledge [18]. At the clinics, we visited we found that every morning the caregivers who bring their children for vaccination are taught about adverse events by the nurse at the clinic. However, this knowledge did not translate into improved reporting of AEFIs. We also found the surveillance system to be simple as the case definition of an AEFI removes the need to ascertain causality, anything that a caregiver comes complaining about after vaccination is considered an AEFI. The skills needed to use the system also existed at each health facility. Our study was not without limitations. Respondents were asked to recall events that had happened in the past, and this may have introduced recall bias into the study. In anticipation of this bias, we only asked all our respondents to recall events of a period not exceeding 18 months. A healthcare worker administered the questionnaires, and this may have introduced social desirability bias on the part of caregivers. We conducted our study in a rural district, and the results of our study may not be generalizable to urban districts.

Conclusion
We concluded that health workers in Guruve district were afraid to report adverse events following immunisation and caregivers were reluctant to report mild adverse events hence the surveillance system was performing poorly and was not useful. However, the stability of the system and the knowledge of both health workers and caregivers gives a good foundation for improving the surveillance system. We, therefore, recommended retraining health workers focusing on the importance of notifying all adverse events and allaying their fears of reporting. We also supported the introduction of a participant-centred active surveillance system where caregivers can use mobile phones to notify adverse events to the health facility. This reporting can be done via Short message services (SMS), social media platforms or an unanswered call that will be followed up by the health workers at the clinic.  We also show that fear of reporting is the major reason for the poor performance of the surveillance system in a rural district, which is contrary to what has been reported in urban districts.        Knew to return to clinic if an AEFI occurred to their child 33 (100) Figure 1: The AEFI surveillance system flow diagram, Zimbabwe