Vaccine wastage in Ghana, Mozambique, and Pakistan: An assessment of wastage rates for four vaccines and the context, causes, drivers, and knowledge, attitudes and practices for vaccine wastage

t r


abstract
Vaccine procurement costs comprise a significant share of immunization program costs in low-and middle-income countries, yet not all procured vaccines are administered. Vaccine wastage occurs due to vial breakage, excessive heat or freezing, expiration, or when not all doses in a multidose vial are used. Better estimates of vaccine wastage rates and their causes could support improved management of vaccine stocks and reduce procurement costs. This study examined aspects of wastage for four vaccines at service delivery points in Ghana (n = 48), Mozambique (n = 36), and Pakistan (n = 46). We used prospective data from daily and monthly vaccine usage data entry forms, along with cross-sectional surveys, and in-depth interviews. The analysis found that estimated monthly proportional open-vial wastage rates for vaccines in single-dose vials (SDV) or in multi-dose vials (MDV) that can be kept refrigerated up to four weeks after opening ranged from 0.08 % to 3 %. For MDV where remaining doses are discarded within six hours after opening, the mean wastage rates ranged from 5 % to 33 %, with rates being highest for measles containing vaccine. Despite national-level guidance to open a vaccine vial even when only one child is present, vaccines in MDV that are discarded within six hours of opening are sometimes offered less frequently than vaccines in SDV or in MDV where remaining doses can be used for up to 4 weeks. This practice can lead to missed opportunities for vaccination. While closed-vial wastage at service delivery points (SDPs) was relatively rare, individual instances can result in large losses, suggesting that monitoring closed-vial wastage should not be neglected. Health workers reported insufficient knowledge of vaccine wastage tracking and reporting methods. Improving reporting forms would facilitate more accurate https

Introduction
Since 1990, the global under-five mortality rate has decreased by more than half, due in large part to increased access to immunization [1,2]. Vaccine procurement costs are a significant share of Expanded Programme on Immunization (EPI) costs, estimated to account for 51 % of program costs [3]. Not all procured vaccines are administered, as vaccines may be wasted during transport and storage as a result of vial breakage, excessive heat or freezing leading to loss of potency, or expiration (this type of wastage is referred to in this paper as closed-vial wastage). Wastage also occurs when not all doses in a multidose vial are administered, as they must be discarded within six hours of reconstitution or at the end of the session for vaccines without preservatives; or within 28 days of opening the vial for vaccines with preservatives (referred to here as open-vial wastage) [4]. Multidose vials are commonly used in low-and middle-income countries largely because of lower vaccine price and smaller cold chain volume requirement per dose. Yet with multidose vials, health workers face distinct challenges, such as whether to open a vial when only one eligible child is present or to save the vial until more doses can be used-weighing the balance between vaccine coverage and timeliness versus reducing vaccine wastage [5].
A key factor for sustainability of the immunization program is budgeting adequate resources for vaccine procurement, especially given the higher prices of newer vaccines. One key barrier to correct budgeting is the lack of accurate data on vaccine wastage. In many countries, recording vaccine wastage at each level of the supply chain has been inconsistent and thus this indicator tends to be underreported [6]. This lack of accurate and consistent data on vaccine wastage can lead to overestimating vaccine wastage, driving unnecessary vaccine procurement and increasing supply chain costs, or underestimating vaccine wastage, causing potential stockouts and service delivery interruptions that may negatively impact coverage.
A few published studies have estimated vaccine wastage rates for countries and evaluated how these estimated rates differ by vaccine antigen and presentation, though most of these rely on retrospective data [7][8][9][10][11][12][13][14][15]. Some studies identified several key drivers of vaccine wastage. Specifically, wastage rates were reported as higher for vaccines in multidose vials without preservatives than for those with preservatives, as remaining doses in open vials in the former must be discarded within six hours of opening the vial [9,13,11]. Larger session sizes (higher number of eligible children present at a vaccination session) resulted in lower vaccine wastage rates [13]. A wastage assessment in India found similar results [14], reporting also on the poor documentation of wastage and large variation in wastage rates across the states. Another study in Nigeria [6] found that, to reduce wastage, vaccines without preservatives were offered less frequently at vaccination sessions and some health workers reported waiting for a certain number of children to be present before opening a vial of a vaccine without preservatives, potentially leading to missed vaccination opportunities. However, the study in Nigeria found that wastage rates did not differ much between vaccines with and without preservatives, leading the study's authors to surmise that vaccinators were overly concerned with minimizing wastage rates for vaccines without preservatives and also were not implementing policies consistent with the World Health Organization (WHO) Multi-dose Vial Policy (MDVP) to reduce wastage of vaccines with preservatives [16].
Because very few countries collect data on country-specific wastage rates, countries have had to rely on WHO indicative rates, but these are generic estimates that do not account for local context [17]. Knowledge of country-specific wastage rates and understanding the causes and drivers could support better management of vaccine stocks, potentially reduce costs, and support sustainability of vaccine procurement and increase vaccine availability.
This study sought to provide vaccine wastage data for three countries: Ghana, Mozambique, and Pakistan. The objective of this study was to provide evidence on vaccine wastage in routine immunization programs by quantifying open-and closed-vial wastage rates for four focus vaccines. We also sought to understand the context and major drivers of vaccine wastage, in order to inform forecasting, investment priorities, and planning for immunization service delivery.

Study countries
The study was conducted in Ghana, Mozambique and Pakistan, countries prioritized by Gavi for this research as it provides support to their routine immunization programs. The countries represent a range of target population size for routine immunization with birth cohorts of approximately 853,000 in Ghana, 1,073,000 in Mozambique and 5,820,000 in Pakistan for 2019. All three countries have four levels in the vaccine supply chain. At the time of the study, there were 16 regional vaccine offices in Ghana, 10 of which had vaccine storage warehouses while six were recently established and did not have any walk-in cold rooms and so served in an administrative role only. There were 260 district vaccines stores and approximately 9,000 SDPs providing immunization services. Mozambique had 11 provincial vaccine stores in each of the ten provinces and the capital city, 162 district vaccines stores, and 1611 service delivery points (SDPs) or health facilities providing immunization services (as of 2020). Pakistan had 8 provincial vaccine warehouses, 153 district vaccine warehouses, and 7858 SDPs providing immunization services.

Selected focus vaccines
The evaluation focused on four Gavi-supported vaccines used in routine immunization and prioritized by Gavi for the research: pentavalent vaccine, pneumococcal conjugate vaccine (PCV), rotavirus vaccine, and measles-containing vaccine (MCV). 1 The characteristics of the vaccines are shown in Table 1.

Study sites
The study obtained information on all district-level administrative areas, the population served by each district, and the corresponding service delivery points (SDPs) providing vaccination, and used this as the sampling frame for each country. We used the Effective Vaccine Management (EVM) site selection methodol- 1 At the time of the study, measles vaccine was used in Pakistan; measles-rubella vaccine was used in Ghana and Mozambique.
M. Mvundura, J. Ng, K. Reynolds et al. Vaccine xxx (xxxx) xxx ogy [18] for sample selection, as this has been used by WHO for conducting other vaccine supply chain assessments. The EVM methodology uses a one-stage systematic sampling methodology to randomly select the districts to achieve an 80 % confidence level and ± 10 % precision on the estimated vaccine wastage rates. As recommended by the methodology, we randomly selected two SDPs in each district to include in the study. The district storage facilities, corresponding regional vaccine warehouses, and the national vaccine warehouse were also included as study sites. The sample sizes for each country are shown in Table 2. Primary data collection began in November 2020 in Mozambique, in December 2020 in Pakistan, and in January 2021 in Ghana.

Data collection 2.4.1. Prospective vaccine wastage data
To inform the calculation of open-and closed-vial vaccine wastage and obtain insights on the types of vaccine wastage occurring, including estimates on variables not collected in routine reporting, we conducted a 3-month prospective data collection in the respective countries. No vaccine campaigns were held in any of the study countries during this 3-month period.
We oriented the immunization staff at each SDP to use a daily data entry form to capture information on vaccine usage and wastage. We requested that staff complete these forms using data they already report, augmenting it with additional vaccine stock information requested as part of the study. For closed-vial wastage, the forms included quantities of vaccines wasted due to expiry, vaccine vial monitor (VVM) indication, freezing, breakage, missing inventory, and damaged labels.
We provided similar forms at national, regional, and district vaccine storage facilities. In Ghana, district vaccine stores documented data daily while the regional and national vaccine warehouses documented data monthly. In Mozambique, district and provincial warehouses documented data monthly. In Pakistan, prospective data were initially documented monthly by all administrative levels; however, due to low completeness and poor data quality, the data for the initial 2 months were discarded by the study team, and staff at the study facilities re-collected data for another 3 months using a daily form to increase data completeness and quality.
The study employed strategies to improve completeness and quality of the data collected at the SDPs, including working with district-level supervisors to encourage better record-keeping, and encouraging increased monitoring and supervision of lower-level facilities by EPI or health program managers. We conducted follow-up visits to facilities not returning the data to increase compliance and data completeness.

Facility interviews: cross-sectional surveys and in-depth interviews
The study used structured questionnaires to collect data from each SDP in the sample to better understand the facility's context, which could impact vaccine wastage. The survey was conducted with the staff in charge of the immunization program, including vaccine stock management at each facility. At some facilities, several respondents participated in the facility interview, when the person in charge of immunization was not the one responsible for vaccine stock management. To provide additional context, we conducted in-depth interviews using semi-structured discussion guides at a subset of study facilities to assess the knowledge, attitudes, and practices of the staff related to vaccine wastage. Discussions were audio recorded and transcribed. We purposively selected the sites for in-depth interviews, with an effort made to include both urban and rural sites and to represent the different regions or provinces in the sample. Table 1 Characteristics of the focus vaccines. Similar questions were used in each country and modified for country context. Table 3 shows the topics for interviews by administrative level. The cross-sectional surveys and in-depth interviews were conducted by data collection teams during the same facility visit. Data collectors were trained before conducting these interviews.

Data analysis
For country-specific data analysis, a separate analysis for each country was done using methods standardized through a common data analysis plan and as described below.

Prospective vaccine wastage data
Using the prospective vaccine wastage data for each SDP, we calculated monthly open-vial wastage rates following the formula:

Doses opened for use À number of children immunized Doses opened for use X100
For the SDPs and vaccine storage facilities, we computed the monthly closed-vial wastage rates following the formula:

Number of doses discarded unopened Start balance þ number of doses received X100
For each vaccine, to obtain a monthly mean of vaccine wastage rates across all facilities at the same level, we calculated a perfacility mean wastage rate across all 3 months for each country. This was followed by a calculation of a mean across all facilities by administrative level. The calculations were done in Microsoft Excel (Microsoft Corporation). Country-specific estimates are reported. Median and ranges of the wastage rates across facilities are also reported in the results section.

Cross-sectional survey data
The cross-sectional survey data were tabulated in Microsoft Excel and R-Studio software. For numerical values, the mean, median, and range were calculated. For categorical variables, the number of responses in each category was tabulated and frequencies were provided as numbers of valid responses and percentages.

Regression analyses of MCV wastage rates and cross-sectional data for SDPs
A pooled linear regression analysis was run using Stata (Stata-Corp LP) to identify the drivers of vaccine wastage for MCV, which had the most consistent presentation and handling policy across the countries. The estimated wastage rate for each vaccine was the dependent variable; the independent variables were the statistically significant variables (with p < 0.20) from a binary subgroups cross-tabulations of the wastage rates with the context variables from the cross-sectional analysis. We also included some control variables for country, location, target population, and frequency of holding immunization sessions in this regression. The coefficient values and p-values from the regression were reported. We acknowledge that in some cases, correlations do not imply causality. Regressions were done separately for open-and closed-vial wastage rates.

In-depth interview data
To analyze the in-depth interviews, we conducted a thematic analysis to produce a description of key concepts and themes [19]. Transcripts were imported into ATLAS.ti version 9 (Scientific Software Development GmbH) for data analysis and were analyzed based on a codebook using both inductive and deductive approaches using preliminary data. For the data in each country, two coders analyzed initial transcripts and reviewed them together to ensure alignment. Transcripts were then coded independently by one member of the coding team and later exchanged and reviewed by another member. Disagreements in code application were resolved through discussion. Resulting key themes across the study countries are reported.

Ethics reviews
The study was determined by the PATH ethics review committee to be exempt from US institutional review board (IRB) oversight. It was also reviewed and approved by the country IRB committees: the Ghana Health Service Ethics Review Committee, the Mozambique Comité Nacional de Bioética para Saúde, and the Pakistan Research Ethics Subcommittee of the National Bioethics Committee.

Vaccine wastage at service delivery points
The number of SDPs included in the Ghana study was 48 and in Pakistan, 46. In Mozambique, 46 SDPs had been initially selected for inclusion but only 36 are reported as some were inaccessible due to socio-political conflict. In Ghana, the respondents were mostly community nurses and the years of experience ranged from 1 to 18 years with a modal value of 6 years. In Mozambique, they were mainly preventive medicine technicians with experience ranging from 0 to 11 years, with a modal value of 4 years. In Pakistan, they were mainly vaccinators and lady health visitors with experience from 2 to 39 years with a modal value of 5 years.

Open-vial wastage at SDPs
The histograms and associated statistics on kurtosis show that wastage rates are mostly not distributed normally (results not shown). Therefore, we report medians in addition to means and ranges. Table 4 shows the estimated open-vial wastage rates. The openvial wastage rates for vaccines in multidose vials without preservatives (i.e., MCV) were higher than for vaccines in multidose vials with preservatives (i.e., pentavalent vaccine in Ghana and Mozambique; PCV in all countries). (Liquid vaccines with preservative can be kept after opening for up to 28 days according to the MDVP.) Yet the open-vial wastage rate for PCV was higher in Pakistan than in the other two countries, because at the time of the assessment, opened vials of PCV taken for outreach sessions are discarded within 6 h of opening. Mean and median wastage rates were zero or close to zero for single-dose presentations, which was the case for rotavirus in Mozambique and Pakistan and for pentavalent in Pakistan. The average wastage rate for rotavirus vaccine in Ghana, which is in a multidose presentation, was 19 %, as remaining doses in open vials were discarded during the time period of our assessment, due to a packaging issue that would be resolved in subsequent shipments.
For some vaccines, we observed large variations in wastage rates across facilities, as seen in the range of wastage rates reported even when the mean or median is small, such as for pentavalent vaccine and PCV in Ghana and rotavirus vaccine in Pakistan. The largest wastage rates and the largest range of wastage rates across all three countries was for MCV; yet for some SDPs in Mozambique and Pakistan, the number of children vaccinated with MCV was the same as the number of doses opened for use, resulting in zero wastage and hence the minimum wastage rate of zero.

Closed-vial wastage at SDPs
Across all three countries, 17 % to 50 % of SDPs in the sample experienced at least one incident of closed-vial wastage during the prospective period (Table 5). This percentage was greatest in Mozambique, where half of SDPs experienced at least one occurrence of closed-vial wastage. The closed-vial wastage rate estimates in some cases include opened vials discarded after storage in refrigerators, as some sites incorrectly reported these data together and it could not be parsed during analysis. The causes of closed-vial wastage reported at SDPs included vaccine expiration, vial breakage, heat or freeze damage, and damaged labels. Across the three countries, closed-vial wastage was reported for all focus vaccines, although the mean closed-vial vaccine wastage rates for most were below the standard WHO indicative rate assumption of 1 %. The mean closed-vial wastage rates were above 1 % for pentavalent vaccine in Ghana and Mozambique, for PCV in Ghana, and for rotavirus vaccine in Mozambique. At the SDPs where the wastage occurred, the proportional wastage rate was large, as reported in the table.

Closed-vial wastage at storage facilities in Ghana and Mozambique
In Mozambique, of the 23 district vaccine stores initially selected for inclusion, only 17 are reported, as some were inaccessible due to socio-political conflict. Additionally, the stock data to estimate closed-vial wastage were of poor quality for Pakistan and hence not reported.
At the district level, Ghana reported no closed-vial wastage of pentavalent vaccine but there was some closed-vial wastage of PCV, rotavirus, and MCV, with missing inventory and vial breakage being the most common causes, but the total number of vials wasted was low. In Mozambique, one district vaccine store reported closed-vial wastage of pentavalent caused by breakage; there was no wastage of any other focus vaccines. At regional/ provincial level, the median monthly proportional vaccine wastage rate in closed vials was 0 %. The estimated proportional vaccine wastage in closed vials is shown in Table 6; all are below the standard assumption of 1 %. Table 7 shows the results of the cross-sectional surveys at SDPs, which aimed to provide context to the reported vaccine wastage data.

Geography
The majority (73 %) of SDPs in the Mozambique sample were rural compared to Ghana and Pakistan, where the proportions for rural/urban classification were closer to 50 %.

Vaccination session type and frequency
In all three countries, at least 79 % of the SDPs in the study sample reported that they conduct both fixed and outreach sessions; other SDPs conducted either one or the other type. The median frequency of fixed sessions at SDPs was daily in Mozambique and Pakistan compared to weekly in Ghana.

Availability of vaccines at vaccination sessions and willingness to open a vial
Across the three countries, MCV is less available at fixed and outreach sessions than the other three vaccines. National policy in all three countries calls for opening a vial of this vaccine when only one eligible child is present at a session. However, in a fixed session only 34 % of the SDP respondents in Ghana reported they Table 2 Number of sites included for the cross-sectional and prospective data collection in the study countries. would open a vial of MCV for a single child and 31 % in an outreach session, with these percentages at 47 % and 36 % in Pakistan, respectively. In Mozambique, we asked the average number of children present when opening vials, not the number of children who must be present to open a vial; 58 % of respondents said one child is present on average before a vial is opened, and 42 % said more than one child is present on average.

Handling of vaccines after vaccination sessions
The majority of SDPs in Ghana and Mozambique reported keeping the remaining doses in opened vials of pentavalent and PCV after fixed and outreach sessions, compared to Pakistan where most SDP respondents said they discard them. However, across all three countries, >85 % of respondents reported that closed vials for all vaccines are returned to the refrigerator after outreach.

Minimum and maximum vaccine stock levels
Across the three countries, minimum or maximum stock levels for vaccines were not defined or not known by at least 20 % of the SDP respondents in any country, with a much higher percentage in Ghana.

Supervision visits and discussion of wastage rates
In Ghana and Mozambique, supervision visits were reported as occurring every 3 months by almost 50 % of the SDP respondents, whereas in Pakistan most SDP respondents (74 %) said these occurred monthly. Respondents across all three countries reported that wastage rates are discussed at some but not all supervision meetings.

Knowledge about target wastage rates
The target wastage rates at national level for Ghana were 5 % for both PCV and rotavirus vaccine (for the single-dose presentation rotavirus vaccine), 15 % for pentavalent vaccine and 25 % for MCV. Mozambique had similar targets as Ghana for both PCV and rotavirus vaccine, while the target was 10 % for pentavalent and 40 % for MCV. In Pakistan, the national target wastage rate was also 5 % for both pentavalent and rotavirus vaccines, 10 % for PCV and 20 % for MCV. Knowledge about whether these target wastage rates existed varied by country, with some respondents not knowing whether there were target wastage rates. Those who responded that there were target wastage rates defined reported widely different targets for the same vaccine, showing either a lack of common knowledge of the correct target wastage rate or that some targets vary across contexts.

Knowledge, attitude, and practice about vaccine wastage
For the knowledge questions (Table 8), about 40 % (Ghana), 28 % (Mozambique) and 43 % (Pakistan) of respondents at the SDPs agreed or strongly agreed that vaccine supply was affected by vaccine wastage, with fewer making the linkage to closed-vial wastage than open-vial wastage. At least 60 % of the respondents said there should be more training on monitoring and managing vaccine wastage, with higher agreement on this need in Ghana and Mozambique. For the questions related to vaccine management, respondents generally reported knowledge of the MDVP. About half to two-thirds of respondents reported knowing the form to complete and what information to record for vaccine wastage on the forms, with these percentages lowest for Ghana respondents, at 42 %.

Regression analysis of open-vial wastages for MCV
The regression analysis of the MCV wastage rates was done using the cross-section survey variables that were found to be statistically significant in the cross-tabulation (Supplementary  Table S1) as independent variables. Results are shown in Table S2 and show that Mozambique had a statistically higher wastage rate Processes and practices related to handling of remaining doses of vaccines at the end of vaccination sessions.
Processes and practices for recording and reporting vaccine wastage.
Perspectives on vaccine wastage rates, the utility of having target wastage rates, and strategies to meet both coverage and wastage targets.

Vaccine storage facilities
Not reported Processes and practices for ordering and storing vaccines; where vaccine wastage occurs.
Processes and practices taken to reduce or prevent vaccine wastage.
Availability of contingency plans in the case of cold chain failure, and processes and practices implemented.
Guidance provided to service delivery points in their jurisdiction about handling of remaining doses of vaccines at the end of vaccination sessions.
Processes and practices for recording and reporting vaccine wastage.
Perspectives on vaccine wastage rates, the utility of having target wastage rates, and strategies to meet both coverage and wastage targets.

M. Mvundura, J. Ng, K. Reynolds et al. Vaccine xxx (xxxx) xxx
for MCV compared to Ghana and Pakistan combined. SDPs serving catchment sizes in the first quartile (those SDPs serving the lowest 20 % of the catchment sizes in the study sample population distribution) had higher wastage rates compared to those serving population sizes in other quartiles (those SDPs serving 20 % or higher of the catchment sizes in the study sample population distribution).
In addition, wastage rates were statistically lower at SDPs where respondents said that they wait for more than one eligible child to be present before opening a vial; however, the wastage rates were higher at SDPs where they said MCV was not offered at all fixed sessions.

In-depth interviews
The analysis included 98 in-depth interviews: 24 from Ghana, 21 from Mozambique, and 53 from Pakistan.

Stakeholder feedback on open-vial wastage
A key theme apparent across the three countries is the effort health workers make to balance the two aims of minimizing wastage and maximizing coverage, which can be in competition when vaccines are in multidose vials without preservatives. The MDVP is considered an effective strategy for minimizing openvial wastage and is well known across the interview respondents in Ghana and Mozambique. In Ghana, in locations with vaccination sessions more frequent than once monthly, health workers report they are more comfortable to open a vial for only one child present when the vaccine can be kept under the MDVP for later use. Some workers at facilities without cold chain equipment described returning open vials to cold chain equipment at the district or neighboring health facilities. In contrast, there was poor knowledge or practice of the MDVP in Pakistan, particularly in outreach sessions.
Interviews revealed differences across countries when it comes to the policy of opening a vial even when only one eligible child is present. In Ghana, while this was the official policy stated by the national-level respondent, health workers at SDPs acknowledged they are reluctant to open vials of some vaccines (i.e., with multi-ple doses per vial lyophilized or without preservatives) for only one child, due to the increased wastage that results. In Mozambique, health workers were generally more likely to state their adherence to the policy of opening a vial for only one child. In Pakistan, health workers stated that while the district guides them not to open vials of some vaccines unless there are multiple eligible children present, they are likely to open vials for only one child, especially during outreach activities. Workers in all countries described practices to use multiple doses once the vials are open, for example by going door-to-door to find children in need of the vaccine, or by sharing vials across facilities.
If there is even one child, we have to open BCG and measles. Like when they come from far-flung areas to our Center, we have no choice but to administer it; otherwise, next time he won't come here. -District supervisor of vaccines, Pakistan. The policy says anytime you meet a child and due for vaccines, the vaccine should be opened, but the field staff use their discretion. At least they want to get seven or eight to vaccinate them, which leads to missed opportunities, but people do it to make sure they minimize wastage. -Cold chain manager, upper-level administration, Ghana.

Stakeholder feedback on closed-vial wastage
In all three countries, interviewees described vaccine stock management practices to minimize closed-vial wastage at both storage and service delivery levels; for example, when picking stock for distribution or use, attention is paid to date of expiry and status of the VVM to ensure use of vaccine before a discard endpoint would be reached. At vaccine storage facilities, staff described the importance of demand estimation and/or adhering to maximum stock levels to try to avoid overstocking, which could lead to wastage.
Those that have expiry dates near, we use those ones first. First in, first out-it also depends on the VVM as well. Sometimes the expiry dates might be near, but the VVM might be better than the [vaccine with later expiration date] so those with the Table 4 Monthly proportional open-vial wastage at service delivery points, estimated using 3 months of collected prospective data. MCV: measles-containing vaccine; PCV: pneumococcal conjugate vaccine.

Table 5
Monthly proportional closed-vial vaccine wastage at service delivery points, estimated using 3 months of collected, prospective data. While confident in storage temperature practices, some interviewees expressed concern that vaccines may be susceptible to heat exposure and vial label damage during transportation, leading to discarded vials. Although cold chain failure was not reported as common and staff could describe actions to be taken when refrigerators failed, written contingency plans for this event were not necessarily available in all countries.
There is a lack of a specified vehicle that can transport the vaccines... ...We have no way to protect the vaccines, we don't have those covers to protect the vaccines, or the boxes themselves. -Preventive medicine and environmental sanitation technician, rural district warehouse, Mozambique.

Reporting of vaccine wastage and supervision
Across the three countries, interviews revealed challenges with reporting of vaccine wastage. Respondents noted that reporting forms do not include all the causes for wastage experienced at the SDPs, creating confusion when reporting. Furthermore, instruction and supervision related to wastage reporting are inconsistent across locations, even within countries. Some respondents consolidate all reporting activities for the monthly report and may lose track of day-to-day detail that could improve accuracy of vaccine wastage data. The emphasis on wastage rate targets may influence accuracy of reporting as well as willingness to open a vial.
There is no separate record/form for vaccine wastage. We are reporting wastage in the monthly report only, and all sort of wastage is counted under leftovers, though there is no other type of vaccine wastage. -EPI technician, peri-urban SDP, Pakistan.
In Ghana and Pakistan, respondents reported that they perceive their performance to be based more on coverage than wastage. In Mozambique, wastage rates were sometimes perceived to reflect on staff performance. Across the three countries, there were conflicting perspectives on target wastage rates. Some said they are useful for forecasting and for prioritizing wastage reduction, but many pointed out that they are not practical and highlighted nuances such as differences in achievable wastage rates between fixed versus outreach immunization strategies.
All they want is [vaccine] targets to be achieved....... Wastage is not their concern; they tell us to ask for double the stock if need be; however, if we miss a child then that's on us. -Vaccinator, rural SDP, Pakistan. It [target wastage rates] prevents some of the children to be immunized because the health workers want to get [a number of] children before opening the vaccine. When we tell a mother to go and come for the following month, maybe she might not come. -Community health nurse, SDP, Ghana.

Discussion
The data from this study in three countries-Ghana, Mozambique, and Pakistan-highlight some known drivers of vaccine wastage in routine immunization settings. Open-vial wastage rates were higher for vaccines in multidose vials without preservatives than for multidose vials of vaccines with preservatives. Similarly, the implementation of the WHO multidose vial policy, which prescribes conditions under which remaining doses in open multidose vials may be kept for up to 28 days, [16] reduced open-vial wastage rates; wastage rates were higher where this was not implemented, such as for rotavirus vaccine in Ghana and PCV in Pakistan at the time of the assessment. In Ghana, the rotavirus vaccine stock in MDV was shipped prior to a WHO approval allowing this vaccine to be kept open for up to 28 days. Subsequent shipments from the manufacturer include changes to the package and label, and wastage rates for this vaccine are expected to decline. Similarly in Pakistan an older EPI policy of discarding doses after outreach sessions was established prior to a change in the WHO recommendation, which allows storage for 28 days after opening during outreach. The national policy was updated in early 2021 to align with WHO policy. It is possible that wastage rates may be higher for recently introduced vaccines when the presentation at introduction requires remaining doses to be discarded within 6 h. These practices at introduction may persist even after global policy changes to allow for storage of remaining doses in opened vials for longer durations.
Our findings are generally aligned with the limited primary data-based literature on this subject. A study conducted in Bangladesh found that wastage rates were higher for measles vaccine at 69 %, compared to 44 % for diphtheria, tetanus toxoid, and pertussis (DTP) vaccine [13], while a study in Northern India [11] found these rates to 22 % for measles-rubella and 8 % for pentavalent, and a study in South Sudan [10] estimated a wastage rate of 54 % for measles vaccine and 30 % for pentavalent vaccine. A retrospective wastage assessment conducted in five states in India [14] over a 6-month period in 2009 to 2010 found that vaccine wastage rates Table 6 Monthly proportional closed-vial vaccine wastage at storage facilities, estimated using 3 months of collected, prospective data.   [9] using retrospective data found similar wastage rates for pentavalent and MCV, which is rather unusual given the different handling practices of the two vaccines. A comparison of our open-vial wastage rate estimates with the WHO wastage rates calculator [20] suggests that the calculator consistently estimates higher wastage rates, even with the calculator accounting for a 5 % closed-vial wastage rate. (According to the WHO calculator, the wastage rate in routine immunization settings for pentavalent vaccine (10-dose vial) is 9 %; for PCV (10-dose vial), 9 %; for single-dose vial rotavirus, 4 %; for measles-rubella (10-dose vial), 72 %.) The differences in the wastage rate estimates could be due to the underreporting of wastage rates in the prospective data collection, or it could be that the WHO calculator truly overestimates wastage rates.
Similar to previous studies, we found that in order to reduce open-vial vaccine wastage, vaccines without preservatives, such as MCV, may be offered less frequently at vaccination sessions than vaccines with preservatives, even though this may cause missed opportunities for vaccination. Across the three countries in our study, MCV was less frequently available at fixed and outreach sessions than the other three vaccines, and vaccinators in Ghana and Pakistan reported they are less likely to open a vial of this vaccine when only one eligible child is present at the session. A study conducted in Nigeria [6] resulted in similar findings.
Holding fewer sessions could reduce wastage rates as children would be aggregated at fewer sessions, thus increasing session sizes. Our statistical analyses, however, suggest that facilities that held fewer sessions had higher wastage rates for some vaccines, a possible indication that even the sessions that were held had small session sizes, leading to higher wastage. This association may be confirmed by the finding that wastage rates of MCV were higher for SDPs serving population sizes in the lowest quartile compared to those serving larger populations. Although we did not have data on session sizes and could not explore this further, this could be done in future studies. Our study finding that session frequency may be less correlated to open-vial vaccine wastage than factors such as the size of the target population for vaccination is different from the assumption in the WHO wastage rate tool where session frequency is an important input.
The guidance to vaccinators from the national level is to open a vial of any vaccine even when only one eligible child is present at a session. However, the practice of vaccinators at SDPs often differs from that policy according to our survey data, and those SDPs where health workers said they wait until more children are present had lower wastage rates. It is possible that the goal to lower wastage rates results in missed vaccination opportunities, yet a higher wastage rate can also lead to missed vaccination opportunities when it leads to stockouts between vaccine replenishments [21]. A previous study evaluated possible thresholds and concluded that it is context-specific but should account not only for vaccine wastage, but also the health impact of missed vaccination opportunities [22].
Data on closed-vial wastage are generally not collected at SDP level, but our study found occurrences of this. Averages hovered around 1 % for our focus vaccines, confirming the standard planning assumption. Looking across SDPs, closed-vial wastage occurrences are relatively rare, but individual instances can result in relatively large losses. Some immunization program managers may pay more attention to closed-vial wastage at vaccine storage facilities given that wastage events at those levels can lead to the loss of large quantities of vaccines; our findings suggest that monitoring of closed-vial wastage should not be neglected at SDPs.
A recurrent finding across our data collection activities was health workers' insufficient knowledge of how to track or report vaccine wastage; this was supported by the finding of a desire for more training on vaccine wastage. At SDP level, closed-vial wastage is often not reported, but subsumed in open-vial wastage calculations, eliminating the opportunity to work on issues that may lead to closed-vial wastage. Updating reporting forms to include more reasons for wastage would facilitate more accurate reporting. More frequent supervision may help prevent or correct poor practices and provide routine support, but these visits were not conducted frequently in most facilities in our sample. Guidance, training, monitoring systems and other mechanisms are needed to better differentiate which wastage is avoidable, which is not, and to implement mechanisms to reduce avoidable wastage.
The study findings led to some recommendations. At country level, first there is a need to enhance vaccine wastage related training, sensitization, and supportive supervision of staff at SDPs. Supporting the reporting and monitoring of wastage in a non-punitive way could build transparency and trust around a common goal of decreasing vaccine wastage while maintaining and improving vaccine coverage. This may involve reconsidering the use of target wastage rates, particularly if they are not consistently relevant across settings and strategies, and may be hindering immunization goals. Second, if vaccine handling guidelines such as those outlined in the MDVP change after vaccine introduction, countries should promptly update local guidelines and educate staff about the change in order to reduce open-vial vaccine wastage. Third, the monthly EPI reporting forms and vaccine ledger should be updated with comprehensive reasons for closed-vial wastage and include reporting of such wastage at SDPs that would inform wastage reduction. Lastly, where available and feasible, countries should consider using different vial sizes in different vaccination contexts, with lower-dose vials used at smaller-size sessions to reduce openvial wastage [21,23,24].
At the global level, efforts toward reducing the number of doses per vial for lyophilized and/or unpreserved vaccines could both reduce wastage and increase the likelihood that these vaccines are offered at more sessions [21]. In addition, our study findings suggest there may be a need for global efforts to validate and refine the global vaccine wastage tools.
The study had some challenges and limitations. First, the onset of the COVID-19 global pandemic created delays and challenges with data collection. Second, since daily reporting of vaccine usage and wastage was not part of the regular routine or was not prioritized by many health workers, prospective data collection was challenging and required significant supervision and follow-up. However, despite having some challenges with data quality, our findings are consistent with findings by other researchers, and we believe that the data quality issues did not impact the estimates generated. Third, the 3-month study period wastage rate estimates may not be representative of annual wastage rates if there are seasonal variations. Our estimated wastage rates during the prospective period may have been lower as health workers were aware that they were being monitored. Fourth, we acknowledge that qualitative data may be reflect in some cases health workers' desire to report a perceived expected or desired practice rather than real events. Lastly, given the relatively small sample sizes compared with the number of SDPs in the country, the results may not be robust or allow meaningful statistical associations as the pooled sample may not be representative of health facilities across the three countries. These limitations point to possible improvements for future studies.

Conclusion
This study provided country-specific wastage rates for four vaccines and highlighted some known drivers of vaccine wastage and staff knowledge, attitudes, and practice that impact vaccine wastage. The study found that, to reduce open-vial vaccine wastage, vaccines without preservatives may be offered less frequently at vaccination sessions than vaccines with preservatives, even though this may cause missed opportunities for vaccination, and SDPs in less populous areas have higher wastage rates of these vaccines. Further research could be done that specifically looks at session sizes, which this study did not collect. Further research could also explore methods that provide for validation of interviewer-reported information received through records review. The study showed that avoidable vaccine wastage issues are more prevalent with new vaccine introductions; countries should pay particular attention to those. Providing means to track closed-vial wastage as well as supportive supervision to identify its causes could reduce overall vaccine wastage in countries. The study also showed a desire by staff for more training on vaccine wastage. Additional studies in these and other countries could be done to provide more robust wastage estimates leading to improved planning for vaccine procurement.

Data availability
Data will be made available on request.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.