An impact assessment of the use of aerial logistics to improve access to vaccines in the Western-North Region of Ghana

.


Introduction
Prevention of infectious diseases among children is crucial to improve child health and survival [1]. However, many countries in Sub-Saharan Africa struggle to achieve the Global Vaccine Action Plan (GVAP) of universal access to immunization, with only 13 % and 19 % achieving the 80 % coverage target in 2015 and 2016 respectively [2] due to several supply chain challenges, which lead to vaccine shortages, stockouts, missed opportunities for vaccination, and increased overall costs to the health systems.
The vaccines and health products supply chains within the Ghana Health Service are structured in a multi-tiered system, with the National Vaccine and Health Commodities Depot serving as a central hub for storage and distribution. Regional Health Directorates manage supply chains within their regions, distributing vaccines to District Health Directorates and healthcare facilities. Despite this structure, the supply chains face challenges such as limited infrastructure, especially in remote areas, leading to inadequate storage, unreliable transportation networks, and weak cold chain systems. Disruptions due to stockouts, delayed deliveries, or poor forecasting and planning can lead to the unavailability of vaccines.
Vaccination deficits are life threatening to children and pose an economic threat to health and social protection systems. economic toll of $5,230,035 per year ($777 per case) in the region due to PCV13-type Pneumonia and Meningitis cases [3]. In addition, rural areas were disproportionately affected because of insufficient storage capacities and challenges of traveling long distances by ground transportation to pick up vaccines. Although Ghana has made significant progress in tackling vaccine preventable diseases [4], there is still room to improve certain supply chain performance metrics, like stockouts, wastage, and cold chain storage capacity.
Drone technology has evolved to become a key tool in strengthening supply chains for health commodities. It has the potential to overcome difficult terrain, reduce health worker labor, solve vehicular issues with its associated costs [5], overcome the logistic challenges in hard-to-reach areas [6], and improve planning capacity.
With the aim to make additional progress towards Universal Health Coverage, the government of Ghana contracted Zipline in 2019, an aerial end-to-end logistics technology, to set up distribution centers (DCs) and improve equitable distribution of health products, particularly in rural communities. The operation is based on autonomous drones, which deliver essential medicines, blood, and vaccines to health facilities [7]. With six hubs in Ghana, Zipline covers more than 2,200 health facilities that provide services to approximately twelve million people. As of November 2022, over 6 million doses of vaccines from the National Immunization Programme and about 2.4 million medical products had been delivered by drones in the country.
Districts who have challenges with logistics, especially to deliver vaccines to hard-to-reach facilities, have made Zipline an integral part of their supply chain (Ghana's Ministry of Health -Programme of Work, 2021). The hub located in the Western North Region currently serves 365 facilities across 19 districts. Among those facilities, 202 (55.3 %) have Zipline as their sole provider for vaccines.
In this study, we combined analytical methods to explore associations between aerial logistics and vaccine related outcomes, including supply chain performance, vaccination coverage, vaccine-preventable diseases, and providers' perceived quality and satisfaction.

Study design
The study combined two types of designs. A cross-sectional approach was used to explore the performance of the supply chain for vaccines, providers' behaviors when vaccines were stocked-out, and their satisfaction with access to vaccines. Served and nonserved facilities were used for comparison purposes. In addition, a retrospective quasi-experimental design was used to evaluate changes in vaccination coverage and clinical outcomes at the district level within the same region between the years 2017 and 2021. This part of the analysis was conducted at the district level, and the comparison involved high and low levels of exposure to aerial logistics.

Study population
For the cross-sectional component of the study, we randomly selected 78 health facilities that were served by Zipline for the distribution of vaccines, and 78 facilities that were not. Data collection involved phone interviews to health providers in charge of vaccines at the facilities. The sample of facilities was stratified to ensure comparability across facility types (e.g., National Community Health Planning and Services (CHPS Compounds), Health Centers, and Hospitals). The interviews were conducted by Zipline's fulfillment operators and data was digitally registered in forms that fed the study dataset.
For the quasi-experimental component, district level data on vaccination coverage and vaccine preventable diseases was provided by the Western North Regional Health Directorate of Ghana for the years 2017-2021.

Exposure to aerial logistics
For the cross-sectional component, since data was collected at the facility level, the exposure to aerial logistics was obtained from Zipline's fulfillment database, while the list of unexposed facilities was obtained from the Ghana Statistical Service. For the retrospective quasi-experiment, data was obtained at the district level, and we used the proportion of facilities within the district that were exposed to aerial logistics to define the exposed/non-exposed categories. Across districts, the exposure proportion to aerial logistics ranged between 23 % and 85 %. High exposure was defined for districts with exposure proportions greater than 75 % (Suaman, Aowin, Sefwi Akotombra, and Bodi), whereas districts with exposure proportions under 26 % were considered as low exposure (Bia West and Bia East). Districts with exposure levels between those thresholds were excluded from the analysis (Juaboso, Sefwi Wiawso, and Bibiani Anhwiaso Bekwai).

Statistical analysis
For the cross-sectional component, we used descriptive statistics to identify supply chain and provider satisfaction outcomes between exposed and non-exposed facilities. Chi-Square tests, T-Tests and ANOVAS were used depending on the nature of the variables. For the retrospective component, district-level difference-in-differences (DID) models were fit for each vaccine, setting the time of exposure as the first quarter of 2020, when the Zipline's distribution center began operating in the area.
Unadjusted means for vaccination coverage across all vaccines were estimated and presented graphically. In addition, the DID models were adjusted by projected births and population poverty proportions, obtained from the latest Population and Household Census in Ghana. Results were reported as pre-post changes in percentage points. For health outcomes, DID models were not conducted given violations of the parallel trends assumption. Alternatively, multiple linear regression models were fit using post-intervention data and adjusting by total births and districts' poverty levels. Post-regression marginal effects were estimated.
For both components of the investigation, the analysis was conducted on Stata V.17.0 (Stata Corp, TX, 2021). The research protocol was submitted to the Ghana Health Service Ethics Review Committee and approved on 22nd November 2021 (reference number GHS-ERC 015/09/21).

Results
For the cross-sectional component, 156 participants including nurses, physician assistants, medical doctors, midwives, and public health officers were interviewed, of which 78 were served by Zipline and 78 were controls. Most of the participants (63.5 %) were within the 40-49 age range, and 67.3 % were identified as males. In terms of their health professions, 63.5 % were nurses, followed by physician assistants (12.8 %).
The Zipline group tended to be younger (over a third of the sample under 29 years old), and there was a relatively higher presence of women among that group (39.7 % vs. 25.6 %). Nurses were more present in the Zipline group (71.8 % vs. 55.1 %), while a higher proportion of physician assistants was found in the comparison group (16.7 % vs. 9.0 % in the Zipline group) (See Table 1). Fig. 1 shows the number of times each vaccine was mentioned when asked about frequent stockouts. The anti-meningococcal vaccine was the most frequently mentioned (65 times by controls and 57 times by Zipline-exposed facilities), whereas the oral polio vaccine was frequently mentioned by control facilities (50 vs. 13 times). Except for the rotavirus vaccine, all other vaccines (except DPT), were more frequently mentioned by control facilities than by Zipline served facilities.
The mean duration of the last stockout in days was 30 % lower in the Zipline group (30 days for served facilities vs. 43 days for non-served ones, P < 0.05). Almost two thirds of control facilities reported at least one missed opportunity for vaccination during the previous month (65 %) as compared to 36 % of served facilities. This relative reduction of 44 % was statistically significant (p < 0.01). Fig. 2 presents the strategies that providers reported when in need of a stocked-out vaccine. Among served facilities, 62 providers (79.5 %) reported ordering from Zipline. Interestingly, 10 providers (13.5 %) at facilities not served by Zipline also mentioned placing orders with the company, probably indirectly via facilities within the served areas. On the other hand, the most common behavior among non-served facilities was to borrow the vaccines from other facilities (48 providers; 61.5 %).
Other less frequent behaviors for the Zipline group were to borrow from other facilities (22 providers; 28.9 %) and to order from the Immunization Program (EPI) (18 providers; 24.1 %). Facilities in the comparison group less frequently ordered from the EPI (12 providers; 15.4 %) and referred the patient to other facilities (13 providers; 17.3 %).

Perceived changes and satisfaction
In terms of perceived changes in access to vaccines since aerial logistics were implemented in their areas, 71 providers (91.1 %) at served facilities perceived that access to vaccines improved considerably, and 4 (5.1 %) mentioned that access improved slightly. Only one provider (1.3 %) reported no changes.
All providers (n = 156) were asked about their satisfaction with access to vaccines in their districts using a numerical scale from 1 to 10 (higher score = better access). The average satisfaction score was 8.6 (95 % CI 8.41---8.80) for served facilities, and 7.8 (95 % CI 7.34---8.33) among the non-served ones. The difference was statistically significant (p-value < 0.05).

Factors associated with providers' satisfaction
Additional statistical analyses were conducted to explore factors associated with increased or decreased satisfaction among providers (Fig. 3). In the unadjusted analysis, t-test for independent samples showed that being served by Zipline (mean 10 % increase in satisfaction) and ordering from Zipline when stocked out (mean increase of 7.8 %) were positively associated with increased satisfaction overall. Providers' reduced satisfaction was associated with missed opportunities to vaccinate (À9.0 %), stockout of yellow fever vaccine (À7.4 %), oral polio vaccine stockout (À8.1 %), pneumococcal vaccine stockout (À11.4 %), and BCG vaccine stockout (À14.1 %).
In addition, we conducted adjusted multiple linear regression analyses to quantify the association between being served by aerial logistics and improved satisfaction ( Table 2). After adjusting for provider characteristics, including sex, age, time in the facility, and role within the healthcare team, a significant association persisted. Being served by aerial logistics was associated with almost 0.8 additional points in the 1-10 satisfaction scale.

Difference in differences models
For each of the vaccines under analysis, graphical representations were created for vaccination coverage before and after the implementation of aerial logistics for the high and low exposure groups. Pre-intervention parallel trends were assessed for each vaccine and only those that satisfied the assumption were kept for further analysis. The third dose of rotavirus vaccine and the Meningococcal vaccine were excluded from the analysis due to unparallel trends pre-exposure to aerial logistics. Fig. 4 presents, as an example, the vaccination coverage for BCG, IPV, Pentavalent, Rotavirus, and Pneumococcal vaccines, before and after the incorporation of aerial logistics (represented by the black vertical line). It is relevant to notice that the lower exposure group showed consistently higher vaccination rates, including proportions above 100 %, a fact that was validated by the Western North Region and explained as relatively inaccurate estimations of the denominators due to the use of population census projections (see the charts for all vaccines in Supplemental Materials).
For the analytical component of the DID analysis, we fitted individual models and adjusted by births and district poverty. As example, Table 3   Fig. 5 presents summary findings for each of the individual models. Significant differences in vaccination rates between high and low exposure to aerial logistics were evident in all cases (pvalue < 0.05). The second dose of pentavalent virus vaccines showed the highest pre-post difference (37.5 percentage points), followed by the 2nd dose of the pneumococcal vaccine (36.7 percentage points). All vaccines incorporated to the chart resulted in statistically significant results, setting alpha at 0.05, and the prepost differences ranged between 37.5 and 13.1 percentage points (second dose of MR vaccine).
Using published data of potential lives saved per thousand vaccinated children [8] a country specific back-of-the-envelope estimation added up to 727 (227-1,319) potential lives saved due to increased vaccination of over fifteen thousand additional children

Associations between aerial logistics and health outcomes
A violation of the parallel trends assumption precluded the fit of DID models for pneumonia and infectious diarrhea cases diagnosed at public health facilities. Therefore, we explored associations between aerial logistics and these outcomes via multiple linear regression models using only data from the years 2020 and 2021 (post-exposure). The aim for this part of the analysis was to generate hypotheses based on associations between the exposure to aerial logistics and certain diseases that can be prevented through vaccines. While there were no significant associations between the incidence of pneumonia for any age group and the exposure to aerial logistics, there was a strong inverse association between high exposure to aerial logistics and infectious diarrhea among children between the ages of 5 and 9 years old after adjusting for total births and poverty at the district level.
The model was fit with and without vaccination coverage for rotavirus and adjusted by births and district level poverty. The association remained significant in both cases, suggesting an independent mechanism beyond the increased vaccination rate.
As presented in Table 4, aerial logistics were associated with 36.5 fewer monthly cases of diarrhea in this age group, representing a reduction of 41.6 %, which also translates into approximately 430 fewer cases per year in the region.

Discussion
This study explored supply chain, programmatic, and health outcomes in relation to aerial logistics in the Western North Region of Ghana. According to providers, certain vaccines (Meningococcal, Polio, and Rotavirus) were reported as problematic in terms of stocking issues. The oral polio vaccine was disproportionately mentioned as problematic by the comparison group.
Missed opportunities to vaccinate children can have long lasting effects on vaccination coverage [9] and expose children to infectious diseases, leading to disability and mortality while also imposing a negative economic impact to the country [10]. The assessment of missed opportunities showed 44 % reduction in facilities exposed to aerial logistics, a finding that aligns with previous reports on missed opportunities for vaccination in relation to aerial logistics in Ghana [11]. This difference could be attributed to improved availability due to centralized storage, and improved access for facilities who lack or have insufficient onsite cold storage capabilities.
This scenario is also visible when respondents reported on their strategies when facing a vaccine stockout. Almost 80 % of Ziplineserved providers reported placing orders with Zipline. This strategy allows them to access the vaccines in <60 min, which prevents them from referring the patients to other facilities or rescheduling the visit, among other undesirable alternatives.
Health providers reported considerable improvements since aerial logistics were implemented in the region. Zipline-served providers graded their satisfaction significantly above the nonserved ones. Also, both being served by Zipline and ordering from the company when facing a stockout showed to be significantly associated with increased satisfaction with access to vaccines. On the other hand, frequent stockouts of diverse vaccines, and missing opportunities for vaccination were associated with decreased satisfaction.
It is important to address that stockout rates among Zipline facilities were reduced for most but not all vaccines. The rotavirus vaccine presented a unique case where facilities served by Zipline reported higher stockouts compared to non-served facilities. This discrepancy could be attributed to several factors, including a shift in supply request patterns, issues related to delivery scheduling and demand forecasting, and challenges in inventory management. In addition, Zipline generates a different scenario, in which facilities are incentivized to stock less vaccines, given the quick access to them when critical inventory points are reached. Stockouts in rotavirus vaccines among Zipline facilities could also be explained by a temporary unavailability of those vaccines at Zipline's Distri-  bution Center, reflecting the need for better demand forecasting on the side of the company. The DID model aimed to explore causality in the association between aerial logistics and vaccination coverage. Gooding et al. [9] found that stockouts of vaccines significantly decrease the number of children immunized in the short term, usually for several months. When a stockout occurs, about half of the demand is permanently lost. In our study, districts with higher presence of Zipline showed consistently positive pre-post differences in vaccination rates. Importantly, even in cases where vaccination rates in the higher exposure group did not show sharp increases, most of the pre-post effect was driven by reductions in the performance of the low exposure. This effect could likely relate to the disruptive effect of the COVID-19 pandemic, where the provision of health  services was negatively impacted across the globe. The pandemic disrupted child immunization, especially among countries with limited resources [12,13]. Restriction of movement across cities, shortage of health staff due to high demand of healthcare takers and isolation of staff who had been infected by COVID-19, among other factors contributed to a reduction in routine vaccine coverage during the pandemic [14]. In this context, and with no reason to assume a differential impact of the COVID-19 pandemic between the high and low exposure groups, observing stability or mild improvements in the high exposure group represented a meaningful and positive impact. With regards to health outcomes, the association between aerial logistics and a reduction in the incidence of infectious diarrhea in children between 5 and 9 years old potentially suggests an indirect effect. The finding of positive and significant association initially supported the hypothesis of reduced cases as a result of increased vaccination for rotavirus diarrhea. However, the stepwise fitting of the model allowed to observe that the effect was not solely captured by vaccination rates (although this independent variable showed a significant negative association). In this context, we considered alternative interpretations of the reduction in diarrhea cases in association with aerial logistics that remained significant after adjusting for vaccination coverage. A reduction in diarrhea cases could be an indirect result of the increased capacity of response at primary care centers, resulting in increased demand, increased trust in the health services and improvements in the delivery of preventive measures. This interpretation is supported by research presenting a negative association between lack of resources in rural facilities and the incidence of diarrhea and pneumonia in children [15]. Aerial logistics improves the availability of a wide range of products, including oral rehydration salts and other products to prevent and treat infectious diarrhea. In addition, an increased demand and improved access to health services would trigger more and better health interventions, including the prevention of infectious diseases, nutritional counseling, and other activities impacting the incidence of diarrhea beyond the role of vaccines. Furthermore, it is worth considering that rotavirus vaccination mainly contributes to reductions in diarrhea cases among children under the age of 2 years. Hence, it is likely that a reduction among children between five and nine years old would be a result from other factors. Among them it would be necessary to rule out changes in water and sanitation infrastructure, which are key determinants of infectious diarrhea. This type of analysis exceeded the aims of our study, and more research is needed to confirm or reject our hypothesis.

Limitations
Several limitations of this analysis need to be addressed. First, the groups of served and non-served facilities in the first part of the study showed significant demographic differences among the responding providers which were likely related to higher levels of rurality among Zipline facilities. We cannot rule out a certain amount of bias impacting the findings due to a suboptimal exchangeability between the groups.
Another relevant limitation of this analysis relies on the potential of spillovers. Certain facilities that were not captured in Zipline's fulfillment database reported receiving products from the company. This situation is not unknown and takes place when a facility that is out of the radius of operations to be served, but orders products through neighbor facilities that are within reach.   This type of spillover effect would create bias in the results by underestimating the effectiveness of aerial logistics. In addition, it is safe to assume a certain amount of response bias since primary data was collected by Zipline enumerators. This could have influenced the participants in providing more favorable responses regarding the effect of aerial logistics. Also, the absence of a dichotomous definition for the exposure at the district level presents a limitation, since both groups (high and low exposure) were composed by a mix with served and non-served facilities. This incorporates a spillover effect that could bias the estimates in any direction.
In addition, we observed unreliable vaccination coverage metrics in the lower exposure group of districts, with coverage proportions over 100 %. This is a frequent issue in several LMICs where the population denominators suffer from flawed estimations, usually not accounting for internal migrations and other demographic changes. Additionally, individuals often receive vaccines in locations different from their places of residence, influenced by factors such as availability, affordability, and other factors. Although methods exist to correct these data flaws, such as satellite imagery of nighttime radiance and triangulation with household surveys [16], these resources were not available in the Western North Region of Ghana. Nonetheless, we expect our results to remain meaningful given that the focus of the analysis was placed on the pre-post differences in coverage, and we assume the distortions to remain relatively unchanged during the timespan of our analysis.
While certain findings from this study could be considered valid for other regions in Ghana, and even for other countries with similar socio-demographic, infrastructure, and health systems' characteristics, other results are unique for the Western North Region. For example, the findings on increased access to vaccines at facilities after the adoption of aerial logistics could have a correlate in other regions or countries where rural facilities experience frequent stockout due to road conditions and other infrastructure challenges. On the other side, the analysis showed that part of the positive effects in vaccination coverage were driven by reductions in the performance of non-served districts, which could be attributed to the disruptions caused by the COVID-19 pandemic. In this regard, our findings are likely not generalizable to other geographical settings where health systems were better prepared or more resilient.

Conclusions
End-to-end aerial logistics has potentially contributed to shortening immunization gaps and seemed to have increased access to vaccines within the Western North region of Ghana. Healthcare providers served by aerial logistics were more likely to immunize children without missing opportunities for vaccination. In light of these findings, it is likely that aerial logistics have increased Ghana Health System resilience in the context of the COVID-19 pandemic, not only avoiding vaccination rates declines, but even increasing coverage across served districts. Although more research is needed to understand and quantify these effects, it was also suggested that an improved supply chain performance could generate indirect effects, enabling better health outcomes not necessarily related to the commodities being supplied. Nonetheless, the sociodemographic characteristics of the Western North Region in Ghana and its health systems and services need to be considered before extrapolating these results.

Funding
This study was funded by Zipline International LLC.