Data Collection on Donor-Supported Health Workers: Lessons Learned From Improving Health Workforce Data Supported by the U.S. President’s Emergency Plan for AIDS Relief (PEPFAR)


 The Global Strategy on Human Resources for Health: Workforce 2030 has called for the improvement of health workforce data and implementation of health workforce registries. It is critical to capture the health workforce supported by donors in order to have a complete understanding of the health workforce across countries. The United States Agency for International Development (USAID) developed an innovative pilot human resources for health (HRH) data collection system (including a data entry template and structured dataset) to collect HRH data for the health workforce supported by the U.S. President’s Emergency Plan for AIDS Relief (PEPFAR). The pilot system filled HRH data gaps in nine key countries, providing valuable insight for program planning. The implementation details of this exercise can be used as a case study on collecting and applying data on health workers, including those supported by donor funding.


Introduction
The fourth objective of the Global Strategy on Human Resources for Health: Workforce 2030 focused on strengthening availability of data on health workforce to guide stronger monitoring and overall accountability of strategies for health workforce. 1 There were several proposed milestones for this objective that were to be achieved by 2020. Country governments were encouraged to make progress on the development and use of health workforce registries and to share data through avenues like the National Health Workforce Accounts. 2 Bilateral and multilateral agencies were also requested to strengthen assessments of the health workforce and share available information.
When reported in full, the National Health Workforce Accounts include data on the number of active health workers in a country, such as details on their density, activity levels, demographic characteristics, and distribution. But there is far less data on health workers supported by bilateral or multilateral institutions, such as UN agencies, the Global Fund for AIDS, Tuberculosis, and Malaria, development banks, and global health initiatives such as the U.S. President's Emergency Plan for AIDS Relief (PEPFAR). 3 Staff supported by donors often bridge critical gaps to support delivery of essential or emergency services across countries, including targeted services for tuberculosis, malaria, and HIV. This has been further demonstrated during the COVID-19 pandemic, as countries have faced critical health workforce gaps in the context of the underlying global health worker shortage. 4 Donor-supported health workers have been utilized to meet surge support needs for the pandemic response, including testing, community mobilization, and COVID-19 vaccination efforts. 5 Due to the lack of standardized monitoring of health workforce activities across global health initiatives, it is di cult to fully quantify investments. Previous estimates of donor funding for human resources for health, based on available data, were estimated to be over $18.5 billion (2017 US dollars) between 1990 and 2016. 3 Data on the numbers, skill mixes, locations of work, and costs of donor-supported staff are important to better assess the totality of available workforce in countries and determine optimal utilization of workers in alignment with national strategic plans. This type of coordination is important for maximizing impact and long-term sustainability of health workforce investments. Knowing the full range of active health workers, where they are located, and associated expenditures is essential for mobilizing surge sta ng efforts to respond to urgent needs and to plan for long-term workforce needs. It's hard to plan for what you don't know exists.
For a number of years, PEPFAR has increasingly placed focus on collection of data on the number of health workers supported and amount of expenditure, but this data has been historically incomplete and lacked su cient detail to understand the range of sta ng footprint and models of sta ng supported to help achieve HIV service delivery targets. 6 , 7 , 8 PEPFAR has made substantive investment in additive health worker sta ng to advance HIV service delivery and achieve UNAIDS 90-90-90 targets. In 2019, PEPFAR reported supporting over 290,000 workers at an expenditure of more than $921 million. 9 , 10 , More comprehensive data on health worker sta ng and related expenditure, including location and types of services supported is needed to inform optimal sta ng to achieve HIV program goals and to inform long-term sustainability of investment.
In 2020, the USAID O ce of HIV/AIDS conducted a data collection exercise across a targeted subset of PEPFAR-supported countries. The exercise was intended to ll key health workforce sta ng data gaps and inform more standardized PEPFAR reporting that would initiate in 2021. The details of this reporting exercise and implementation can be used as a case study on standardizing collection and utilizing data on health workers supported by donor funding and will be discussed in more detail in the following sections.

Case Description
Developing the health workforce data collection pilot Prior to 2021, PEPFAR collected disaggregated annual HRH data, such as number of workers, high level cadres, and geographic location. However, the data was insu cient, resulting in inadequate information to guide future optimal investments and program planning. Additionally, PEPFAR nancial data collected the total sta ng investments by broad categories, such as salaries for healthcare workers, but without the context of number of individuals, cadre, or geographic location. Given the data gaps in both sets of data re ecting HRH investments and sta ng, it was di cult to comprehensively answer PEPFAR's key analytical questions regarding investments and sta ng footprints.
To address analytical questions that could not be answered with existing PEPFAR data collection and bridge gaps between data sources, USAID convened a team of staff with expertise in health workforce and expenditure reporting to develop a data collection pilot system designed to obtain more granular data on health worker sta ng supported by PEPFAR. These questions included a better understanding of sta ng footprints across PEPFAR country programs and associated remuneration, sta ng composition and distribution across geographic hierarchy, roles of staff and sta ng models, and alignment of sta ng to PEPFAR program targets for HIV services.
The USAID team then identi ed potential e ciencies in recon guring health worker sta ng data collection to be better aligned with other existing PEPFAR data streams for critical data elements, such as expenditures and geographical hierarchy, in order to be able to answer these analytical questions. Each of the nal data elements (Table 1) were included to answer the above key analytical questions, and each was designed to streamline and simplify data entry by restricting data entry options and clearly de ning each term. See the material section for a copy of the nal pilot data collection template. These key data elements then informed the design of a structured data set that would allow a data consumer/decisionmaker to examine the data at different levels of analysis, and across different observations. Interaction type (service delivery, non-service delivery) In the design of the data collection template, the USAID technical team placed signi cant emphasis on structure, in order to enable more e cient data entry and reporting. The most innovative element of this data collection approach was the format which allowed for aggregated data entry of groups of similar types of staff supported but still provided granular insight into the extensive numbers of health worker staff being supported. Previous data collection exercises had placed lesser attention on the user experience with data entry, which was time intensive for both entering data and checking for errors, and led to very large nal datasets.

Data Collection
The data collection and analysis period spanned September 2020 through January 2021 in nine countries: Botswana, Burundi, Cameroon, Côte d'Ivoire, Eswatini, Haiti, Nigeria, Tanzania, and Zimbabwe. These countries were selected due to their high level of PEPFAR HRH investment and relatively incomplete detail on that investment. Data collectors received standardized training and checklists from USAID to promote maximum data validation and data quality during the review period.

Results
After the data were collected and packaged, nine individual datasets and dashboards were produced, in addition to a global dashboard and dataset. Visualizations were mapped to PEPFAR's HRH analytical questions to aid the data consumer in using the new information to set strategic priorities. Finally, during the data dissemination period, USAID in-country managers received a unique data dashboard with tailored visualizations, which launched a discussion on the results of the fully comprehensive USAID/PEPFAR HRH landscape data that was previously incomplete or not fully understood within a given country's context.
Across the nine surveyed countries, sta ng constituted 41% of total reported program expenditures, con rming the high percentage of total expenditure going to sta ng for HIV programs. The data showed 39,595 staff at a total cost of $106,721,765. The majority of staff (75%) were reported as providing services directly to PEPFAR bene ciaries. Table 2 illustrates a sample of the global data collected, which were used to examine sta ng patterns and inform programmatic shifts. The data also showed extensive variability of sta ng patterns used across country programs to achieve HIV goals. Table 2 shows most staff were based in facilities (41.8%), compared to (34.7%) communitybased staff. This highlights the need for adequate focus not only on better understanding of roles and requirements of health workers who are based in facilities but also communities for further advancement and sustaining HIV gains. Additionally, data demonstrated that some partners used roving staff models, where staff (5.6%) are moving across facilities and not fully based or assigned to one location.
These greater insights from comprehensive sta ng and expenditure data enabled USAID decisionmakers to more effectively utilize funds for sta ng, aligned to HIV program needs, and ensure optimized service delivery to clients. Development and use of standardized HRH datasets can also enable the advancement of consistent utilization of health workforce data across countries to guide PEPFAR programming; ensure alignment with governments, including other donor investments; ensure e ciencies and lack of duplicative effort; and promote a country-led response.

Discussion And Evaluation
Lessons learned from pilot data collection The HRH data collection exercise was a novel approach to more easily gain insight into the types of staff supporting implementation of PEPFAR programs and related expenditures. Throughout data collection and the subsequent planning cycle, the USAID team kept in touch with the country teams participating in the pilot exercise to ensure correct interpretation of the guidance and data. The data was translated into a key set of analytical dashboards which were presented to countries to inform ongoing program implementation and annual planning. Finally, a data interpretation guide was developed to assist incountry USAID managers to nd the information they needed within the dashboard and follow a set of parameters for use and planning, based on the analytical question being asked.
Post-data collection, the USAID team compiled feedback on the data collection template, guidance document, and user experience to identify key considerations for future health worker data collection.
One key concern was ensuring consistency in interpretation of reporting guidance and de nitions. HRH data collection is complex, with many terms that are not well understood across the wide range of stakeholders who have a role in reporting. Among countries that participated in data collection, there were some consistent areas of misinterpretation of guidance and data elements. Areas of confusion included incorrect reporting of sta ng employment titles and location. To allow standardized reporting, templates were populated with a drop-down list of select employment titles to choose from. Some countries reported that the titles did not match country context, which risked variance in how partners reported staff. In addition, guidance speci ed that staff be reported based on location of work and roles performed as supported by PEPFAR, yet it was common for staff to be reported based on professional training and location of employee partner headquarter o ces. This required careful data quality review prior to nal submission of reporting. It highlighted the need for not only reporting guidance, but also additive tools and standard operating procedures to ensure data quality.
Another key concern was the calculation of full-time equivalence (FTE). FTE represents the percentage of a full-time worker's level of effort. A full-time worker would have an FTE of one, and a half-time worker an FTE of 0.5, and so on. For many staff entered into the data collection tool, this calculation is simple, but it can easily get more complex. For example, staff working seasonally or only partially funded by PEPFAR may have fractional FTEs which are more di cult to calculate. In reviewing the data and responding to country team questions, it became clear that a tool was needed to assist country teams in calculating their workers' FTE for some of these more complex situations. The new PEPFAR HRH requirement has an FTE calculator tool to assist those completing the template.
Impact of the data on country program implementation and planning The USAID HRH data collection pilot was a critical step towards standardizing comprehensive HRH data collection for PEPFAR programs. The goal of the exercise was to be simple, straightforward, and valuable to country teams. The data collection tool was built to align with other PEPFAR data streams, including geographies and tracking of expenditures, facilitate easy data entry, and enable standardized analysis across countries.
The USAID team shared the data and accompanying dashboards in individual meetings with country teams to ensure comprehension of the new data. Many countries used their new data to better understand their sta ng investment and inform program planning. The new data increased USAID country team comprehension of the sta ng footprint of PEPFAR programs and has allowed stronger links to be drawn from sta ng to HIV service delivery indicators, such as number of HIV patients on ART treatment and number of positive HIV tests in a country. Additionally, this data has been used in discussions with national governments for long-term sta ng decisions. It has highlighted the value of greater HRH data availability and analyses to inform program implementation and planning for advancing and sustaining HIV epidemic control.

Impact of the pilot exercise on PEPFAR HRH reporting
The pilot exercise con rmed proof of concept in HRH data collection that resulted in alignment with other PEPFAR data systems, including expenditure reporting, and ensured all required data elements for strategic HRH planning were collected with reduced reporting burden and packaged in a tidy, structured dataset to facilitate easier building of analytical products.
Starting in October 2021, PEPFAR is collecting annual HRH data on all PEPFAR supported staff, with the successful launch of this pilot activity informing the new framework, so previous data gaps are addressed and pitfalls are identi ed from the beginning. Lessons learned (see above) from this pilot exercise informed the PEPFAR-wide design and rollout of the revised, routine HRH reporting requirement. 11

Conclusion
This exercise provided valuable insights on data collection of donor-supported health workers. The data collected provided a comprehensive understanding of the PEPFAR-funded health workforce sta ng in USAID programs, which has demonstrated great utility in informing more optimal impact and long-term sustainability of health worker investments. Donors such as PEPFAR make signi cant investments in support of health worker sta ng in order to meet immediate health demands across countries. This support has been critical to meet surge sta ng required during the COVID-19 pandemic. Having a more comprehensive understanding of the donor-supported health workforce is important for yielding greater accountability and shared understanding of full sta ng available and enabling a more e cient and