Cost analysis and provider preferences of low-dose, high-frequency approach to in-service training programs in Uganda

Background Many countries in sub-Saharan Africa still face significant challenges in maternal and child health where low numbers, uneven distribution, and training deficits of the health workforce impede quality care. Low-dose, high-frequency training (LDHF), an innovative approach to in-service training, focuses on competency, team-based repetitive learning and practice in the clinical setting. In Uganda, we conducted cost analyses of local organization LDHF training programs for Post-abortion care (PAC) and Pediatric HIV to assess cost drivers and cost efficiency and compare costs to traditional workshop based training. Methods We collected costs with bottom up, activity based costing in LDHF and workshop training programs. All costs reported from a programmatic perspective in US$2015 across a two year analytic time horizon. A survey of trained providers was conducted to understand costs and incentives of participation as well as experience and training preferences. Findings PAC training with the LDHF approach cost US$29 957 corresponding to US$936 per provider; the traditional training of the same content was delivered at a total US$10 551 corresponding to US$527 per provider. Pediatric HIV training with LDHF approach cost US$41 677 or US$631 per provider; traditional training of Pediatric HIV cost US$18 656 or US$888 per provider trained. In traditional training programs, costs to providers were nearly equal to incentives given. In LDHF training programs, financial incentives and costs to participate were not equal and varied by roles and programs; all district trainers’ incentives outweighed their costs of participation, trainee incentives were higher than costs of participation in the PAC training, but in the Pediatric HIV program, trainee incentives were lower than the costs of participation. Conclusions Local training programs differ widely in applying LDHF principles to design and implementation thus leading to variation in costs and cost-efficiency. LDHF can be more cost-efficient than workshop based trainings if programs take advantage of the wider scope of trainees available for the facility-based trainings. Incentive differences between district trainers and trainees may influence participation and perception of training. The perspectives of providers participating in LDHF or traditional workshop training should be integrated when developing future programs for maximum uptake and participation for in-service training.

Globally, 66% of maternal deaths and 50% of deaths in children under the age of 5 occurred in sub-Saharan Africa in 2015, the region has seen the least improvement in maternal, newborn, and child health (MNCH) over the last two decades [1][2][3]. Increasing coverage of key interventions and achieving the associated improvements VIEWPOINTS PAPERS in outcomes for MNCH requires significant human resources; however, often there is a lack of adequately trained health care providers where they are needed most [4,5]. Uganda faces public sector workforce shortages and slow growth with only 6.3 health professionals per 10 000 people available in 2015 (far below the WHO-recommended 25 per 10 000) [6,7]. Inadequacies in number and distribution of health providers combine with deficits in pre-service and in-service training to become a complex barrier to improving MNCH through the delivery of quality clinical care.
In-service training of health care professionals is a key strategy for improving the knowledge and capacity of existing health professionals to deliver high-quality care [8]. The low-dose, high-frequency (LDHF) capacity-building approach for in-service training delivers competency-focused content in small quantities with frequent repetition over time while promoting team-based, simulation-based practice with immediate feedback, and supervised direct patient care [9]. LDHF emphasizes on-site, facility-based training/ simulation and follow-up where clinical practice occurs, minimizing disruption to service delivery while expanding the number of providers available to participate in the training and teambuilding. While evidence on LDHF training is limited [10,11], this approach may achieve similar or increased competency to previous classroom-based traditional workshop approaches while also reducing direct costs (travel refunds, per diems, hotel accommodation, and hall rental) and indirect costs (opportunity costs of leaving clinical practice and provider travel time to training).
With support from the Barr Foundation, Jhpiego-Uganda collaborated with the Association of Obstetricians and Gynecologists of Uganda (AOGU) and the Infectious Diseases Institute (IDI) of Makerere University to apply the LDHF approach to post-abortion care (PAC) and pediatric HIV/AIDS management, respectively. Both organizations conducted a comparison training utilizing a traditional workshop approach. The primary study objective was to explore the financial costs and cost-efficiency, or the cost per unit of provider trained, of utilizing the LDHF approach vs a workshop based training approach with each curriculum. The secondary objective was to examine provider preferences and perceptions of trainings through structured interviews with trainees from both deployments.

Study setting
LDHF training for PAC was implemented in 16 health facilities (Health Center IIIs, IVs, and Hospitals) in the semi-urban districts of Wakiso and Mukono in the Central Region. Traditional PAC training included trainees from 18 health facilities in another Central Region district, Mpigi. In Uganda, illegal and often unsafe abortions contribute both to the need and complexity of PAC delivery [12]. As these PAC complications contribute significantly to maternal deaths, training has been recommended for high and midlevel health workers [13,14]. PAC is restricted further by absenteeism and task-shifting, which often leaves midwives stretched to provide services [15].
LDHF training for Pediatric HIV/AIDS management was implemented in 8 health facilities across four districts: Hoima and Kibaale in Western Uganda and Kyankwanzi and Kiboga in Central Uganda. Traditional training on pediatric HIV/AIDS was delivered for a separate group of providers representing 7 other facilities within the districts. Uganda' s high HIV general population rate at 7.4% in 2012 has been the focus of significant response resources, and the most recent National Strategic Plan outlines additional steps for prevention of mother to child transmission, provision of antiretroviral therapy (ART), strategies for early detection, treatment initiation, and management of ART especially for child cases of HIV [16][17][18][19]. Failures in early diagnosis and care linkages led to 72% of children presenting, in a 2012 study, with advanced disease at the initial visit to an HIV clinic and only 41% of eligible children receiving ART in 2013 [16,20]. The Ministry of Health has specified efforts to combat Pediatric HIV with communication strategies, community mobilization, and training for health care workers [21].

Program description
The PAC program began in May 2015 with a two-month development phase. Start-up activities lasted one-month, which included District Health Team meetings, facility visits to select training sites, training of trainers (TOT) for selected district trainers, orientation and PAC content introduction for trainees, and baseline assessment data collection. Implementation spanned four months, during which the 12 district trainers led LDHF capacity-building for the 20 trainees -one-hour continuing medical education (CME) and one-hour hands-on practice -twice monthly until 6 training modules had been successfully com-VIEWPOINTS PAPERS pleted. Final evaluation of trainees included a structured written knowledge assessment proctored by AOGU staff as well as an Observed Structured Clinical Exam (OSCEs) with five simulated case stations which tested providers' skills. care LDHF training program began March 2015 with a two-month development  phase including orientation meetings with the District Health Teams and facilities, baseline assessment, and LDHF curriculum development. Start-up activities included a two-week TOT for district mentors. The implementation phase included biweekly district mentor led LDHF sessions -CME trainings for 1 hour for 4 months (8 visits) with 2 months accounting for planning, delays, and follow up -open to all staff but targeted to 54 mentees. An additional 5 mentoring sessions were scheduled for district mentors to see patients with mentees, practice application of knowledge and provide feedback. The LDHF training targeted all cadres of staff involved in pediatric HIV/AIDS care services. The endline assessment included a written knowledge test and direct observation of skills during patient care visit using a scored checklist. Table 1 defines the differences in the training programs, Table 2 details activities differences of the LDHF training approaches, and Table 3 summarizes which activities occurred between the programs' different LDHF and traditional training approaches.

Measurement
We prospectively monitored costs to determine full training program cost of the LDHF and traditional workshop approaches for each partner from March to December 2015. Costs were disaggregated into three distinct windows of time: development (two months), start-up (one month) and implementation (six months). Development and start-up activities were considered investments with an estimated useful life of 2 years and were amortized, or reduced and calculated to be used gradually over time, using a 3% discount rate; the analytic time horizon of 6 months reflects the implementation phase. Development and start-up activity costs included initial stakeholder meetings, adapting the curricula for LDHF, procuring medical supplies, holding the district trainer trainings and orientations, and costs of personnel. Implementation phase activities of the program included costs of personnel time, LDHF training sessions at facilities, evaluation of trainees as well as building and support costs. Furniture and equipment, which did not have a local cost data source, were valued by standards of the region set forth by WHO CHOICE [22]. Final costs presented in 2015 US$ for a 9-month analytic time horizon aligned with program implementation.
We additionally sought to assess opportunity costs of provider participation in trainings through a participating provider survey conducted as a quasi-independent study with Jhpiego Uganda and Johns Hopkins School of Public Health. The survey included quantitative questions on demographics and training participation and open-ended questions to elicit qualitative responses about preferences and perceptions of training.

Sampling
PAC training with the LDHF approach was delivered to 20 health providers in 16 facilities while the traditional workshop approach was utilized to train a comparator group of 20 health providers from 18 facilities. Once identified as target health facilities for post-abortion complication referrals, the

Data collection
All training program activities and resources were defined through semi-structured interviews by external costing experts with personnel from each training program. Cost data were then collected by training program personnel using a standardized Excel tool developed externally based on the reported activities outlined through semi-structured interviews with program personnel. Cost data were sourced from financial cost reports, receipts, accounting systems, or other data sources identified by the training partner staff who had participated in program costing methods instruction.
The training participant provider survey was developed, pilot tested, and programmed a tablet-based, 45-minute to 1 hour in-depth interview conducted in English with local language clarifications by Ugandan midwives trained as research assistants in qualitative data. The survey collected key data on provider income, financial incentives of trainings, and time spent on and travel to either training. Open-ended responses were captured in handwritten form by the interviewer and entered digitally into the tablet after the interview concluded to ensure interviewers focused on responents rather than devices. Data from the open-ended provider surveys were analyzed following general qualitative thematic analysis guidelines. Responses were manually coded through an iterative process; themes and sub-themes were identified and responses were re-analyzed in order to code presence of these defined concepts in each response.

Ethics approval
The institutional review board (IRB) at the Johns Hopkins University, Bloomberg School of Public Health in Baltimore, Maryland, USA determined the study protocol to be non-human subjects research. This IRB determination was shared with local partners and each conducted their own ethics review process as well.   Table 2). Of total cost for the training program, one-time development and start-up main costs were medical supply procurement (16%), District Trainer TOT and Trainee Orientation (16%), curriculum adaption (8%), personnel (7%), and initial stakeholder meetings (4%). Of the total program costs, implementation costs were driven by LDHF facility based training (11%), personnel costs (8%), supportive supervision (5%), final stakeholder meetings (5%), monitoring and evaluation (4%), endline provider evaluation (4%), and other costs ( Table 4). When amortizing development and start-up phase costs over the expected 2-year lifespan of the training, the total cost for the LDHF training was US$29 957 and the cost per person trained was US$936. For each training approach, details on total financial cost are found in Table 4 and details on total annualized costs found in Table 5.
PAC workshop training corresponded to a total financial program cost of US$10 808, including $348 for one-time start-up activities and US$10 460 for implementation. The program had two main activity cost drivers; 78% of traditional training costs were incurred during the workshop (US$8378) and 11% of costs (US$1149) were incurred during a shared final day evaluation ( Table 4). Personnel salary cost to support traditional training was low as staff had only a short time devoted to planning. Salaries were US$975 or 9% of the traditional training cost ( Table 4). After amortizing the start-up activities over the expected 2-year lifespan of the training, the estimated cost for a 6-month implementation period for traditional training was US$10 551, with a corresponding cost per training recipient of US$527 ( Table 5).

Provider incentives and costs of participating for PAC
The cost to participate -travel, transport, and opportunity cost of income that could have been generated in the time spent on the training program -for the PAC LDHF training differed between district trainers and LDHF trainees. District trainers had a total cost to participate of $192 and were given a total facilitation incentive of US$376, including the 6-day training of trainers and 6 LDHF sessions conducted. The

VIEWPOINTS PAPERS
LDHF trainees had a total cost to participate of $49 and were given a total facilitation incentive (lunch, travel) of US$66. The facilitation incentives were given by the training organization and are included in the programmatic costs listed above; the cost to participate is drawn from the provider survey data for district trainers and LDHF trainees ( Table 6).
The provider cost to participate as a trainee in the PAC workshop training was a total of $38. Workshop trainees were given a total facilitation incentive (lunch, travel) of US$34. The facilitation costs were included in the programmatic costs listed above, whereas the cost to participate indicate the opportunity cost for traditional workshop trainees.

Costs of in-service training for Pediatric HIV
Pediatric HIV/AIDS training through the 9-month LDHF program had a total financial cost of $64 712, including development and start-up activities with an estimated cost of US$31 183 (48% of overall program cost) and the cost of LDHF training session implementation which was US$33 529. The cost driver of one-time start-up activities was the two-week District Mentor TOT at US$15 000 -23% of the overall program cost. Among recurrent costs, personnel salary and benefits were the leading drivers; accounting for 39% of the total program cost ( Table 4). The provision of training itself was conducted through 8 visits by district mentors at a cost of US$5081 (8% of program cost) and 4 supportive supervision visits costing an estimated US$4396 (7% of program cost) ( Table 4). When amortizing development and startup phase costs over the expected 2-year lifespan of the training program, the 6-month implementation of the LDHF training cost an estimated US$41 677 and the cost per training recipient US$631 ( Table 5).
Pediatric HIV/AIDS training through the traditional workshop corresponded to a total of $20 220 in financial costs over the course of 1 month, including $2118 used for start-up activities and $18 102 for implementation activities. Over half of the total program budget, US$13 125 (65%), was spent on the conduct of traditional training workshop. Another cost driver during the implementation phase was personnel (11%) and endline provider assessment (9%), which took place at the facility and was conducted by staff ( Table 4). After annualizing development and start-up activities, the estimated cost for a 6-month implementation period for traditional training was US$18 656, with a corresponding cost per training recipient of US$888 ( Table 5).

Provider incentives and costs of participating in-service training for Pediatric HIV
The cost to participate -travel, transport, and opportunity cost of income that could have been generated in the time spent on the training program -for the Pediatric HIV LDHF training differed between district trainers and LDHF trainees. District trainers had a total cost to participate of US$220 and were given a total facilitation incentive of US$429, including a 10-day training of trainers and average 8 LDHF sessions conducted. The LDHF trainees had a total cost to participate of US$33 and were given a total facilitation incentive (lunch, travel) of US$17. The provider cost to participate as a trainee in the pediatric HIV workshop training was a total of US$36. Workshop trainees were given a total facilitation incentive (lunch, travel) of US$34 ( Table 7).

Provider preferences for LDHF training
The responses by training program participants provided insight on provider preferences and perspectives regarding LDHF training and traditional workshop training. Among those who had direct experience with the LDHF approach, 70% reported preferring LDHF as compared to traditional training. Most providers that preferred LDHF felt that it improved learning methods for transferring knowledge and critical skills, provided a forum for including and engaging many staff, and was beneficial for its facility location, allowance of handson learning, and interaction with supervisors for the mix of theory and practical experience (Figure 1).
Providers felt LDHF facilitated course focus in small manageable quantities and accommodated practice.
I prefer LDHF because we grasp little by little making it easy to learn, and because of the frequency, remembering is easy. -LDHF mentee, Pediatric HIV, Nursing student Respondents noted greater transfer of knowledge and skills among providers in the facility and helpfulness of frequent follow-up and repetition.
LDHF is preferred to traditional methodology, because some of the health care workers who attend the traditional trainings fail to understand the importance of passing on the same knowledge to the colleagues. -LDHF mentee for Pediatric HIV, Lab Technician Whereas traditional trainings often select only one or two providers from a health facility, LDHF was perceived to benefit more participants and promote teamwork at the facility. Another location-based benefit relayed by respondents noted that LDHF training on-site saved patients' time, avoided service provision interruptions, minimized provider absenteeism, and gave hands-on learning opportunities with outside facilitators.  Many preferred traditional training off-site due to time and concentration away from patient interruptions.

DISCUSSION
Low-dose, high-frequency training principles emphasize competency-focused content, simulation and case-based learning, appropriately spaced and brief periods of training, in-service learning, team approach and peer leadership, and supported ongoing practice [9]. In Uganda, implementing partners participated in the initial Jhpiego-led workshop on the LDHF approach and the same overall budget for the training program. However, each program evolved and established unique LDHF delivery strategies, personnel teams, district trainer preparations, numbers and cadres of participants, assessments and incentives. These differences contributed to variation in cost drivers and overall cost-efficiency of reaching a provider and improving critical competencies.
In exploring barriers to accepting LDHF, we hypothesized that providers might resist the transition from a traditional workshop to the LDHF approach for training delivery given differences in financial incentives as training per diems or allowances have been considered a supplement to low wages [23,24]. We estimated the opportunity cost to providers by measuring the daily income value from all income-generating activities as well as value of time spent in LDHF as compared to traditional training. This cost to participate in the traditional training was similar for both PAC and Pediatric HIV, US$38 and US$36, respectively. This is slightly higher than the actual incentive given to traditional training participants; thus, on average, traditional workshop trainees could in fact have earned slightly more if they had forgone the workshop and opted to continue with their other income-generating activities -US$4 among PAC group and US$2 among Pediatric HIV group. This finding is contrary to popular opinion that off-site training incentives are excessive and appealing because so much higher than normal income.
In LDHF training, shorter duration and facility-based follow-up translated to lower overall incentives received by trainees. In comparison to the traditional workshop trainee incentives, the incentives for LDHF trainees did not include per diem, accommodation, or transport. Between the LDHF approaches, incentives varied significantly depending on organization and role as trainer or trainee. Over the course of the PAC training, LDHF trainers received US$376, the LDHF trainees received US$66 on average. The Pediatric HIV program also had similar variation in total incentives; LDHF trainers received US$429, LDHF trainees received US$17 which did not cover their estimated costs of participation. These variations may address unique aspects of the LDHF programs such as travel difficulty, time requirements, or participant skill levels; however, standardization of participants' incentives based on time estimated for participation should be considered in future LDHF programs. After accounting for the cost to participate, PAC trainees earned more with incentives provided by the LDHF program than their cost to participate and more than they would have earned from participating in the traditional workshop. Pediatric HIV trainees experienced the opposite; the US$17 LDHF program incentives did not cover the average US$33 costs to participate which made them worse off than they would have been without the program and at a significant opportunity cost from their alternative participation in the traditional workshop.
Providers who preferred LDHF believed this approach at the facility led to better learning, helped patient care, engaged many staff, and allowed hands-on learning with supervisor interaction. Respondents' perspectives on potential barriers to acceptance for the LDHF approach shared key themes related to concern for increased patient wait times and workload, low staffing and scheduling issues, low attendance of all sessions, and small financial incentives. Future training program design, should closely consider these preferences and perspectives. If trainees do not accept and engage in the training, lack of participation could undermine potential improvements in competency.

Comparison with other training programs
Continuing medical education can be very effective in capacity building, however the many moderating variables and poor quality of cost-effectiveness studies which assess costs and measured outcomes for multiple training alternatives have inhibited the development of clear and concise guidelines on the optimal training delivery methods for different cadres and practicing sites [25][26][27][28]. Comparisons across the processes and structures of different LDHF training programs and traditional training alternatives has led to a more robust understanding of the feasibility of large-scale LDHF programs. LDHF training principles have been incorporated with promising results for maternal and newborn health in Uganda and Ghana as well as in high-income country hospitals for a training dose-response in higher retention of CPR skills with completion of additional on-site trainings [10,11,29]. Additionally, the literature suggests that training environments that are similar to those in which health providers practice are most efficient for the uptake of knowledge and skills and interactive, repetitive trainings improve learning as compared to di-

Disclaimers:
The views and conclusions included in this report are those of the various co-authors and do not necessarily reflect the views of their respective organizations or the organizations which supported project implementation.
Funding: This study was carried out with support provided by the Barr Foundation through the project of Establishing Pathways to Scale-Up Low Dose High Frequency Training.
Authorship contributions: Study activity activities were conceived by AL, PJ, and GC. Data collection instruments were developed by AL and revised by MW. MW collected costing data and oversaw the implementation of the provider survey with support from implementing partners. Data were compiled by MW, verified and reviewed by AL. The first draft of the manuscript was written by MW and AL, with inputs from all other co-authors. All authors approved the final version.

Competing interests:
The authors have completed the Unified Competing Interest form at www.icmje.org/ coi_disclosure.pdf (available on request from the corresponding author) and declare no competing interests.
dactic, singular trainings [30]. These benefits of LDHF were noted by providers in their perspectives of the approaches, but without acknowledging and accounting for provider preferences for peer networking, appropriate incentives, and chances to access expertise and new equipment these benefits may not be fully realized in future training programs.

Limitations of study
The PAC program provider assessments could not be used to measure attributable effect of LDHF training on knowledge and competency due to changes in the written exam, modification of competency testing methods, and different providers selected to assess at baseline vs endline. The absence of this robust training effectiveness data and the short window of program implementation pose limitations for our study. Additional challenges during LDHF on-site training -for both the PAC and Pediatric HIV programssuch as absenteeism or human resource transfers affected attendance of training sessions. On average, PAC training participants attended 73% of LDHF sessions and pediatric HIV participants attended 60% of LDHF sessions, which may have affected knowledge and competency. Training organizations did collect some facility health service delivery data, but none of these data were used to measure the effectiveness of the training program due to the short window of implementation and challenges with attribution. Beyond challenges with program evaluation, funding and program delays influenced the work of all partners and led to vastly differing implementation and assessment timeframes for the LDHF program and the comparison traditional workshop. Future research studies of training approaches should adopt a longer analytic time horizon, evaluate a possible moderating effect of health worker turnover or absenteeism on knowledge and competency gains, and optimally measure value for money along with changes in service delivery and related health outcomes such as deaths or infections averted.

CONCLUSION
Interpretation of the low-dose, high-frequency training as a concept varies greatly depending on the content, context, and implementing organization. The cost-efficiency of LDHF compared to traditional training depends on how many providers are reached and how the training is deployed. For the Pediatric HIV training, program costs per participant were lower in the LDHF approach than in the traditional approach, but the reverse was seen with the PAC training programs. The unique application of LDHF with two curricula and two different organizations has created an opportunity for collaboration and greater knowledge gained about this emerging training approach. The LDHF approach appeals to health providers and other stakeholders, but some preferences for workshop training persist. Future training programs need to understand the balance between costs of participation and incentives or training structures that encourage full participation and completion of LDHF training sessions; analyzing cost-effectiveness of these future programs will require connecting the knowledge and competency taught by the different approaches to changes in provider practice and patient health outcomes. LDHF training has potential as a cost-efficient approach to build capacity of health providers and, ultimately, improve patient care.