Variation in quality of primary-care services in Kenya, Malawi, Namibia, Rwanda, Senegal, Uganda and the United Republic of Tanzania

Abstract Objective To analyse factors affecting variations in the observed quality of antenatal and sick-child care in primary-care facilities in seven African countries. Methods We pooled nationally representative data from service provision assessment surveys of health facilities in Kenya, Malawi, Namibia, Rwanda, Senegal, Uganda and the United Republic of Tanzania (survey year range: 2006–2014). Based on World Health Organization protocols, we created indices of process quality for antenatal care (first visits) and for sick-child visits. We assessed national, facility, provider and patient factors that might explain variations in quality of care, using separate multilevel regression models of quality for each service. Findings Data were available for 2594 and 11  402 observations of clinical consultations for antenatal care and sick children, respectively. Overall, health-care providers performed a mean of 62.2% (interquartile range, IQR: 50.0 to 75.0) of eight recommended antenatal care actions and 54.5% (IQR: 33.3 to 66.7) of nine sick-child care actions at observed visits. Quality of antenatal care was higher in better-staffed and -equipped facilities and lower for physicians and clinical officers than nurses. Experienced providers and those in better-managed facilities provided higher quality sick-child care, with no differences between physicians and nurses or between better- and less-equipped clinics. Private facilities outperformed public facilities. Country differences were more influential in explaining variance in quality than all other factors combined. Conclusion The quality of two essential primary-care services for women and children was weak and varied across and within the countries. Analysis of reasons for variations in quality could identify strategies for improving care.


Introduction
Although substantial progress has been made in reducing child and maternal deaths in the past 15 years, many women and children in low-and middle-income countries continue to die of avertable causes. 1 To stimulate a concerted effort to narrow the gap between rich and poor countries, the United Nation's sustainable development goals (SDGs) include new targets to reduce maternal mortality to less than 70 per 100 000 live births and to reduce deaths of children younger than five years to 25 per 1000 live births by 2030. 2 The global strategy to date has been to promote higher utilization of health services to treat the diseases that contribute most to mortality among children and women. 3 Integrated management of childhood illness -a simplified approach for diagnosing and treating malaria, diarrhoea and pneumonia -is one such strategy. 4,5 Another approach is antenatal care, which can provide important health benefits to the pregnant woman (e.g. malaria treatment and diagnosis of human immunodeficiency virus infection) and her child (e.g. tetanus toxoid vaccination). 6 Coverage of these and other essential health services is increasing, 1 aided by global initiatives to measure and compare coverage across countries, such as the Countdown to 2015 initiative: a multi-country collaboration to collect and publish comparable data. 7 These data have informed programmes to promote utilization of health care, by providing information, insurance schemes and utilization incentives for communities, among other means. [8][9][10] Less is known about the quality of health services received by women and children when they reach a health-care facility. Some studies have pointed to quality deficits in the delivery of basic maternal and child-health services. [11][12][13][14] However, unlike for coverage, there is no systematic examination of health-care quality that would permit benchmarking and tracking of progress over time. This is increasingly important as there is renewed interest in strengthening the role of integrated primary care in countries where investments have predominantly targeted communicable diseases. [15][16][17][18] A functioning primary-care service offers continuous care via preventive and curative services 18 and is therefore wellpositioned to deal with the double burden of infectious and chronic diseases now facing low-and middle-income countries. While primary-care performance is regularly measured in wealthier countries, there are almost no data from lower-income regions. 19,20 In this paper, we analyse the variation in the quality of processes of care in health facilities in seven countries in sub-Saharan Africa for two primary-care services: (i) antenatal care and (ii) care of sick children, using observations of clinical care, a gold standard measure of process quality. The results will inform policy-makers about current performance and provide a starting point for a broader discussion of quality measurement in the SDG era.

Research
Primary care in sub-Saharan Africa Margaret E Kruk et al.

Study sample
The study sample was drawn from service provision assessment surveys conducted by the demographic and health survey programme. The surveys include four instruments: audits of service readiness in health-care facilities; interviews with health-care providers; direct observations of consultations; and exit interviews with patients. We focused on sub-Saharan Africa and included all surveys between 2006 and 2014 that had data on observations of antenatal and sick-child care (Kenya, 2010;Malawi, 2013;Namibia, 2009;Rwanda, 2007;Senegal, 2012Senegal, -2014Uganda, 2007; the United Republic of Tanzania, 2006; Table 1). These surveys use nationally representative samples, or censuses or near censuses (in Malawi, Namibia and Rwanda), of the country's health facilities. [24][25][26][27][28][29][30][31] The resulting data provide the most detailed, nationally representative information available on primary-care quality.
Within health-care facilities, up to five clients per provider per clinical area were selected for observation using systematic random sampling. Observers, who were specially trained researchers, assessed: (i) first visits or follow-up visits for antenatal care; and (ii) consultations for children aged five years or younger presenting with illness.
For this analysis we selected data from all primary-care facilities, defined as any facility that was not designated as a hospital by the country. The antenatal care analysis was restricted to first visits, as those had substantially more of the recommended clinical content than did follow-up visits. We calculated sampling weights for each observed visit to adjust for different likelihood of facility and patient selection into the sample. The final weighted results are approximately representative of women and children seen in the health system during the survey timeframe.

Study outcomes
Using guidelines from the World Health Organization (WHO), 32,33 we identified essential elements of clinical care for mothers and infants, and matched these to the indicators available in the service provision assessment surveys. We then created composite quality indices of clinical care for the two services. Each index had items covering history-taking, physical examination, diagnosis, and counselling and management actions that should be done for all patients, regardless of the reason for presentation or the local epidemiology. There were eight items for antenatal care and nine for sick-child care ( Table 2). We calculated the percentage of items fulfilled per visit, to provide a continuous quality process score scaled from 0 to 100, whereby a higher score corresponded to greater adherence to the recommended clinical actions.

Covariates
To construct an explanatory model for observed quality, we drew on Rowe's framework for explaining the performance of health-care workers. 34 This framework includes factors related to patients, providers and facilities, as well as the broader health system and community context. 34 We identified covariates in the data that corresponded to the key constructs in the Rowe framework at the visit, provider, facility and country levels. Visit-level covariates included: p a t i e n t's a g e ( t e e n a g e w o m a n [age < 20 years] at antenatal care visits; infant [age < 12 months] at sick-child visits), educational attainment of the caregiver present at the visit, and case complexity (late first visit [≥ 24 weeks gestation] for antenatal care; number of complaints for sick children). Patient-level data came from patient exit interviews and observations. We identified afternoon visits to assess the influence of time of day on provider performance.
For health-care providers, three measures were available: cadre (physician, nurse, or nursing assistant/other), experience (completed preservice training > 5 years previously) and supportive environment. Physicians included medical doctors as well as clinical officers and associate medical officers (paraprofes-sionals with authority to diagnose and treat routine illness). Nursing classifications varied too much across countries to consistently distinguish these further; the category of nurse included all midwives. The final category included nursing aides, assistants and any other personnel (e.g. counsellors and social workers). Providers were considered to have a supportive environment if they reported at least one of the following: clear job description, knowledge of opportunities for promotion or availability of performance incentives.
Facility covariates included: ownership (private versus government) and measures of general service provision readiness (number of services provided; number of clinical staff per bed [small facilities without beds were assigned a value of one to permit comparison of staffing with larger facilities]; equipment availability; facility infrastructure; and facility management practices). For the last three measures we created indices composed of multiple items; details are available from the corresponding author. We calculated the natural log of the number of services offered by the facility and staff per bed for easier interpretation of the results. Finally, as this was a pooled analysis of all seven countries, we used an indicator variable for country as a proxy for national factors that may influence quality.

Statistical analysis
To compare quality across countries, we calculated mean and interquartile range (IQR) for antenatal care and sick-child care quality. For each process quality score and explanatory covariate, we estimated the mean and standard deviation (SD), weighted based on client sampling weights. Bivariate analyses were then performed for quality on each covariate. Variables were included in the final model if they were statistically significant at the P < 0.10 level for at least one type of visit (antenatal care or sick-child visits) or were conceptually important. We estimated two-level random intercept regression models with visits nested within providers for each service. The large proportion of clinics with a single provider prevented construction of a three-level model (visit, provider and clinic). Estimates of between-provider difference thus include both facility differences and provider differences. Malawi served as the reference category as it was the poorest country in this study (Table 1). To test the impact of the Hawthorne effect on the results (a change in behaviour as a result of being observed 35 ), we conducted sensitivity analysis without the first observation per provider within each service. More details are available from the corresponding author.
We calculated the percentage of variation in quality explained by the covariates as the difference in variance between the adjusted model and the null model divided by the null model variance. We quantified the explained variance for each group of covariates (country, facility, provider and visit) by progressively adding blocks of variables to the multilevel random intercept models. Regression analyses are unweighted due to adjustment for factors associated with respondent selection; models are clustered by facility.
All statistical analyses were carried out using Stata version 14.1 (Stata-Corp, College Station, United States of America).

Results
Across the seven countries, 4613 of 4798 sampled facilities were successfully assessed (96.1%); 2902 of these facilities were primary-care facilities with at least one clinical observation in antenatal or sick-child care. A total  . Score range is 0-100, with higher scores corresponding to greater adherence to the recommended clinical actions. b Other category includes counsellors and social workers. c Provider-supportive environment is a binary indicator that takes the value of one if any of three elements are present: clear job description, knowledge of opportunities for promotion, or availability of performance incentives. d Staff per bed adds one to the bed count in each facility to include facilities reporting zero beds in the analysis. e Infrastructure index is the proportion of 20 supply-side factors present in each facility, including the availability of a functional ambulance and uninterrupted essential drug supply over the past month. f Equipment index is the proportion of equipment essential for visits for antenatal care (seven items, e.g. functioning fetal stethoscope and weighing scale) and sick children (four items, e.g. functioning thermometer and stethoscope) observed in each facility. g Management index is the proportion of 10 indicators of facility management practices fulfilled in each facility, including regular quality assurance reviews and supervisory visits. Notes: Data were pooled from service provision assessment surveys of health facilities in each country (survey year range: 2006-2014). Descriptive statistics were weighted using client sampling probabilities in the surveys. Total number of observations for antenatal care were 2594 unweighted, 2638 weighted; for sick-child care were 11 402 unweighted, 11 814 weighted. of 2594 first antenatal care visits and 11 402 clinical consultations for children younger than 5 years were fully observed in primary-care facilities. These visits were to 3902 unique providers (1077 for antenatal care and 3144 for sick-child care; 319 providers were observed in both services).
Of the visits for antenatal care, 430 (16.7%) were by teenage women and over half (1268; 52.9%) by those presenting late for the first visit at that facility (Table 3). Of the observed sick-child visits, 4073 (35.1%) were for infants; the average child presented with close to three symptoms. Three-quarters of antenatal visits and almost half of sickchild visits were handled by nurses; only 74 (2.8%) of antenatal visits but 3565 (30.5%) of sick-child visits were dealt with by physicians or clinical/associate medical officers. Overall, 11 387 of 14 452 (78.8%) of patient visits were to public health-care facilities.
Overall quality of care was low, with a mean score of 62.2% (IQR: 50.0 to 75.0) for antenatal visits and 54.5% (IQR: 33.3 to 66.7) for sick-child care visits. Quality varied considerably across the countries surveyed, as shown in the comparison of quality scores by country in Fig. 1. The quality of care for pregnant women was typically higher than for sick children. Fig. 2 displays the variance in clinical quality when providers were grouped according to quartiles of average quality by country. Variability within quartiles was not consistently associated with average quality: both poor and good providers displayed considerable variability. Table 4 presents the results of the fully adjusted, multivariable, random intercept regression models. The analytical sample included 2173 antenatal visits (83.8%) and 10 646 sick-child visits (93.4%) with complete data on covariates. For antenatal care, higherrisk women received significantly worse care than other women (−1.9 percentage points out of 100 for teenage mothers and −1.6 percentage points for late first visits). Within providers, the only significant association was underperformance, relative to nurses, of the small number of clinicians providing antenatal care (−8.3 percentage points difference). Quality of care scores were higher at private facilities (4.5 percentage points better than public facilities), at facilities with more staff (2.0 percentage points increase for each doubling of staff per bed), and at facilities with better infrastructure and equipment (differences of 9.8 and 16.5 percentage points for a score of 1 versus 0 on these indices). Using Malawi as the reference, the six other countries had significantly higher antenatal care quality, with differences up to 33.4 percentage points for Kenya and 32.5 percentage points for Namibia, the two highest income countries in this study. The intra-class correlation in the unadjusted model was 81.4%, indicating relatively low variability in quality of care within providers (i.e. between visits).
Results for quality of sick-child care differed in several ways (Table 4). Higher-risk children, i.e. infants and those with more symptoms, received better care (differences of 2.0 and 2.6 percentage points, respectively) than other children. Physicians provided similar quality of care relative to nurses, with assistants and aides significantly worse than nurses (−3.1 percentage points). More-experienced providers provided significantly higher quality care. Of facility characteristics, only private facilities and better management practices were significantly associated with higher quality of care. All other countries except Senegal provided higher quality care on average than Malawi, notably Namibia and Uganda (> 20 percentage points higher). The intraclass correlation in the unadjusted model was 59.0%, evidence of moderate between-visit variability in providers' quality of care.
Overall, the full models explained 37% of the total variance in antenatal care and 20% of the variance in sickchild care. Over 80% of explained variance in each service was due to the country variable. Only facility characteristics for antenatal care (19% of explained variance) and visit characteristics for sick-child care (10% of explained variance) contributed meaningfully to the model's explanatory power. Findings for both services were largely unchanged in sensitivity analysis excluding first observations. Variance estimates and sensitivity analysis are available from the corresponding author. Fig. 3 depicts the scope for improvement in quality in each country. Enabling providers to provide quality of care at their own peak performance would result in gains of over 5% in antenatal care quality and 10% in sick-child care quality. Bringing all visits up to the standard of the top quartile of facilities would result in linear increases of over

Discussion
In this analysis of nearly 14 000 clinical consultations in seven countries, we found relatively weak quality of care for pregnant women and sick children: providers performed half to two thirds of a minimal set of recommended clinical actions. Providers for antenatal care were primarily nurses, whereas sick children were seen by both nurses and clinical officers. Nursing assistants conducted one in five visits for both services. Other studies in similar settings, often done in the context of quality improvement, have found that the care of sick children was weak. 36,37 Performance differed substantially across countries, not only due to differences in national wealth or health-worker supply. For example, Kenya's average antenatal care quality was comparable to Namibia's despite having half the per capita number of physicians and onequarter the national income. Uganda, with one-tenth the national income and one-third the physicians of Namibia, performed nearly as well in sick-child care. Quality was not consistent across services; countries with strong performance in antenatal care did not always do well in sick-child care. Factors such as national leadership and governance, health-care budget allocation, financing schemes, the role of donors, geography and the quality of basic and medical education are likely to be important in explaining the differences. 38 While there are no comparable cross-national, quality data from lower-income countries, data from the Organisation for Economic Co-operation and Development also show heterogeneous performance that is not linked to national wealth or health-system resources in high-income countries. 39 This is an important area for further study, as country indicators accounted for the majority of the explained quality variation in our analysis.
We analysed a large range of factors previously found to influence quality of care. Antenatal care quality was strongly associated with facility factors: staffing, infrastructure and equipment. In contrast, sick-child care was related not to facility equipment but to better facility management. Private facilities outperformed public facilities for both services, even after controlling for a range of facility inputs and management practices. We were unable to assess for-profit and notfor-profit facilities separately as not all countries collected these data; this would be an important area for future work.
Higher qualifications of healthcare providers did not guarantee better care. Clinical officers and physicians performed substantially worse than nurses in antenatal care and did no better than nurses in sick-child care. Previous research has shown mixed results when comparing nurses with other health-care professionals. 40,41 Nursing assistants and aides were not as competent in sick-child care as physicians and nurses. More experienced health-care providers performed better in sick-child care. Although we included a variable for supportive job environment, including opportunities for promotion, we did not have information on intrinsic motivation, quality of supervision or remuneration, any of which may have influenced performance. 34 While the majority of the variation in performance (81.4% for antenatal care  and 59.0% for sick-child care) stemmed from differences in quality of care across providers (including their country and clinic factors), individual providers also gave different care to different patients.
Patient and visit factors were influential in explaining the quality of sick-child care in particular. Care quality was higher for younger and sicker children. Antenatal care was weaker for teenage patients and those presenting after 24 weeks of pregnancy (potentially due to prior antenatal care). While a patient's specific presentation and case severity can alter providers' clinical actions in a consultation, the items included in our quality indexes represented basic medical procedures that should have been done for all patients. 42 The best performance by providers and clinics in each country suggest considerable scope for national improvement in quality. All countries in this study could make large gains in quality if providers performed at their best and if all facilities performed at the level of the top quarter of clinics. The visit-to-visit variation within individual providers may be decreased by better adherence to guidelines and intensive supervision to promote more consistent performance of essential functions.
This analysis has several limitations. Although direct observation is the gold standard of clinical quality measurement, it is subject to the Hawthorne effect and observer error. We did not find evidence that the Hawthorne effect materially influenced the intra-provider variation within the relatively small number of observations per provider. 35 However, we cannot rule out mistakes in the observers' recording of clinical care contributing to between-visit variance; some researchers consider between-visit variation a nuisance parameter reflecting statistical noise. 43 Our quality indices were defined based on items asked in all service provision assessment surveys and matched with evidence-based guidelines. While other items could be added to provide a more complete assessment of quality, these indices represent a minimal level of quality. We were not able to link the process of clinical care to patient outcomes. Measurement of provider and facility characteristics differed across the service provision assessment surveys. For example, we were not able to investigate differences within classifications of nurses due to lack of disaggregated data in some countries. The data did not contain contextual factors that may contribute to variation in the processes of care, such as local epidemiology, and community factors that may influence clinic performance, such as accountability charters or strong local district management. To limit the role of such factors in this analysis, we constructed quality metrics limited to a Quality coefficient is the expected difference in visit quality (scale 0 to 100) given a 1 unit difference in the exposure, holding all other covariates constant. b n is the number of observations with complete data on covariates. Notes: Data were pooled from service provision assessment surveys of health facilities in each country (survey year range: 2006-2014). All standard errors are clustered by facility. Information on indices (e.g. provider support, infrastructure) is in the notes to Table 3. Intraclass correlation between visits for providers in the unadjusted model was 81.4% for antenatal care and 59.0% for sick-child care.
only the most essential clinical functions. In future, it would be valuable to assess more aspects of compliance with WHO guidelines 32,33 on clinical care for mothers and infants. Finally, the service provision assessment contains observations for only a few services. Whether quality differs for other primary-care services should be explored.
Analysis of variations in quality of care processes can lay the groundwork for quality improvement. 44 Equipment, staffing and management factors affected quality of care and these provide concrete areas for improvement. However, the substantial variation in quality of care across the study countries after accounting for these measured factors should prompt examination of national standards for professional education of health-care providers and health-system policies to support quality care. The finding that quality also varied across clinics in the same country and even among consultations done by the same provider suggests that identifying and replicating local best practices will be valuable. Efforts are under way to design better models of antenatal care and to test innovations in primary care. [45][46][47] The first step to closing the quality gap is to measure it. Governments of lower-income countries that want to enhance their health outcomes and provide better services to citizens can use these data as a baseline for improvement. Global partners should support the means to fund comparative analyses, develop efficient measures, assist in improving of routine information systems, and train local health system researchers. Reaching the SDGs will require a shared commitment to this new agenda. ■ Competing interests: None declared.  из девяти мероприятий по уходу за больными детьми во время посещений, совершенных под наблюдением исследователей. Качество дородового ухода было выше в медицинских учреждениях, которые были лучше укомплектованы персоналом и оборудованием, и ниже у врачей, чем у медсестер. Опытные поставщики медицинских услуг в медицинских учреждениях с лучшей системой управления обеспечивали уход за больными детьми более высокого качества без каких-либо различий между врачами и медсестрами или между клиниками с лучшим и худшим оснащением. Частные медицинские учреждения превзошли общественные медицинские учреждения. Страновые особенности были более значимы при объяснении различий в качестве, чем все остальные факторы, вместе взятые. Вывод Качество двух основных услуг первичной медикосанитарной помощи для женщин и детей было низким и варьировалось как между странами, так и внутри отдельной страны. Анализ причин различий в качестве позволил бы определить стратегии для усовершенствования медикосанитарной помощи.