Measuring the effectiveness of maternal delivery services : A cross-sectional and qualitative study of perinatal mortality in six primary referral hospitals , Kenya

The effective performance of hospitals is critical to overall health Background: system goal achievement. Global health system performance frameworks are often used as part of global benchmarking, but not within low and middle-income countries as part of service delivery performance measurement. This study explored the utility of perinatal mortality as a measure of hospital effectiveness. A cross sectional, mixed methods study of six primary referral Methods: hospitals, differentiated by ownership, was conducted from 10th June to 9th October 2015. Monthly summary hospital data of maternal delivery services (MDS) were abstracted to determine the perinatal mortality. Tests of associations were used to correlate bed turnover, skilled staffing, method of delivery and perinatal mortality. Additionally, 40 questionnaire interviews were held with hospital board members and the management team to assess the availability of standard operating procedures (SOP) in MDS. Qualitative data was analysed thematically. All six hospitals reported having SOP in managing MDS. The average Results: perinatal mortality rate for all the hospitals was 24.63 per 1,000 live births. However, a perinatal death was 2.6 times more likely in public hospitals compared to private hospitals (29.8 vs 11.4 per 1,000 births respectively). The average caesarean section rate for all hospitals was 25.9%, but the odds of a caesarean section were 1.67 higher in a private hospital compared to a public hospital (P<0.001 95% CI: 1.58-1.77). Perinatal mortality was associated with bed turnover ratio (R squared 0.260, P=0.001), and skilled staff availability (R squared 0.064,P<0.001). The high perinatal mortality reported in public hospitals may be Discussion: due to high bed turnover and relatively low caesarean section rate. Input measures of performance such as reporting standards of care and staffing levels are not useful performance indicators. Perinatal mortality as a performance indicator may be an ideal measure of the effectiveness of hospitals.


Introduction
The hospital sector represents approximately 45-69% of government health expenditure in sub-Saharan Africa and the effective performance of the sub-sector is therefore critical to overall health system goal achievement 1 . Globally considerable efforts have been made to develop health system performance (HSP) assessment frameworks that take into consideration the peculiarities of health systems and the multiplicity of stakeholders in health with different perspectives 2 . But HSP benchmarking is often done between countries as part of a global health comparison, rather than being used at a subnational level, where policymakers in low income countries with high disease burden seek to understand how well the delivery of healthcare meets the needs of citizens 3 .
In Kenya, overall health status is measured by indicators including life expectancy, and under-five and maternal mortality 4 . However, health system performance is measured mainly through process input indicators such as health per capita spend and human resource availability 5 . This disconnect leads to poor performance accountability defined as "demonstrating and accounting for performance in the light of agreed-upon performance targets focusing on services, outputs and results" 6 . An ideal health system performance indicator would link hospital process and outcomes to overall health system effectiveness, allow for hospital comparisons, be sensitive to outcomes under the control of the health system and ensure provider accountability 6,7 .
Maternal delivery service (MDS) indicators and outcomes, such as skilled delivery levels, coverage of caesarean sections and neonatal mortality, are sensitive indicators of the effectiveness of the whole health system 8 . The core impact indicators are also well defined; however the Every Newborn Action Plan (ENAP), launched in 2014, recognised that efforts are needed to improve data quantity and quality, with only 17 countries that have a policy for reporting and reviewing stillbirths and neonatal deaths 9,10 .
Roughly one third of 363 interventions in the Kenya Essential Package for Health focus on reproductive health 11 . Despite the focus, Kenya did not meet the Millennium Development Goal (MDG) target for maternal deaths of 147 per 100,000 live births by 2015, and little advancement has been made in reducing mortality among newborns, which now accounts for 45% of all child deaths 4,12 . Facility-based delivery has gained traction as a key strategy for reducing perinatal mortality in developing countries 13 . In Kenya, healthcare provision is devolved to the 47 counties, which provide care to geographical defined populations 14 . In the delivery of MDS, primary referral hospitals are expected to provide comprehensive emergency obstetric care, which includes all basic emergency obstetric care interventions and caesarean sections 15,16 .
Efforts to reduce maternal mortality and morbidity in lowresource settings often depend on global standards and indicators to assess obstetric care. However these standards often do not take into account the local context especially in terms of skill and resource availability 17 . Moreover, using a national average does not provide timely and accurate measurements of levels and trends at local levels, which are crucial to assess progress, allow benchmarking and provide policymakers with the data to prioritize the areas of greatest need 18 .

Objective
This study explored the utility of perinatal mortality as a measure of hospital effectiveness in six primary referral hospitals in Kenya.

Study setting
A cross-sectional study of six primary referral hospitals in Kiambu and Nairobi Counties differentiated by ownership was conducted. In 2013, Kiambu County was estimated to have a population of 1,838,397 including 59,191 pregnant women 19 . In Kiambu, there were six faith-based, one private and four government hospitals. Nairobi County's population in 2013 was estimated at 3,554,261 including 172,143 pregnant women 19 . Nairobi had four faith-based, seven private and two government hospitals. Kiambu and Nairobi Counties were chosen for this study because compared to the national averages (32%), health facilities in Kiambu and Nairobi counties (40% and 48%, respectively) had above average maternal health service readiness 19 . Census data analysis of the county Maternal Mortality Ratio (MMR) estimated Kiambu and Nairobi at 230 and 212 per 100,000 live births, respectively, roughly half the national average (495 per 100,000) 20 .
All the level four health facilities, that is primary referral hospitals, were picked from the list of hospitals in the two counties. The hospitals were grouped according to ownership, public (government), not for profit, faith-based and for profit hospitals. In the two counties there were six public, eight private and ten faith-based hospitals. Hospitals that did not offer maternal delivery services were excluded. A list of all public hospitals was developed and computer generated random numbers were used to select three government hospitals, which were selected and then matched by bed capacity with two faith-based and one for profit hospitals across both counties.

Data collection
Data was collected from 10th June to 9th October 2015. Monthly summary hospital data of patients who had been admitted to the maternity unit of each selected hospital in the period 1st January 2014 -31st December 2014 were abstracted between 10th June and 9th October 2015 to determine: number of patients admitted, type of delivery, skilled staff per 1,000 deliveries, length of stay, bed capacity, bed turnover ratio, caesarean section rate, number of perinatal deaths, perinatal mortality per 1,000 live births.
Additionally, 40 questionnaire interviews were held with board members and members of the hospital management team to assess the availability of standard operating procedures in MDS. In each hospital, a minimum of three board members (including the chair, chief executive and one other), and ensuring that at least one third of members were interviewed. For each hospital management team, the medical superintendent, hospital nursing officer in charge, administrator and nurse in charge of maternity unity were interviewed. Consequently, the combined participants from the six facilities provided at least 40 interviewees -an adequate medium size sample pool of interviews (Baker and Edwards, 2012). Informed written consent was sought with interviews audio recorded except where participants were uncomfortable, only field notes were taken (Supplementary File 1). The length of stay was determined by abstracting dates of admission and discharge from 200 randomly selected patient files from each hospital.

Data analysis
Effectiveness of MDS was defined as the extent to which the hospital manages all major causes of maternal and newborn mortality as measured by the perinatal mortality rate. The World Health Organization defines perinatal mortality as the "number of stillbirths and deaths in the first week of life per 1,000 total births". The perinatal mortality rate was calculated as: (No. of perinatal deaths / total No. of births (still births + live births)) x 1000.
Correlations and tests of associations of chi-square (X2) were used to show the relationships between MDS patients, bed turnover, average length of stay, skilled delivery staff, bed capacity and patient outcomes of normal, caesarean section; and perinatal mortality. Data was analysed using the Statistical Products and Service Solutions (SPSS) and MS-Excel.
Qualitative data was analysed thematically, by manually reviewing the transcripts. Using priori codes emanating from the questionnaire, a code book (Supplementary File 2) was developed that provided a working analytical framework that was then used to code the transcripts. Two independent coders reviewed the transcripts and consequently agreed on emergent codes and resultant thematic findings.

Ethical statement
Ethical clearance was obtained from the Ethics and Research Committee of Kenyatta National Hospital and University of Nairobi (P128/03/2015). To facilitate carrying out the study, administrative consent was obtained from both Kiambu County and Nairobi County to facilitate access to the hospitals. Before starting data collection at each hospital, written consent was obtained from each facility in-charge. Respondents in the study were asked to provide informed written consent before being interviewed.

Results
The six hospitals ranged in maternity bed capacity from 13 -70 with a median of 55 beds. Total deliveries in the calendar year ranged from 381 at the 13 maternity bed private hospital to 8,279 at a 70 bed public hospital. The bed turnover ratio ranged from 29 -163 with a median of 80. The lowest number of perinatal deaths was 1, while the highest was 208. The average length of stay varied from 0.7 to 5.1 days and was associated with perinatal mortality P<0.001, 95% CI: 0.6472-0.7542) ( Table 1).
The perinatal death rate was 2.6 times higher in public hospitals (29.76 per 1,000 births) compared to private hospitals (11.39 per 1,000 births). The number of skilled delivery staff Methods of monitoring standards of care From the 40 interviews conducted the following information was found. All six hospitals reported having standard operating procedures in managing MDS. Three of the facilities, P2-Kiambu, P1-Kiambu and FBO2-Kiambu, reported having annual work plans. All six hospitals had a scheme of service and code of conduct for their employees. None reported having been inspected by the national Ministry of Health, but county health teams had visited all the hospitals in the past year except for PFP hospital. With respect to inspection by a non-MOH regulator such as the National Environmental Agency, only FBO2-Kiambu and PFP-Nairobi had interactions (Table 3).

Maintaining standards of care
All the hospitals reported having SOPs in managing maternal delivery services. All had a scheme of service and code of conduct for their employees. None reported having been inspected by the national ministry of health, but county health teams visited.
"…I want to say that the county comes to the ground very often and especially to check and monitor maternal outcomes and hold discussion…which we do with them..." (Hospital Management Team Member, FBO1-Kiambu).
Half the facilities reported having annual work plans. With respect to following external regulations only the non-government hospitals were subjected to some inspection by non ministry of health regulator such as NEMA. Of interest is that the hospitals with the worst perinatal mortalities reported to have the same number of methods to maintain standards of service as the hospital with the best mortality figures ( Table 2).
Respondents at senior management level also reported a lack of engagement with ministry of health at county and national level in strategy development of maternal delivery services.
"…The county ministry of health have not given us an opportunity to contribute towards the formation of the county health strategy...we would want to be equal partners in provision of strategy and implementation strategy...we will go for discussions… they will set a circular and we are told from now on xyz will be happening and you are not involved..." (Hospital Management Team Member, FBO2-Kiambu.

Discussion
County referral hospitals play an increasingly significant role in maternal delivery services. Kenya recorded an increase in the proportion of facility based deliveries from 44% in 2008 to 61% in 2014 20 . A total of 186,688 deliveries in 2014 occurred in such facilities, roughly 15% of all deliveries in Kenya 21 . In this study the total number of deliveries were 24,534 in 2014. The average length of stay for patients admitted for maternal delivery services was just under 2.7 days (median 2 days). This is in line with global practice; in a 92 country review of hospital the mean length of stay after child birth ranged from 1.3 to 6.6 days with the majority of women staying too short a time to receive adequate postnatal care 22 . However, the average masked considerable differences between the different hospitals, with the public hospitals discharging patients in less than 24 hours compared to two days for private hospitals. Developing countries have reported neonatal infection rates 3-20 times that of developed countries due to poor intra-partum and postnatal infection-control practices 23 . Neonatal infection in the first week of life account for 26% of neonatal deaths in sub-Saharan Africa 24 . The short time available to monitor the mother and newborn could explain the association between the high bed turnover ratio in the public hospitals and high perinatal mortality. This study reported the average perinatal mortality rate for all the hospitals at 24.63 per 1,000 live births. This is a little higher than the national average of 22.5 per 1,000 live births in 2015 25 . Yet it is reported that in sub-Saharan Africa the risk of perinatal mortality is 21% higher for home compared to facility-based deliveries 13 . In agreement with our study results, a Bangladesh study looking at whether facility delivery modified the risk of intra-partum related perinatal deaths found that the risks were higher for facility deliveries compared to home deliveries 26 . The reported perinatal mortality in the present study compared poorly with an assessment of facility quality and association with neonatal mortality in Malawi; it found an average of 17 per 1,000 live births with the newborn mortality rate of 28 per 1,000 births at low-quality facilities and of 5 per 1,000 births at the top 25% of facilities 27 . However, perinatal mortality in South Africa was reported at 33.4 deaths per 1,000 births in 2013 28 ; while a cross sectional descriptive study of eight major hospitals in Dar es Salaam in January 2009 established a perinatal mortality rate was 44/1000 births (range: 17 -147) 29 .
Public hospitals had a perinatal mortality rate 2.6 times higher than private hospital (29.8 and 11.4 per 1,000 live births respectively) in the present study. In Bangladesh, the risk of perinatal mortality in a public health facility was twice that of a private facility, with the difference being attributed to quality of care 26 . Emergency obstetric care including caesarean section has been recommended as the first priority intervention in reducing stillbirths 30 . One of the key differences observed in our study was that mothers in private hospitals were almost twice as likely to undergo a caesarean section compared to those in a public hospital. The reported difference here is consistent with studies that show that regardless of a woman's risk and contextual factors, for profit hospitals are more likely to perform caesarean sections compared to not for profit hospitals because of financial incentives 31 . As expected, the hospitals with higher caesarean section rates also reported lower perinatal mortality 32 .
Previous studies have shown strong associations between patient mortality and low staffing levels 33,34 . However, there was weak association between the ratio of skilled birth attendant and patients, indicating that perhaps primary referral hospitals had met the threshold of minimum number of staff required. This finding contrasts with a study that found that the presence of a doctor at birth reduced maternal and infant mortality 35 . Since staffing numbers weakly predicted mortality rates, it can be hypothesised that the quality of clinical decision making in not identifying mothers requiring caesarean sections was poorer in those hospitals with relatively low caesarean section rates.
All the hospitals in this study reported having at least two, with a median of three, methods of monitoring standards of care, including SOPs in managing maternal delivery services. Public hospitals in particular had more methods compared to private, though none reported having Ministry of Health oversight. This finding may appear to contradict an assessment of the quality of maternity care in an Indian metropolitan city that concluded that public hospitals practices fell short of evidence-based guidelines, while there was relative overuse of interventions in private hospitals 36 . However, it is known that reporting having certain quality improvement methodologies is not enough to lead to an outcome of high standards of care without an imbedded culture of quality improvement 37 . It is possible that a weakness in the health system building blocks of leadership and health information are the weak links in not enabling health workers to champion the process of improvement and use of best practice guidelines to monitor performance 10 .
In this study, none of the hospitals reported having been inspected by the national Ministry of Health, but county health teams had visited the three public hospitals. Despite these visits and the reporting of available protocols to ensure quality of care, the public hospitals reported worse perinatal outcomes. Quality-of-care audits have been promoted as useful in identifying and changing suboptimal care, and therefore reducing perinatal mortality; however a study in South Africa, did not demonstrate that quality-of-care audits improved perinatal mortality 28 . However, in the monitoring of external non medical regulations only the non-government hospitals were subjected to inspection by a non ministry of health regulator such as the National Environmental Agency, symptomatic of many policies in developing countries where there are often conflicting existing laws and legislations and overlapping bureaucratic mandates leading to policy implementation failure 38 . This uneven treatment of hospitals does not augur well for policymakers and hospital boards efforts to relate hospital performance to overall health system effectiveness.

Study limitations
Focusing on a specific service such as MDS allows for greater comparison between hospitals because patient heterogeneity, which can be a major factor in measuring effectiveness, is reduced 1,39,40 . This study chose mortality as the health outcome because it is relatively easy to measure and therefore likely to achieve valid results. However, the study relied on hospital records, and therefore there may have been elements of underreporting. Perinatal mortality includes live births and still births, which is a comprehensive indicator for assessing outcomes of both intrapartum and immediate post-partum care services; however without perinatal audit systems in place, there is often underreporting of stillbirths 13,41 .

Conclusions and recommendations
The study demonstrates that the average perinatal mortality in primary referral hospitals was high with considerable variation between public and private hospitals. This is despite all the hospitals reporting having various methods of maintaining standards of care. While there was considerable variance in size and patient numbers among the hospitals, staffing levels were not associated with perinatal mortality, suggesting that the quality of clinical decision making as measured by the caesarean section rate was a factor in improving outcomes. Given the heterogeneity of primary referral hospitals, the use of perinatal mortality as a performance indicator to measure the effectiveness of maternal delivery services and hold hospitals to account in relation to the entire health system is recommended.

Data availability
Dataset 1: Effectiveness of maternal delivery services and perinatal mortality in six primary referral hospitals. DOI, 10.5256/ f1000research.14862.d206064 42 The datasets generated and/or analysed during this study, other than those provided herein, are not publicly available as ethical restrictions apply to publicly sharing the qualitative interviews transcripts due to potentially identifiable information detailed in the transcripts. Excerpts of the data are however available from the corresponding author on reasonable request and approval by the Kenyatta National Hospital/University of Nairobi-Ethics Review Committee (KNH/UoN-ERC) by contacting ayah@uonbi.ac.ke or uonknh_erc@uonbi.ac.ke.

Competing interests
No competing interests were disclosed.

Grant information
The author(s) declared that no grants were involved in supporting this work.

Is the work clearly and accurately presented and does it cite the current literature? Yes
Is the study design appropriate and is the work technically sound? Yes

If applicable, is the statistical analysis and its interpretation appropriate? Partly
Are all the source data underlying the results available to ensure full reproducibility? Yes

Are the conclusions drawn adequately supported by the results? Yes
No competing interests were disclosed.

Competing Interests:
I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
The benefits of publishing with F1000Research: Your article is published within days, with no editorial bias You can publish traditional articles, null/negative results, case reports, data notes and more The peer review process is transparent and collaborative Your article is indexed in PubMed after passing peer review Dedicated customer support at every stage For pre-submission enquiries, contact research@f1000.com