Use of a participatory quality assessment and improvement tool for maternal and neonatal hospital care. Part 2: Review of the results of quality cycles and of factors influencing change

Background Information about the use of the findings of quality assessments in maternal and neonatal (MN) care is lacking and the development of tools capable to effectively address quality gaps is a key priority. Furthermore, little is known about factors that act as barriers or facilitators to change at facility level. Based on the extensive experience made with the WHO Quality Assessment and Improvement MN (QA/QI MN) tool, an overview is provided of the improvements in quality of care (QoC) which were obtained over time and of the factors influencing change. Methods All documented reports on the implementation of the WHO QA/QI MN tool were searched and screened for inclusion. Reports were considered if bringing evidence from both the baseline assessment and the reassessment. Changes were considered in four domains: maternal care, neonatal care, infrastructure and policies, with reference made to WHO maternal and neonatal care standards. The observed improvements were categorized according to intensity and extent across the sample of health facilities. Factors influencing change were categorized into internal and external and further classified as barriers or facilitators. Results Changes were documented after an average period of 1.2 years from first assessment in 27 facilities belonging to 9 different countries in Central and Eastern Europe (3), Central Asia (3), sub-Saharan Africa (2) and Latin America (1). Improvements were observed in all areas of care but were greater and more frequently observed in areas related to appropriate case management and respectful care for both mothers and newborns. Although widespread across most facilities and countries, the observed improvements were not covering all the quality gaps observed at the baseline assessment nor were always sufficient to achieve standard care. Factors facilitating change as well as barriers were mainly related to the capacity of the managers and head of units to involve and motivate their staff members. Conclusions The use of WHO QA/QI MN tool proved effective in promoting significant changes in quality of care. The review of observed improvements and of factors influencing change at facility level shows that participatory assessment tools that promote a constructive dialogue with hospital managers and staff and support them in acquiring capacity in this role are crucial to implement effective quality cycles.


VIEWPOINTS PAPERS
To respond to the widely documented gaps in maternal and neonatal hospital care (MNHC), international agencies and country health authorities have strengthened their efforts to improve quality through action at system as well as at facility level [1][2][3][4][5]. However, while quality assessments have been intensified using a variety of tools, there is still scarce evidence about the results of quality improvement cycles in MNHC over the whole continuum of care from admission to discharge [6][7][8]. A review of existing quality assessment tools focusing on MNCH highlighted that information about the use of the collected information for quality improvement, let alone about the results of action taken, was lacking and indicated the development of assessment tools capable to help addressing important quality gaps as a key priority for research [9]. Furthermore, while substantial progress was made in identifying factors that determine quality at system level little is known about factors that may represent barriers or facilitators to change at facility level [8,10]. This makes it difficult for national health authorities, international agencies, development partners and local managers to respond to the call to identify and implement effective approaches to improve the quality of MN hospital care and ensure adherence to WHO MN standards [11].
The experience with the WHO Quality Assessment and quality improvement maternal and neonatal (QA/QI MN) tool [12] is the largest documented so far in maternal and neonatal quality of care (QoC) assessment, as it covers 25 countries and a wide variety of health systems [13]. The results of the quality cycles initiated through this approach have been documented in published papers or in unpublished reports. A few of these assessments included also an analysis of factors influencing the observed changes in quality of care [14][15][16]. Based on this rich body of information, we made an overview of the improvements in QoC which were documented in subsequent assessments and of the factors influencing change.

Inclusion and exclusion criteria
All documented reports on the implementation of the WHO QA/QI MN tool were first searched and then screened for inclusion in this review. They include papers published in peer reviewed journals, reports retrievable from WHO or other Agencies' websites. In order to capture the changes in QoC, reports were considered if bringing evidence from both the baseline assessment and the reassessment after a period of time, with a detailed list of areas of care where changes were observed. Within this subset of reports, information related to factors influencing the observed changes was only considered if: a) resulting from an explicit attempt to capture this dynamic as a study objective; b) based on interviews, either face-to-face or through web-based forms with key staff and managers of the assessed facilities; and c) if factors could be identified as internal or external to the facility and acting as barriers or facilitators.

Retrieval of information and methods for the analysis of QI improvements cycles
In order to ensure consistency with the way findings related to main quality gaps in MNHC as assessed by the WHO QA/QI MN tool were reported [13], the same framework was used to describe results. The framework differentiates four main components (maternal care, neonatal care, infrastructure and policies) with reference to WHO maternal and neonatal care standards [11]. The information retrieved from each report included country, reach of the assessment, year of the assessment and reassessment, involved authorities and agencies, composition of assessment team, list of observed changes in MNHC at reassessment.
The observed improvements were categorized according to intensity of change and the extent of the observed changes across the sample of health facilities. Within each report, observed improvements in quality of care were analysed for their intensity with reference to the scoring system used in the WHO QA/QI MN tool [12]. Intensity was considered moderate when the improvement observed in a specific area as compared to the baseline assessment was not sufficient to achieve a score corresponding to recommended standard care; substantial when it allowed to achieve a score corresponding to recommended standard care. To provide a semiquantitative idea of the extent of the observed changes, we used countries as unit of observation rather than facilities, as the number of facilities included for each VIEWPOINTS PAPERS country was quite variable. This approach led to four possible categories, which were used for reporting the evidence of the review: 1) moderate change in a minority of countries; 2) moderate change in a majority of countries; 3) substantial change in a minority of countries; 4) substantial change in a majority of countries.
This semiquantitative approach still allows to provide a measure of the observed changes and reduces the risk of over or under-representing the extension of improvement which would result from taking into account the facilities as unit of observation.

Retrieval of information and methods for the analysis of factors influencing change
When the analysis of factors influencing change was available, data on the reported factors were extracted and factors were categorized into internal and external. The first category includes elements and dynamics internal to the facilities while the second includes factors depending on the outside environment (health system, and beyond). These factors were further classified as barriers or facilitators, ie, elements or dynamics that were hampering or driving quality improvement, and the information was ultimately classified into four categories: a) factors internal to the assessed facility that facilitated improvements; b) factors internal to the assessed facility that hampered improvements; c) factors external to the assessed facility that facilitated improvements; 4) factors external to the assessed facility that hampered improvements. Factors relevant to at least two countries were identified and reported. This analysis was made by two of the Authors (GT and AB).

RESULTS
Overall, a reassessment was done and could be documented in 27 facilities belonging to 9 different countries, belonging to Central and Eastern Europe (3 countries), Central Asia (3), sub-Saharan Africa (2) and Latin America (1).
Time elapsing from first to the second assessment varied from around a year to almost 4 years. In all countries the reassessment was carried out by a national multidisciplinary team with the supervision of an international team [12,13]. In most countries the exercise was led by international agencies in collaboration with national authorities, in two by NGOs in collaboration with local authorities. In one the assessment was made within a project funded by a national research institution. Table 1 provides an overview of countries, dates, number of hospitals involved and organizing agencies.

Observed quality improvement
The following tables describe the improvements in the quality of MN care which were observed at reassessment as compared to baseline assessments. Results are framed according to maternal care, neonatal care, infrastructure and policies, each presented in a separate table (Table 2, Table 3, Table 4 and Table 5).  Increased use of epidural anaesthesia.

"Actions to address inappropriate interventions and the increasing caesarean section rate have been implemented, for example with adoption of Robson's classification"
"Increased accessibility to regional analgesia for CS" "Prophylactic administration of antibiotics was implemented to prevent infections after CS, instead of previous long (3)(4)(5)     For each main area where improvements were observed, a semiquantitative description of the intensity and extent of the observed changes is given according to the four categories described in the Methods section. To provide a more detailed and granular description of changes, quotes directly taken from reports are reported.

Factors influencing change, barriers and facilitators
Three studies met the selection criteria to be included in the review of factors influencing change ( Table 6).
The first study was carried out in Uzbekistan in 2015 and included 4 regional perinatal centres [14]. The second was carried out in the State of Pernambuco, Brazil, in 2016 and included 6 maternity hospitals [15]. The third was carried out in Ethiopia in 2016 and involved a single district hospital [16]. All three studies were made at the time of reassessment to explore factors influencing change. In the case of Uzbekistan and Brazil, the time elapsed between baseline evaluation and reassessment was one year, whereas in the case of Ethiopia the reassessment occurred after 4 years. All studies were made through interviews with managers and key staff, although with a diversity of methods and sample. Table 7 and Table 8 list factors emerging in all three studies or at least in two out of three, as facilitating or hampering improvement VIEWPOINTS PAPERS of quality. Time constraint was not reported as an obstacle by any of the interviewees. On the contrary, most professionals reported that in order to maintain commitment, the time elapsing between assessment and reassessment should not be too long.

DISCUSSION
To our knowledge this is the first report documenting, in a variety of low and middle income countries and health system settings and by using a homogeneous approach, improvements in the quality of MN hospital care along the continuum of care from admission to discharge. We believe that the review provides several useful indications about the observed improvements and their adherence to WHO standards [11], as well as about key factors influencing change.
Improvements were observed in all areas of care and were particularly important and more frequently observed in the areas corresponding to appropriate case management and respectful care for both mothers and newborns (WHO standards 1 to 6) than in those related to staffing and infrastructure (WHO standards 7 and 8). This is not surprising since changes in the latter areas are more difficult to be achieved by the hospital management and key staff alone, as they usually require action at higher health system levels. Although remarkable and widespread across most facilities and countries, the observed improvements did not cover all the quality gaps that were observed at the baseline assessment nor were they always sufficient to achieve standard care. This could be attributed to the relative shortness of the period elapsing from the baseline assessment to reassessment. However, none of the three studies investigating factors influencing change reported time constraints as a major obstacle, and many assessors involved in reassessment confirmed that change either initiated soon after the assessment or it did not occur. This further underlines the importance of including the development of a draft action plan with timelines and responsibilities to address priority quality gaps as part of the baseline assessment visit, since this allows to use the momentum created by the feedback on quality gaps to create the commitment and identify the individuals who would take on responsibility for it.
While we underline the uniqueness of our findings, in terms of both comprehensiveness of coverage along the continuum of care and variety of settings, we must also recognize their limitations. These derive on one side by the fact that in most countries we were not able to document the details of the process between the two assessments and the extent to which the leading agencies were able to ensure follow-up and support, on the other by the difficulty in disentangling the effects of the inputs provided by the baseline

VIEWPOINTS PAPERS
quality assessment from the changes produced by independent factors acting in the same period, such as new laws, regulations and development assistance. However, the very specific features of many of the observed improvements, particularly regarding medical and nursing procedures are hardly explained by inputs other than internal leadership and commitment by involved professionals, and the merit can be attributed with reasonable certainty to the process started through the WHO QA/QI MN tool. In particular, the tool adopts a participatory approach that builds awareness among both managers and staff about the quality gaps and their causes, provides professional support in identifying appropriate practices and providing evidence, and introduces guiding concepts and tools such as the WHO quality of care standards. Indeed, standards cannot be implemented without a learning process that needs professional and contingent inputs along the continuum of care [17], not just bureaucratic recording of what is missing and written protocols of what should be done. The participatory and professional feature of the assessment required by the WHO tool is key to prompt this learning process, as indicated also by the review of the studies on barriers and facilitators. This analysis identified some commonalities across the variety of health system contexts. Among these, the human resource component emerged as crucial, in all its dimensions of manpower availability and deployment, training, continuous professional development and motivation. The capacity of hospital managers and of key professionals, such as unit or department heads, to motivate their staff, and to provide opportunities for professional development emerged as a factor that ultimately made most of the difference at facility level. Although staff numbers, qualification and salaries are obviously important, the issue of human resource management by hospital leading staff is still greatly undervalued, when not neglected. In fact, the differences in the quality of care provided among hospitals belonging to the same system and struggling with the same structural and budgetary difficulties point to the human factor dimension as something which should have a higher place in the health system agendas, and to the need of integrating preservice curricula of all health professionals, from nursing and midwifery staff to managers [18][19][20][21]. Based on this evidence, it appears crucial that quality assessments not only provide a snapshot of the existent, but include a dynamic component, namely by initiating the development of a plan of action and defining a time frame for its implementation, as envisaged in the process promoted by the WHO MN tool. Ideally, in addition, prospective identification of barriers and facilitators may help to avoid potential frustration of good will by setting realistic targets for the action plan thus increasing the likelihood of success. A recent review of MN quality assessment tools indeed identified the lack of a clear path leading to action as one of the main pitfalls of the existing tools [9].
In spite of recent progress, the attention paid to quality of care, the capacity to implement quality improvement cycles and take into account patients' rights and perspectives is still insufficient in all health systems and at all levels [4,9,10]. At facility level, quality assessment tools should promote a constructive dialogue with hospital managers and key staff and build their awareness of quality issues and their capacity of implementing quality cycles also including patients' views. The review of the experience made with the WHO QA/QI MN tool is encouraging as it shows that this is possible. This refocuses the need of policy changes at national level in support of QI at local level, such as efforts to build capacity for supportive supervision and coaching of auditing processes, including patients' views: professional staff, managers and patients are all key players of a quality movement [10,22,23].