Training, supervision and quality of care in selected integrated community case management (iCCM) programmes: A scoping review of programmatic evidence

Aim To describe the training, supervision and quality of care components of integrated Community Case Management (iCCM) programmes and to draw lessons learned from existing evaluations of those programmes. Methods Scoping review of reports from 29 selected iCCM programmes purposively provided by stakeholders containing any information relevant to understand quality of care issues. Results The number of people reached by iCCM programmes varied from the tens of thousands to more than a million. All programmes aimed at improving access of vulnerable populations to health care, focusing on the main childhood illnesses, managed by Community Health Workers (CHW), often selected bycommunities. Training and supervision were widely implemented, in different ways and intensities, and often complemented with tools (eg, guides, job aids), supplies, equipment and incentives. Quality of care was measured using many outcomes (eg, access or appropriate treatment). Overall, there seemed to be positive effects for those strategies that involved policy change, organisational change, standardisation of clinical practices and alignment with other programmes. Positive effects were mostly achieved in large multi–component programmes. Mild or no effects have been described on mortality reduction amongst the few programmes for which data on this outcome was available to us. Promising strategies included teaming–up of CHW, micro–franchising or social franchising. On–site training and supervision of CHW have been shown to improve clinical practices. Effects on caregivers seemed positive, with increases in knowledge, care seeking behaviour, or caregivers’ basic disease management. Evidence on iCCM is often of low quality, cannot relate specific interventions or the ways they are implemented with outcomes and lacks standardisation; this limits the capacity to identify promising strategies to improve quality of care. Conclusion Large, multi–faceted, iCCM programmes, with strong components of training, supervision, which included additional support of equipment and supplies, seemed to improve selected quality of care outcomes. However, current evaluation and reporting practices need to be revised in a new research agenda to address the methodological challenges of iCCM evaluations.

It is widely recognized that there are effective interventions to prevent, detect, control and manage the most common diseases in poorly developed contexts, such as those affecting children in low-and middle-income countries [1]. However, it is equally acknowledged that the delivery of these interventions is severely hampered by rudimentary or decayed health systems, where essential dimensions of quality of care, such as availability, access and utilisation of services [2], are hardly fulfilled [3].
Innovative approaches do exist to address health care delivery faults, ultimately aiming at addressing quality of care shortcomings. The Integrated Community Case Management (iCCM) promoted by the World Health Organisation (WHO) / United Nations Children Fund (UNICEF) [4], encompasses a series of strategies and activities taking health care closer to communities. In this approach, Community Health Workers (CHW) typically serve as the first point of contact between communities and services.
As any other intervention or strategy, iCCM programmes have to be tested or evaluated in order to describe successes, failures and factors related to them. Rigorous research evidence on the effects of iCCM is scanty [5,6]. Furthermore, iCCM programmes often encompass multiple components which complicates their evaluations. Often, evidence on iCCM programmes has to come from programmatic documents supported by operational research of varying quality.
The aims of this article are to report on the components of selected iCCM programmes and to draw lessons learned from existing evaluations of those programmes, through a scoping literature review of programmatic documentation. We will not attempt to estimate or synthesise the effects of iCCM interventions in primary or secondary programmatic or health-related outcomes, but will provide illustrative examples.

METHODS
A scoping, structured literature review of programmatic information evidence was carried out. 'Structured review' refers to a review of the literature which pragmatically adapts standard systematic reviews' methodology, such as the one used for Cochrane Review [7], yet remains transparent in relation to its methods and rationale for adaptations. This review was based on documents provided by stakeholders since UNICEF defined the focus and scope of the review. Outcomes were only generically predefined as human resources and quality of care related outcomes.
Included documents referred to programmes reported by selected partners proposed by UNICEF. There were no restrictions based on the types of documents, types of stud-ies or types of evidence within them. Quality of evidence was no formally assessed and therefore there were no exclusions based on this criterion. However, three levels of quality of evidence were defined to support the interpretation of findings on the effects of the programmes: low quality when the source of evidence was based on qualitative data or opinions; moderate quality when quantitative methods were used and described in the source documents; and high quality when findings were presented with some measure of statistical significance ('+'. '++' and '+++', respectively).
Twenty nine programmes were proposed and provided by UNICEF and partners. Three types of data were extracted: (a) features describing the programmes (eg, name, funding, objectives, time frame); (b) programme tools highlighted as promising approaches to improve health care; (c) evidence on the effects of tools and approaches. This information was synthesised across programmes into two thematic areas: human resources and quality of care (only the latter is reported in this article).
Descriptions of iCCM programmes and their features are presented narratively and, where data are available, quantitative information is also included in the text or as tables. Due to the large variability in the amount and in the types and quality of evidence across iCCM programmes, no attempt has been made to carry out meta-analyses of quantitative estimates across iCCM programmes. References to particular iCCM programmes are made within brackets with the terms used in the documents and the country names as appropriate.
There was no overreaching quality of care framework across all programmes and authors accepted an estimate or an indicator to be related to quality of care if it referred to the events in the delivery of care (from availability of care to effective coverage) and health related outcomes. Since this is not a review on the effects of iCCM interventions, we have selected only some indicators best related to quality of care or serving as illustrative examples, from the very large amount of indicators reported in some programmes.

Overview of iCCM programmes' objectives and strategies
A total of 29 iCCM programmes were included in this review. All programmes were implemented in African countries, but one, in Myanmar. Table 1 lists the included programmes alongside the main implementing organisation, partners and programmes' start and end years. The documentation scrutinised referred to programmes or phases plementing Integrated Management of Childhood Illnesses (IMCI). 11 (40% of the 28 programmes with this information) included in their objectives morbidity and mortality targets (eg, 'Backpack plus', 'Concern Niger').
Basic clinical care was often complemented with other strategies (eg, 'Backpack plus', 'MC Sudan South'), such as policy influence or advocacy (eg, 'CORE-group'), health systems strengthening (eg, 'Concern Niger'), provision of ending between 2010 and 2013 (66% of programmes), two others were older (2005 and 2008), another one was ongoing (ending in 2015) and for the remaining seven dates were missing All programmes shared a common objective, which had to do with increasing access to good quality health services by poor populations, with a special focus on infants' and children' s diseases, through the deployment of CHW, im-

Community health workers
CHW are at the core of iCCM. They are designated in different ways depending on the iCCM programme ( Table 2). Names are in part descriptive of the functions CHW carry out but also respond to the names that may have been used in the past in certain countries (eg, Health Extension Workers in Ethiopian programmes). For the sake of clarity and simplification, we use the generic term CHW in this article.
Activities carried out in the programmes (mainly by CHW) could fall into two main groups as identified in the programmes documents: 1) provision of clinical care (with or without other components, such as health promotion and prevention); 2) provision of supplies, mainly medical supplies (eg, drugs), through social franchising schemes (eg, 'PSI Myanmar', 'PSI Uganda -Five & Alive', 'Living goods Uganda') or using regular procurement schemes (Social franchising is the provision of affordable services by the non-profit health sector, complying with franchise stan-dards targeting underserved communities [8]; micro-franchising refers to small scale entrepreneurship by CHW [9]).
Treatment conditions included: malaria, diarrhoea and respiratory diseases assessment and early treatment, conjunctivitis, malnutrition, new-born at risk, ear infections, sexually transmitted infections and HIV testing.
Health promotion and disease prevention focused on malaria, diarrhoea and respiratory diseases recognition and health seeking behaviours, immunisation, nutrition, water and sanitation, maternal and new-born care, reproductive health and family planning, breastfeeding, complementary feeding, insecticide treated nets, malaria preventive treatment, TB prevention and treatment.
CHW were selected using a wide range of different criteria. Interestingly, there was some information on exclusion criteria (ie, candidates who were NOT eligible); eg, political leaders or those imposed by political leadership ('MOH Uganda').
The number of CHW involved was difficult to assess because depended on the time-span of the programme, the degree of scaling up and the different types of health care workers reached. Table 3 shows the approximate number of CHW involved in the programmes, when available in the source documents (median 1441, interquartile range 732 to 2582).
Programme documents described several types of incentives. The majority of incentives were goods and even work equipment and tools (9 of the 17 programmes with data, 53%). Only in Malawi did CHW receive proper salaries.
Incentives also included intangibles such as recognition and reputation. In detail:

Training and supervision
Training schedules, length and approaches varied greatly across programmes. Table 4 details the length of training for those programmes which had this information avail-A scoping review of raining, supervision and quality of care in selected (iCCM) programmes able. Median training length was 2 weeks (interquartile range from 6 to 43 days), depending on the contents and competences to be achieved.   Supervisors undertook a specific training, which ranged from two to nine days, in the six programmes where this information was available. Tools used included guidelines, checklists and training manuals. The frequency of supervisory visits ranged from once a year to three times a month ('PSI South Sudan'); although in some cases there are reports of CHW not having received a single supervisory visit (eg, 'MOH Madagascar'). Meetings were also mentioned as supervision-like strategies in seven programmes (37% of the 19 programmes with this information available) (eg, 'MOH Ethiopia' with biannual meetings; 'PSI DRC' monthly monitoring meetings).
Supervision activities could include any mix of the following areas of work: clinical skills, submission of reports, analysis of reports and feedback, medical supplies, logistics, site management, relations with the community, recommendations or corrective actions.
Interestingly, there were programmes where CHW were working within a more or less formal network of CHW and other providers. For example, teams and team-work was heavily emphasised in 'Save Zambia'; 'Care Group Volunteers' were reported by 'Concern Burundi'; and peer support groups based on the Care Group model were implemented by 'Concern Rwanda KabehoMwana'.
Several tools were identified across the documents, sometimes clearly highlighted in programmes reports and some other times identified by the reviewers as potentially innovative or particularly important programme components. A total of 114 tools have been identified across the whole set of programmes. In summary, they included equipment (eg, a backpack and storage box, a drug calculator, supplies, as complements to CHW activities and to support motivation as well); guides (describing procedures or tasks, such as clinical tasks, assessments or supervision); job aids (eg, home-based management; peer-support groups; case management; counselling cards, mother reminder cards); templates for reporting (eg, register and referral forms; CCM register; CCM supervision form; follow-up visit form; medication stock management form); communication tools (eg, home and community boards; flip charts).
Other tools included an integrated analogue and digital mobile phone application for real-time stock tracking and reporting or an integrated toolkit map to facilitate the planning of activities within the catchment area of CHW.

Quality of care
Eventually all programmes implemented IMCI care protocols in one way or another, which served as an overreaching framework for a number of activities with the main components being guidelines, expansion and training of CHW, supervision and often supplies.
Programmes were not uniform in their underlying 'quality of care' concept or framework, which was in most cases implicit. Therefore, 'quality of care' was approached under different perspectives and dimensions of care across programmes. We extracted information on a limited number of outcomes related to access (ie, utilisation, coverage), appropriateness of care (eg, adherence to guidelines) and health outcomes. Programmes reported very different outcomes and there was no full consistency in measurements and reporting approaches.
As shown in Table 5, we extracted and grouped reports on outcomes, selecting those that seemed to be better related to quality of care indicators and better reported. 43% were categorised as qualitative ('+'), 30% as quantitative ('++') and 26% as quantitative with some estimation of statistical significance.
The synthesis of effects on quality of care suggested that there were positive effects for those strategies that involved policy change ('CORE Cameroon'), organisational change (eg, C-IMCI framework 'CORE Cameroon'), standardisation ('Concern Rwanda'), integration with existing health care services and alignment with other programmes which may ease implementation and scaling up ('CORE Malawi', 'PSI Mali').
Quality changes seemed more remarkable in large multicomponent programmes which included training of CHW, strengthened supervision and improvement of supply change management. Improvements in monitoring and evaluation procedures seemed to have had positive effects on utilisation rates ('MOH Ethiopia'). Interestingly, access improved in most programmes, yet achievements were moderate in absolute terms or compared with formal health care ('Save Malawi'). Geographic and effective access to care increased ('Save Malawi'). A programme with a component of improvements of information transmission through mobile telephones seemed to have increased utilisation of CHW and more prompt management of illnesses.
Other strategies aimed at reinforcing the relations between CHW either with peers or supervisors. Peer-support groups provided a platform for more effective human resources interventions (eg, supervision, trust, accountability; 'Concern Rwanda'); social franchising ('PSI Myanmar') seemed to strengthen networking of providers, alongside an increase in reputation of CHW. Micro-franchising seemed to achieve affordable improvements in the availability of good quality medical products ('Living Goods Uganda'). Social franchising increased the availability of services at equal or lower costs than regular formal services and supplies with specific data on Oral Rehydration Salts (ORS) distribution ('PSI Myanmar'). Equity (differentials in access from different economic strata) was reported in terms of access to CHW, Artemisinin-based Combination Therapy (ACT) and treatments of diarrhoea ('PSI Cameroon'); social franchising also seemed to improve equity, focusing on the most vulnerable populations ('PSI Myanmar', 'PSI Uganda').Yet, at least one programme ('Save Malawi') could not find differences in accessing CHW according to wealth. In contrast, some of the poor outcomes (eg, clinical management) were related to the shift of care seeking between different types of providers; for example, from formal governmental services to CHW community based care. Training large numbers of CHW led to the reduction in the use of traditional healers, although this was seen as a positive effect of the programme ('CORE Malawi'). Introduction of an iCCM programme in an area where care seeking appeared generally high resulted in shifting of care from government health centres, private health facilities and shops to village health clinics.
Although the aims of this review did not include reporting on mortality, it is worth noting that reductions in mortality were occasionally reported with findings suggesting reductions in some geographical areas but not in others, within the same programme ('PSI Cameroon'), or not statistically significant reductions ('IRC Sierra Leone').

DISCUSSION
We have reviewed 29 iCCM programmes in Sub-Saharan Africa and Myanmar. All programmes were based on iCCM guidelines and principles implemented by CHW, although the way programmes were implemented varied greatly. This review had some limitations: it is likely that more programmatic or research information could have been found with Estimated cost per life saved US$ 1200 (based on the project' s total budget).

++
Mothers continued breastfeeding children even when pregnant; children and pregnant women were more likely to eat eggs, food high in protein and essential micronutrients.

++
Residents far less likely to use traditional healers; people stopped using bed nets for fishing; a significant number of traditional healers abandoned their practice and joined the program as volunteers, isolating and undermining the credibility of those who remained working as traditional healers.

+
Save Malawi CHW were the main source of care in intervention areas (at baseline the source was the public sector); shifting care from public to CHW care; checking breathing with timer not systematic; non-statistical significant increase of appropriate treatments. Improved on equity in access. +++ more time and resources; information was typically retrieved from evaluation studies or programmatic documents rather than experimental, controlled research studies providing moderate to low quality evidence; finally, reporting bias could not be excluded since only reports provided by a selection of stakeholders were included in this review. While programmatic information provides invaluable evidence on processes, the lack of more robust evidence on the effects of the programmes precludes any attempt to relate processes with outcomes. Findings from this review are not and cannot be representative of iCCM programmes and cannot be extrapolated to any particular setting. However, they are meant to help to draw lessons from those programmes proposed included in this review.
iCCM was defined by its objectives by WHO/UNICEF in 2012: "to train, supply and supervise front-line workers to treat children for both diarrhoea and pneumonia, as well as for malaria in malaria-affected countries, suing ORS and zinc, oral antibiotics, and artemisinin-based combination therapy (ACT) ... iCCM also enables community health workers to identify children with severe acute malnutrition through the assessment of mid-upper-arm circumference (MUAC)" [4]. This definition shares the objectives and resembles the old definition of selective primary health care (PHC) (1979): "a circumscribed number of diseases are se-lected for prevention in a clearly defined population ... The principal recipients of care would be children up to three years old and women in the childbearing years. The care provided would be measles and diphtheria-pertussis-tetanus (DPT) vaccination for children over six months old, tetanus toxoid to all women of childbearing age, encouragement of long-term breast feeding, provision of chloroquine for episodes of fever in children under three years old in areas where malaria is prevalent and, finally, oral rehydration packets and instruction" [10].Similarities between both approaches suggest that iCCM is not an entirely new strategy, but rather it shares and it may be inspired by key features of selective PHC.
In this article we focused on the description of programmes and quality of care issues. iCCM programmes are composed by a mix of multiple interventions or strategies; namely, disease portfolios, CHW arrangements, clinical skills, supplies, referral systems, training, supervision, community support and policy changes. These components are implemented in different combinations and intensities depending on the country or setting where programmes operate, donors' preferences and country health related policies, among other factors. It was appealing to us that, in fact, the term 'iCCM' embraced a large plethora of very different programmes which may have limited com- monalities between them in some cases. Most studies, trials or evaluations clearly deal with key factors affecting the performance of CHW, the quality of care they provide and, eventually, clinical outcomes. These factors included supplies, CHW supervision, training, quality of care and retention of CHW [11], among others.
A contribution of this review of iCCM programmes has been to systematically identify and present innovative or promising approaches; such us: integration with other programmes to boost effectiveness; integration with the private, public and traditional sectors, coordination with stakeholders to align resources and expertise in different areas of work. Stock management and availability of medicines seemed to be key in several programmes in effectively supporting CHW activities and ensuring credibility.
Measuring quality of care in the context of CCM is challenging and different approaches and measurement methods may lead to different descriptions of the same situations [12]. The number of quality of care indicators is very large and their types extremely varied, for example from the availability of inputs to the achievement of outputs and outcomes, from knowledge to health status outcomes, and the perspectives of supply-and demand-side. Therefore, one could argue that it is easy to find examples of positive effects when a large number of indicators are measured, as was the case in the programmes we have reviewed. As in other reviews, care seeking behaviour and utilisation of treatments tended to show positive, albeit variable, effects across the different programmes [5]. Not so often, examples of no effect were found; even less frequently, negative effects were identified (reporting bias of positive outcomes could not be investigated nor ruled out). Interestingly, the shift of utilisation from traditional healers to CHW was reported as a positive outcome, when actually this might not necessarily be seen as a desirable outcome ('Core Malawi').
We acknowledge, that quality of care is a means to achieve better health related outcomes, such as morbidity and mortality [13]. Only limited evidence on morbidity and mortality has been included in the documents of the programmes we have scrutinised. Interestingly, there are variations in the effects of community based management across conditions. For example, a review of the evidence on the effects of community based management of pneumonia in Africa, which included published studies in English or French (excluding non-published reports), using any primary study design, could not find evidence of impact on morbidity and mortality and raised several implementation concerns related to CHW capacity to manage pneumonia [6]; although more promising findings were reported in another review which included studies from Asia [5]. More positive findings were reported in a systematic review of experimental or quasi-experimental studies in Sub-Saharan Africa on the effects of CCM on malaria outcomes [14]; for example, significant reduction of malaria deaths in Ethiopia and Uganda, although no effects were observed on other clinical outcomes (eg, hospitalisations, anaemia). Often, though, the quality of evidence is not optimal [15].An additional issue to consider is the role of programme characteristics (ie, the specific implementation approaches) and the context. The scope of our review does not allow to drawing conclusions about differences on iCCM performance in different geographical areas (eg, Africa or Asia); however, where evidence exists, these differences have not stood out [5].

CONCLUSIONS
We attempted to provide some insights on the effects of iCCM intervention in quality of care indicators, despite the fact that scoping reviews do not aim at establishing effects of interventions. Large, multi-faceted, integrated iCCM programmes, with strong components of training, supervision, which included additional support of equipment and supplies, provided examples of improvements in selected quality of care outcomes. However, examples of modest, null and somehow adverse effects were also shown.
We could not establish which mix of interventions or strategies (eg, supervision, training and incentives) produced which effects on quality of care. Evidence on the main components exists; for example, on lay health workers [16], supervision [17], training or job aids [18]; but not on the mix of those interventions which lead to better outcomes and under which conditions. We are afraid that this is also the case for the reviews and studies we have recently accessed. Inevitably, the effects of innovative approaches (eg, networking between CHW peers, mentorship), which were used and seemed promising in some cases, remained diluted in the body of low quality evidence that could be extracted from programmatic documents. The lack of good quality evidence is not only a concern for the international health community, but also for policy makers [19] who may not recognise the value of an approach which may not have been robustly evaluated and reported. May be evidence on the strengths and limitations of selective PHC could have also informed more recent initiatives to implement iCCM.
In the absence of good quality evidence, research evidence has to be produced [20] and, in the meantime, good quality global guidance on what iCMM 'formula(s)' are more promising under different circumstances, needs to be elaborated with tools to adapt it to local settings [21]. A research agenda, and eventually guidance developers, would benefit from a series of actions spearheaded by the iCCM task force; namely: (i) standardisation of iCCM concepts, strategies and tools; (ii) establishment and reinforcement of evaluation methodological standards, including protocols for selecting and reporting on primary outcomes, harms and costs; (iii) mapping existing research to avoid duplications and search synergies, emphasising pragmatic research of integral approaches rather than for individual diseases; (iv) establishing a few manageable priority research areas; (v) creating an open, structured, transparent and comprehensive online platform to share evidence on iCCM with quality assessments of the evidence presented.