Integration of parenting and nutrition interventions in a community health program in Pakistan: an implementation evaluation

Policy and program implementers require evidence on whether integrated psychosocial stimulation and nutrition interventions can be effectively delivered at‐scale, how, and at what cost? To address some of these issues, a comprehensive evaluation of implementation was designed for a trial in Pakistan that integrated psychosocial stimulation and nutrition interventions in a community health service. The first objective was to describe, analyze, and assess the quality and accuracy of the implementation of the interventions. The second objective was to identify barriers and facilitators for uptake of interventions. A mixed‐methods evaluation of implementation processes was conducted. Interventions were accepted by the community and health providers and there was evidence for behavior change uptake of the care for early childhood care recommendations. The new interventions did not dilute delivery of routine services. However, fidelity and quality required supportive supervision and active use of monitoring data, which would require attention in scale‐up.


Introduction
The integration of parenting and psychosocial stimulation interventions in health and nutrition platforms was recommended in the first series on "Child Development in Developing Countries" published in the Lancet a decade ago. 1 A systematic review of evidence on the effectiveness of psychosocial stimulation interventions and nutrition interventions (sometimes including care for health and hygiene) showed that the former consistently benefitted early child development (ECD) outcomes, the latter consistently benefitted early child nutrition outcomes, and the combination did not result in a significant loss of effect on either outcome. 2 Proponents for implementing integrated psychosocial stimulation and nutrition interventions argue that combined early childhood packages of care have the potential to benefit multiple child outcomes, 3 families demand holistic interventions, and these strategies are potentially a more efficient use of scarce public health resources. 4 However, there are few examples of integrated psychosocial stimulation and nutrition interventions implemented at-scale. 5 Policy and program implementers require evidence on whether integrated psychosocial stimulation and nutrition interventions can be effectively delivered at-scale, how and at what cost? Implementation research can inform evidencebased integration of combined psychosocial stimulation and nutrition interventions in programs. Systematic reviews of implementation of psychosocial stimulation and nutrition interventions identify features associated with successful program outcomes. [6][7][8] Inputs include the use of a structured curriculum and adequate attention to training and supervision. Behavior change techniques that support delivery include the provision of information, use of small media (e.g., pamphlets and pictorial doi: 10.1111/nyas.13649 aids), performance techniques (e.g., providing caregivers with the opportunity to try out stimulation activities with their young child and receive feedback) and demonstrations (e.g., cooking demonstrations), problem solving, and social support. 6 The greater the number of behavior change techniques applied in a delivery strategy, the more effective the intervention on child outcomes. 9 However, authors acknowledge that few intervention studies and program evaluations either report intended implementation strategies or assess fidelity to implementation protocols.
Moreover, beyond the assessment of fidelity, the evaluation of implementation can address critical questions on integration of early childhood interventions in existing platforms. For example, what impact do new interventions have on existing services and workforce with respect to time, quality, demand, uptake, and cost? Implementation evaluations should also consider a broad range of contextual factors that influence program success: policy and program environment, donor commitment, availability of technical experts, understanding of community dynamics and traditions, capacity development, robust monitoring and evaluation, community mobilization, and effective communication among stakeholders. 10 The importance of sharing both success stories and failures in implementation is a critical component of building the evidence to integrate early childhood interventions at-scale.
The child health context in Pakistan during the implementation of the PEDS trial Pakistan's progress in improving newborn survival rates has been slow with the third highest rate of newborn mortality in the world. 11 For infants and young children who survive, thriving is challenged by high rates of morbidity and an estimated 44% of children are stunted. 12 During the planning and implementation period of the Pakistan Early Child Development Scale-Up (PEDS) trial, a number of political and environmental challenges further hindered progress in addressing child survival and health. In 2007, following significant changes in the federal structure, responsibilities for health were devolved from the federal to provincial governments creating a need for technical facilitation in provincial governments to strengthen public health policies, financing, and systems. 13 In 2010, in addi-tion to the changing political environment governing health, Pakistan experienced severe floods that affected 20 million people destroying homes and health facilities. 14 Despite these challenges, Pakistan's child health landscape also offered opportunities; for example, the lady health worker (LHW) program employs 100,000 LHWs committed to supporting maternal, newborn, and child health in rural and disadvantaged communities. In 2008, the technical steering committee of the LHW program at the federal level granted permission for the Aga Khan University research team to evaluate the integration of early childhood interventions with the potential of strengthening ECD and growth outcomes of young children.

Pakistan Early Child Development Scale-Up trial
In Pakistan, using the infrastructure of the "National Programme for Family Planning and Primary Healthcare" (commonly referred to as the LHW program), the effectiveness of integrating a psychosocial stimulation intervention with or without nutrition interventions was evaluated in a community-based pragmatic cluster-randomized factorial effectiveness trial. 15 The LHWs delivered the enriched interventions integrated with routine health services to caregivers and their young children in the first 2 years of life in monthly home visits and parenting groups.
At 2 years of age, children who were exposed to the psychosocial stimulation intervention (with or without enriched nutrition intervention) had significantly higher cognitive, language, and motor development scores compared with the children who were exposed to either the enriched nutrition intervention or only routine health services. No benefits were observed on child growth outcomes as a result of exposure to the enriched interventions. Mothers of young children who were also exposed to the psychosocial stimulation intervention (with or without enriched nutrition intervention) had significantly benefited in care-related outcomes (e.g., parenting knowledge and practices, motherchild interactions) compared with the children who were exposed to either the enriched nutrition intervention or only routine health services. 16 Benefits to children's development and learning outcomes, and mothers' caregiving skills were sustained 2 years after the end of exposure to the enriched interventions. 17 A cost-effectiveness analysis found that integrated packages of care that included psychosocial stimulation were more cost-effective than an enriched nutrition intervention or current routine health services on promoting ECD. 18 The outcome evaluation of the PEDS trial showed that community health workers in a government program could effectively promote ECD in a strategy that integrated a psychosocial stimulation intervention in the routine community health service. However, these findings do not shed light on questions frequently asked by policy makers and program implementers on how effective implementation was achieved, what were the barriers and enablers to integration of new interventions in an existing program infrastructure, and what key lessons can be learned for uptake, impact, and sustainability of these interventions in the health system? Therefore, a comprehensive evaluation of implementation was designed from the outset of the PEDS trial.
Our paper addresses two objectives. The first was to describe, analyze, and assess the quality and accuracy of the implementation of the psychosocial stimulation intervention, alone or in combination with the enriched nutrition intervention, in the LHW program. The second objective was to identify barriers and facilitators for uptake of the psychosocial stimulation intervention, alone or in combination with the enriched nutrition intervention by the community and LHWs.

Methods
A mixed-methods implementation evaluation of the PEDS trial was conducted between June, 2009 and March, 2012. The trial was implemented in Naushero Feroze district in Sindh province. Naushero Feroze is a predominantly rural and impoverished district with a population of approximately one million. The PEDS trial was implemented across eight Union Councils (subunits of a district) which comprises eight government health facilities including a district head quarter hospital, three rural health centers, three basic health units, and one dispensary. Approval for data collection was granted from the ethical research committee of the Aga Khan University, Karachi, Pakistan.
The integration platform: the lady health worker program The LHW program was established in 1994 in order to increase the utilization of promotive, preventive, and curative services in rural and disadvantaged communities, especially for women and children. 19 The program covers 60% of the population in mainly rural areas. Health indicators in areas with program coverage are generally better compared with uncovered areas.
LHWs are selected by local committees and are typically married women with at least 8 years of education. Recruited LHWs receive 3 months of basic classroom training followed by 12 months of supervised field-based training. They are allocated 15 refresher training days per year. Supportive supervision is promoted and a lady health supervisor is assigned to support around 25 LHWs. There are 22 core tasks in the LHW program including family planning counseling, care for pregnant women, health, hygiene, and nutrition education, child growth monitoring, provision of basic medicines for common ailments, immunization campaigns and polio drives, and referrals to primary healthcare facilities. Each LHW delivers these services to a catchment serving around a 1000 population (100-120 households) through monthly home visits and community group meetings. Financing one LHW costs the Government of Pakistan (GOP) US$570 per year of which more than half is allocated to her stipend. The monthly stipend was increased by the GOP to more than 50% in September 2010 (during the course of the PEDS trial) to US$85 per month.
While there are recognized bottlenecks that affect the efficient implementation of the LHW program, the infrastructure offered an opportunity to promote ECD with caregivers of young children: the LHWs were trusted health providers in local communities, the program covered rural and disadvantaged populations likely to benefit from interventions designed to promote ECD, and the proposed parenting and psychosocial stimulation intervention complemented existing child health and nutrition services.

The interventions: content and delivery strategy
The content and the delivery strategy were designed following formative research, which included an  analysis of the LHW program. The analysis of the LHW program identified gaps in interventions which could strengthen ECD outcomes (Fig. 1). The LHW program had no specific focus on care for early childhood development including guiding caregivers in responsive care and providing early stimulation opportunities to enable a child to explore his/her environment, manipulate objects, problem solve, and build social and communication skills. While nutrition education was provided to caregivers, the nutrition inputs in the LHW program were weak and required strengthening. Therefore, the new interventions to be integrated in the LHW program were responsive stimulation and enhanced nutrition.
In the trial, three intervention combinations integrated with routine services were tested compared to delivering routine services alone: (1) responsive stimulation, (2) enhanced nutrition, and (3) responsive stimulation combined with enhanced nutrition. The responsive stimulation intervention was an adaptation of the UNICEF and WHO Care for Child Development recommendations that use the context of developmentally appropriate play and communication activities to coach parents on responsive caregiving skills. 20 The intervention content also addressed problem solving on care for children's development. The enhanced nutrition intervention enriched the basic nutrition education with a multiple micronutrient supplement (MMS), added responsive feeding recommendations, and included problem solving on care for young child nutrition (Table 1).
Twenty LHWs were trained to deliver each of the enriched intervention packages. The LHWs received 3-day training for delivering the responsive stimulation package, a 2-day training for delivering the enhanced nutrition package, and a 5-day training for delivering the combined responsive stimulation and enhanced nutrition package. The number of training days utilized up to one-third of the allo-  a Each group meeting session focused on a key message from the care for child development module. b The generic training manual, facilitator notes, and video were used to provide the core trainings to the ECD facilitators supplemented with modules on child development. of the trial who continued to deliver routine health services.
It was recognized the basic training would not be adequate to support the development of competencies to deliver the enriched intervention packages; therefore, a 1-day refresher training day was allocated every 6 months designed to be responsive to the skill gaps of LHWs observed during supervision. In addition, a supportive supervision strategy was implemented. The integration of new interventions into existing health platforms requires additional support as noted by Powell and colleagues. 21 To aid the integration of the new and enriched interventions, a supervisory team of ECD facilitators was recruited and salaries were paid for by the research team. This was a team of six women who lived locally with at least a Bachelor's degree. The selection criteria for the ECD facilitators were similar to the selection criteria for lady health supervisors, the routine government supervisors in the LHW program. The role of the ECD facilitators was to facilitate the introduction of the enriched intervention packages in the LHW program infrastructure. The ECD facilitators were expected to provide twice monthly supervision, including an on-the-job coaching to the LHWs during home visits and during a parenting group. In addition, the ECD facilitators maintained a liaison with the government lady health supervisors. The number of expected contacts was the same as the number of expected contacts between LHWs and their supervisors in the program; however, the ratio between ECD facilitator to LHWs was 1:10 which is lower than the program 1:25 ratio. The ECD facilitators received up to 3 months of training prior to the roll-out of the PEDS trial led by the research team (A.K.Y., M.R., and S.S.) covering knowledge about ECD and young child nutrition, experience of delivering the interventions in the local community, and supportive supervisory skills. The intervention delivery model aligned with the LHW program. Enriched interventions were integrated in routine monthly home visits. The LHWs are expected to deliver a monthly community group meeting on health issues; however, this was rarely delivered due to a lack of objectives. The technical steering committee recommended employing the community group meeting to deliver the responsive stimulation intervention (with or without enhanced nutrition packages) because this was a new component to the program rather than an enhancement of routine services. Mothers and children attended the parenting groups together given an important goal of the session was to try stimulation activities with the young child. Table 2 shows the expected knowledge and competencies for each level of the delivery strategy from the research team to the mother-child dyad. A summary of the implementation strategy is shown in the logic model in Table 3.
The implementation evaluation framework, sampling, and sources of data Sources of data, quantitative and qualitative, were selected to enable analysis of the following dimensions of implementation identified from the literature:   r Equality: Procedures used to approach participants.
Data presented below are from three sources: program monitoring data, training and supervision data, and community and LHW feedback. The data collection tools for implementation monitoring, training, and supervision are shown in Table 4 and indicate the construct of implementation informed (e.g., dose delivered). These data generally inform fidelity or accuracy of implementation as intended, while quality, barriers, and facilitators are captured through the qualitative study and supervision. The data comprise independent monitoring of the intervention delivery collected in the three enriched intervention arms and the comparison arm of the trial, as well as data collected by the intervention delivery team (e.g., ECD facilitators) that were collected only in the three enriched intervention arms of the trial.
The perspectives of both the users (families) and the service providers (LHWs) were sought in order to ensure the program was responding to their needs, and to identify how the content, delivery, training, and supervisory strategies could be strengthened. A phenomenological qualitative approach was selected for this component of the implementation evaluation because it facilitated rich discussions on the reasons underpinning the engagement of local families and providers with the responsive stimulation and/or the enhanced nutrition interventions. 22 These data were collected in two rounds; round one data was collected in the summer of 2010 (1 year after intervention roll-out) and round two data was collected in the summer of 2011 (2 years after intervention roll-out).   Information was collected through in-depth interviews and focus group discussions (FGDs). The topics discussed during the interviews included awareness of new strategies being delivered by the LHWs, degree of relevance and acceptance of interventions, opinions on the mode of delivery, perceptions on change in LHWs, caregivers and children, extent of inclusion experienced, change in relationships between different stakeholders, training, supervision, level of successful integration in LHW's duties, future sustainability, and recommendations. Respondents included mothers (including mothers of children with mild-to-moderate disabilities, and mothers who were identified as psychologically distressed), fathers, secondary caregivers, LHWs, lady health supervisors, ECD facilitators, doctors, and teachers. Purposive sampling was used to identify the participants from the community who had exposure to the responsive stimulation and/or enhanced nutrition interventions. A total of 18 FGDs and 12 in-depth interviews were conducted before reaching a point of saturation. Participants for FGDs were homogenous groups comprising either mothers, fathers, secondary caregivers, and ECD facilitators or LHWs. Individual in-depth interviews were conducted with primary school teachers, doctors, lady health supervisors, intervention team leader, mothers identified with psychosocial distress, and mothers caring for children with disabilities.
A moderator and notetaker were involved in the FGD and the in-depth interview data collection.  audio-recorded. Verbatim transcriptions were prepared in Sindhi (local language) by the moderator and notetakers and then transliterated into Urdu (the common language across the research team) for analysis.

The implementation evaluation data management and analysis
Quantitative data collection was monitored by routine and unannounced checks by research supervisors. Prior to data entry, all forms were checked for completeness and consistency. Data entry screens were developed using Microsoft FoxPro (Version 7.0) and employed a range of consistency checks to minimize entry of erroneous data. All data were double-entered and checked for consistency. Analysis was completed using the Statistical Package for Social Scientists (SPSS, version 15.0). Standard statistical techniques were employed to describe trends and compare groups. Qualitative data audio-recordings were rechecked against transcripts and a subsample of translated transcripts were back translated by an independent translator in order to check the reliability of the transcriptions. An emergent thematic analysis of qualitative data was completed and included pilot interview data because no substantial modifications were made to the topic guides following piloting. Narratives of themes were compared and contrasted to understand linkages between themes. Data were triangulated from multiple sources and respondents, verifying the repeatability of an observation or interpretation as well as highlighting distinct differences.
In order to minimize bias, the position of the researchers in proximity to the phenomena of issues collected using the qualitative approach is reported. The moderator, fluent in Sindhi and Urdu, and familiar with the social and cultural context of rural Sindh was not involved with the intervention development or implementation; therefore, she had no preconceived ideas about the outcomes and was able to provide different insights from the other two members of the analysis team (A.K.Y. was the principal investigator of the PEDS Trial and M.R. led the training of the ECD facilitators). Familiarity with the context and the development of the intervention provided contrasting perspectives from the moderator's perspectives. Feedback was also sought from the respondents in community meetings to check whether the preliminary conclusions and interpre-tation were an accurate account of their voices, which increased the validity of the final analysis.

Results
The enriched interventions were delivered for 33 months (June 2009-March 2012) and reached 3500 children less than 2 years old.

Dose delivered and reach
LHWs were expected to deliver a monthly home visit to all households in their catchment area with a child less than 2 years old, which ranged from 120 to 150 households per LHW (or 5-7 home visits per day). In the responsive stimulation and in the responsive stimulation combined with enhanced nutrition, the LHWs were expected to deliver a monthly parenting group meeting.
Independent monitoring of LHW activities showed that 89-99% of households in each intervention group reported receiving a home visit by their LHW (Table 5). On average, a home visit in the enriched intervention arms lasted 10-30 min and in the comparison arm lasted 5-7 min as reported by households, which was close to the one-time independent observations of home visits (10-16 min in the enriched intervention arm and 3 min in the comparison arm). In the groups exposed to the responsive stimulation intervention, more than 70% of caregivers reported discussing care for early childhood development with their LHW. Basic nutrition education is a core component of the LHWs' routine service, yet fewer nutritionrelated information was shared in the control group receiving only routine services where the main purpose of visits recalled by caregivers was related to polio drives. Moreover, problem solving around care for nutrition was reported less among caregivers in the routine services only group compared with caregivers receiving enriched interventions. For example, across all groups, close to 50% of caregivers reported receiving information on exclusive breastfeeding; however, problem solving on challenges to breastfeeding was reported in less than 4% of households in the routine services only group. Overall, the integration of enriched early childhood interventions appeared to support routine health services rather than dilute health recommendations.
The LHWs were able to deliver the expected monthly parenting group; however, the number of parenting groups delivered in the first 6 months of  Note: In household monitoring, response options were not provided to household respondents when questions were asked about when the last visit from the lady health worker was received and on the content of the visit. Instead, the responses received were marked on the most appropriate category on the form or if a response did not fit, the details were added in the open response box and subsequently categorized. roll-out was less than the subsequent period. This may have been because in the early implementation phase the LHWs were learning to organize and facilitate these sessions. To overcome this early challenge, the LHWs ran smaller groups twice per month. During the religious period of fasting and during the flood emergency, the number of parenting groups was reduced. Table 6 shows on average a parenting group session lasted 1 hour and 20 minutes. Seventy-two percent of participants were mothers and the remainder of participants were other female caregivers (e.g., grandmothers and aunts). The responsive stimulation and enhanced nutrition activities were delivered as expected during the parenting groups.
The mixed home visiting and parenting group approach was beneficial for families and LHWs. The community expressed that parenting groups  therefore, the combination with home visits enabled information to reach more caregivers. Some caregivers also suggested that coverage of parenting groups was inadequate with a few households reporting that they did not receive an invitation or that the invitation should more explicitly include secondary caregivers (e.g., grandmothers).
I think elderly, married and unmarried women all should attend the meetings, because the unmarried women will get married. Since, she will have knowledge she will be able to nurture her child well. [ Dosage received: exposure, enactment, and satisfaction It was expected that exposure to the enriched interventions (responsive stimulation and enhanced nutrition) would result in improved knowledge and practices on care for child development and nutrition. The receptiveness to the enriched interventions and enactment of practices related to supporting children's development and nutrition changed over time (Table 7). Fundamental concepts (e.g., children learn from birth) were understood early in the implementation and mothers frequently recalled memorable ice-breakers from the parenting groups designed to convey a message about children's development and growth. While more complex skills such as sensitive and responsive caregiving or observation of developmental milestones to guide what children were able to learn were not observed until later in the intervention-roll out. Knowledge and practice of more complex skills may have been re-enforced by mothers' experiences of trying-out the activities with their young children. A picture book was a onetime incentive provided to participants in the groups exposed to the responsive stimulation intervention, which was both well received and supported promoting developmentally appropriate stimulation activities. While other learning materials advised were homemade or everyday objects in the home, the picture book was selected for distribution because: (1) it covered a breadth of stimulation activities (e.g., fine motor skills to explore the book, communication and language development, and fostering maternal responsiveness); and (2) it could be used with children from as young as 6-9 months old.   The uptake of the MMS in the groups receiving the enhanced nutrition intervention was slow. It was reported that mothers and LHWs perceived that the MMS changed the taste of the food it was mixed in. This challenge was addressed in a refresher training with the LHWs with a blind tasting to demonstrate that there was no change to the taste of the food. The LHWs were accepting of the responsive stimulation intervention and reported trying the activities at home with their own children or grandchildren. The beliefs and attitudes of the LHW were considered an important factor influencing the extent to which she may encourage families to try a new practice.

Quality of implementation of enriched interventions integrated in the LHW program
It was expected that LHWs would use active coaching skills (i.e., provide mothers an opportunity to try a play and communication activities and give feedback on how to they can strengthen the interaction) and be able to integrate these messages with their routine work. Table 4 indicates that families received both routine and enriched messages in home visits. The quantitative data are supported by qualitative data. The LHWs understood the linkages between health, nutrition, and development messages and were better able to leverage contact points with families to provide holistic advice. messages, and encouragement and praise of the mother. Individual skills took time to develop; for example, in January 2010, only 69% of LHWs praised the efforts of mothers but this increased to 91% of LHWs by July 2011. More complex skills were also not observed early in implementation (e.g., responsive care). This was also noted in the qualitative observations by the ECD facilitators. It took 1 year for at least 70% of LHWs to demonstrate a satisfactory level of competencies.
Early implementation challenges included the weaker skills of LHWs and community skepticism about whether the program would have worthwhile content. These challenges were managed through supportive supervision, appealing content and a practical approach to learning and experiencing interactions rather than knowledge transfer alone. Ongoing challenges included environmental challenges (e.g., extreme heat and power failures) which discouraged group gatherings; however, a combined delivery approach was helpful in this regard. Weaknesses in the LHW program such as LHW stipends not being paid on time were demotivating for LHWs. These were counteracted by facilitators such as supportive supervision and continual quality building efforts which engaged the stakeholders and supported positive change in the community.

Discussion
The intervention delivery strategy was designed to align with routine services. The LHWs are expected to make monthly home visits and deliver monthly community group sessions. In general, adherence to the frequency of home visits was observed in this evaluation. Group sessions are seldom delivered in the routine program due to a lack of objectives; therefore, in consultation with the LHW program management, it was agreed to focus the introduction of the new intervention (i.e., responsive stimulation) in group sessions. In this evaluation, the combination of delivery of inter-ventions with home visits supported the groups. Parenting group meetings occurred, as expected with each LHW delivering on average 1-2 meetings per month. The approach regularly reached onethird of female primary caregivers, mostly mothers, in the LHW's catchment. The parenting groups were designed with key messages repeated often and a focus on play and communication activity practice. The strategy built on peer learning and support. This allowed caregivers to drop into sessions when feasible. Importantly, the messages and opportunity to practice stimulation activities were repeated in home visits recognizing that groups would not reach all women (e.g., some women did not receive permission). The groups did play a key role in reaching other female caregivers and creating a local awareness of early childhood development.
The number of expected home visits an LHW should make per day (five to seven households per day) is determined by the LHW program based on the size of catchments and proximity of homes. Over the course of the month, each LHW should visit each household at least one time. This evaluation showed that families in the intervention arms and in the comparison arm generally received their routine monthly home visit from their LHW. The duration of the home visit was as expected for the LHW program in the intervention areas, but shorter in the comparison area indicating underutilization of time. This may suggest the approach to the delivery of recommendations was brief in the comparison arm and may not be conducive to behavior change. Moreover, rather than dilute routine services, families and LHWs perceived an enhancement of services. Families perceived that they were having more contact with LHWs with these strategies. Increased perception of contact may have been the result of parenting groups because home visit frequency did not differ from the comparison group although duration of contact was longer. In addition, home visit contacts in the enriched intervention groups were more likely to cover a broader range of child health, nutrition, and development topics, while the comparison arm (routine services) tended to have a polio/vaccine focus. Families appreciated the new enriched early childhood recommendations. The enhancement may have been supported by using training and supportive supervision as platforms to integrate existing and new messages, and draw on linkages between health, growth, and development. In summary, the design of delivery strategies needs to not only consider the socio-cultural context of the local community, but also the alignment with the existing services. Evaluation should aim to assess feasibility and fidelity as well as any impact (positive or negative) on the existing services. An overall change in knowledge and practices with respect to care for development and feeding was found suggesting receptiveness to both interventions in the community. Satisfaction was indicated by expressed benefits by the community, such as achievement of their child's developmental milestones and a healthier, happier child who would be more ready for school. Communication within the household about both interventions also suggested satisfaction in the community. Use of metaphors and ice breakers to illustrate key messages about how children develop and learn was recalled by families indicating their usefulness in delivering complex information (e.g., a baby's brain can absorb lots of new information like a sponge). Behavior change techniques such as active coaching rather than information sharing alone are likely to have encouraged change in knowledge and practices. 8 Acceptance and uptake of the MMS was slower in the community and a refresher training was helpful for LHWs to address community concerns regarding the MMS, which highlights an important learning about using refreshers to respond to identified knowledge and practice needs in services. It is worth noting that while the implementation evaluation indicates improved delivery of nutrition recommendations in the enhanced nutrition arms of the trial, this did not result in improved growth outcomes. While effective delivery of nutrition messages is essential, the nutrition inputs require further attention. For example, to reduce stunting, more attention might be needed on issues of food security to improve not only feeding behaviors, but also sufficiency and quality of complementary foods, maternal nutrition (preconception and during pregnancy), and birth spacing.
Quality of implementation improved over time. Skills were weaker in the first 6 months compared with the subsequent phases of the intervention roll-out highlighting the importance of continuous supportive supervision and problem solving to overcome challenges to implementation. For scale, successful integration of interventions would require capacity building at the level of supervisory, training, and management personnel in order to support skills and competencies of front line workers. In addition, quality was improved by acting on timely monitoring data. The LHW program has a strong monitoring system, but use of the data in a timely manner to improve implementation is necessary for efforts to scale-up new interventions as well as maintain quality of existing interventions.
The strengths of the implementation evaluation included a mixed methods approach which enabled exploration of the trends observed in the monitoring records, data collection over time enabling an understanding of program changes with respect to challenges, course corrections and skills growth, and the use of multiple sources of data. Weaknesses included not building in opportunities to report implementation to program leaders in the LHW program, which can potentially support decisions about integrating new interventions for scale-up in routine services. The opportunity to collect qualitative information in the comparison group (routine services only) to understand the community and LHWs' perceptions about program implementation challenges, and enablers was missed. Quality was assessed primarily from supervisory checklists and log books because provision for independent monitoring was not feasible; however, independent study of quality would have verified and strengthened data on this key implementation construct.
Key learnings from the implementation evaluation highlight the use of active learning techniques and behavior change techniques to deliver interventions and the importance of continuous quality improvements aided by monitoring data and feedback to guide refresher trainings and make course corrections, and supportive supervision to build skills capacity. Importantly, we did not observe a dilution of routine health services. To this end, integration of new interventions is likely influenced by content alignment and linkages between health, nutrition, and development. For scale, attention has to be given to supporting supervisory and training capacity in the LHW program. of presenting current evidence and evaluations on implementation processes, and to identify gaps and future research directions to advance effectiveness and scale-up of interventions that promote young children's development. A workshop was held on December 4 and 5, 2017 at and sponsored by the New York Academy of Sciences to discuss and develop the content of this paper and the others of the special issue. Funding for open access of the special issue is gratefully acknowledged from UNICEF and the New Venture Fund.
We are grateful to all of the families who participated in this research. We appreciate the commitment and dedication of the lady health workers, our partners, who implemented the parenting and nutrition interventions. The support of the Pakistan National Programme for Family Planning and Primary Healthcare is gratefully acknowledged. We thank UNICEF for funding this research.
The complete evaluation report on the PEDS Trial implementation is available on request to the corresponding author.