Introduction

Parent-implemented intervention (PII) is an established evidence-based intervention for improving social communication skills and decreasing problem behavior in children with autism (National Autism Centre, 2015; Steinbrenner et al., 2020), and parent involvement is a critically important element of early behavioral intervention (Division for Early Childhood, 2014; Hyman et al., 2020; National Research Council, 2001). Several systematic reviews (e.g., Akamoglu & Meadan, 2018; Black & Therrien, 2018; Pacia et al., 2021) and meta-analyses (Hampton et al., 2016; Nevill et al., 2018; Roberts et al., 2019) of PIIs have shown positive outcomes in the child’s language and communication, social engagement, and cognitive and adaptive functioning. Additionally, parents who engage in parent training programs have reported improvements in stress (e.g., Dillenburger et al., 2004; Rosenthal et al., 2019), mental health (Tonge et al., 2006), optimism (Koegel et al., 1982), self-efficacy (Iadarola et al., 2018), and parent–child interactions (Koegel et al., 1996; Oono et al., 2013).

Parent engagement is critical for the effectiveness of PIIs for young children with autism (e.g., Oono et al., 2013; Schreibman et al., 2015; Stahmer et al., 2011). Parent engagement has been operationalized in three primary domains: attendance, adherence, and cognitions (Becker et al., 2015; Staudt, 2007). Attendance (i.e., enrollment, session attendance, and completion of treatment) is necessary for the other two components of engagement. Adherence (i.e., participation during sessions and use of strategies between sessions) helps to enhance learning of new skills (Kaminski et al., 2008; Nock & Ferriter, 2005) and increases the likelihood that they are generalized to the family’s everyday life (Karver et al., 2006; Kazantzis et al., 2010). The final component of engagement involves private events such as agreement with treatment rationale, therapeutic alliance, expectations about treatment outcomes, and satisfaction with treatment.

Simply put, engagement involves a parent who enrolls in and attends coaching sessions, applies strategies learned in those sessions to their daily life, and whose needs, preferences, beliefs, and values are in concordance with the rationale, goals, and methods of intervention. The term “concordance” has been advanced in related fields to replace the term “adherence” (e.g., Bissonnette, 2008; Dickinson et al., 1999; Snowden et al., 2013). Concordance emphasizes the collaborative relationship between practitioners and parents, such that the practitioner is responsible for ensuring goodness-of-fit between their intervention and the family’s context. The present paper will use the term “adherence” to remain consistent with the parent engagement literature, however, references to concordance will be made where applicable.

A Functional, Contextual Approach to Engagement

Parent engagement has been identified as the lowest in behavioral interventions, compared with speech-language therapy, occupational therapy, developmental treatments, dietary interventions, and medication (Hock, Kinsman, et al., 2015; Moore & Symons, 2009; Shepherd et al., 2018). Additionally, parents have demonstrated difficulties with using behavioral strategies outside the intervention setting and after intervention has ended (Moore & Symons, 2009, 2011; Pickles et al., 2016). While this may be discouraging to behavioral practitioners, it is important to identify the function – the why – of low engagement before attempting to address it.

Allen and Warzak (2000) proposed a functional assessment of parental adherence by presenting an analysis of contingencies, including establishing operations (e.g., failure to establish early outcomes as reinforcers for engagement), stimulus generalization (e.g., insufficient training across different contexts), response acquisition (e.g., excessive complexity of intervention strategies), and consequent events (e.g., competing environmental contingencies). For instance, a parent’s engagement may decline when the intervention strategies fail to produce immediate changes in their child’s behavior. By setting realistic expectations from the outset, the practitioner can establish intermediate outcomes (e.g., child appropriately requesting ice cream instead of hitting) as reinforcers for adherence when access to the parents’ end goal (e.g., child tolerating being denied ice cream) is delayed.

Meanwhile, Fryling (2014) outlined a contextual approach to non-adherence, with an emphasis on addressing their wider context, i.e., potential background factors such as stress or isolation. When practitioners respond to non-adherence without considering the family context, they may inadvertently worsen adherence. For instance, in cases where the function of non-adherence is avoiding stress, simply providing additional training as a blanket response could make adherence even more aversive. By providing a function-based response that takes contextual setting factors into account, such as facilitating breaks or respite, improvements can be made not only in adherence but in the family’s health, well-being, and relationships. Thus, identifying the function of non-adherence and specific barriers to engagement can have a positive impact on intervention outcomes, along with collateral benefits for the family.

Factors Affecting Engagement

Identifying and functionally addressing barriers to parent engagement is a challenging and prevalent facet of applied practice (Ingersoll et al., 2020), and as such, has garnered research attention, a sample of which is outlined below. The Barriers-to-Treatment Participation Scale (BTPS; Kazdin et al., 1997), which was developed to measure barriers to participation in outpatient therapy for children and families, suggests that barriers to treatment include stressors and obstacles that complete with treatment, treatment demands and issues, perceived relevance of treatment, and relationship with the therapist. While this measure was not specifically intended for families of children with autism, parents of children with autism have indeed been found to be more likely to engage in behavioral interventions if the intervention is perceived to be effective (Bowker et al., 2011; Moore & Symons, 2011; Solish & Perry, 2008) and if it is not too burdensome (Carlon et al., 2013) or complex (Osborne et al., 2008). Other factors that have been found to impact engagement include access to funding, treatment contextualization (i.e., the degree to which the treatment focus matches family needs and the approach fits with their available resources and caregiving style), spousal agreement on goals and strategies, parental confidence and self-efficacy, and perceived acceptance of the child in the family and community (McConnell et al., 2015; Moore & Symons, 2011; Shepherd et al., 2018). Barriers to engagement can be thus multiple and complex, and span a wide range of factors, including logistical, sibling, child, parent, and intervention factors.

Qualitative studies have also been used to elucidate common themes in parents’ barriers and facilitators to treatment engagement (e.g., Amsbary et al., 2020; Hock, Yingling, et al., 2015; Mackintosh et al., 2012; Raulston et al., 2019; Stahmer et al., 2017). The most common themes were parents’ relationship with the practitioner and program logistics such as distance, scheduling, and childcare. Other themes included perceived effectiveness and relevance of the intervention, costs in terms of money, time, and energy, parent stress and overwhelm, social support, effective training approaches, match between intervention design and child skills, and overall satisfaction, particularly relating to improved parent–child relationships. Given the body of literature regarding barriers to engagement, identifying the resources necessary to adhere to the treatment, monitoring the ongoing burden on families, and discussing and working collaboratively to find solutions to anticipated barriers could help improve engagement (Hock, Yingling, et al., 2015).

Variations in the use of PIIs may reflect a mismatch between the interventions and the needs, values, and preferences of families, particularly across socio-economic and cultural groups (e.g., Burkett et al., 2015; Iland et al., 2012; Jegatheesan et al., 2010). Qualitative studies of minority experiences showed unique barriers such as language difficulties, limited services in rural communities, friends and family being dismissive or trivializing, stigma, blaming poor parenting, cultural concerns, and unmet needs from service providers (DuBay et al., 2018; Stahmer et al., 2019). Suggested adaptations and strategies to improve engagement include reduced intervention complexity, improved parent-professional collaboration, more explicit use of family-centered goals, better adult learning strategies, and more peer support for parents (Pellecchia et al., 2018; Stahmer & Pellecchia, 2015).

Challenges for Behavior Analysts

Despite the importance and the challenges of parent engagement in PIIs, only 27% of behavior analysts report having taken a course on parent training towards their degree or certification, and only 15% reported receiving training in a specific parent training approach (Ingersoll et al., 2020). Behavior analysts reported challenges such as having difficulty engaging families, families not progressing, not having enough skills or knowledge about parent training, and difficulty tailoring parent training to individual families (Ehrhart et al., 2014; Ingersoll et al., 2020; Shapiro et al., 2012).

An important component of parent engagement is therapeutic alliance. Parents of children with autism have reported that a positive and supportive relationship with practitioners contributes to family satisfaction and intervention engagement (e.g., Amsbary et al., 2020; Freuler et al., 2014; Hock, Yingling, et al., 2015). A survey of parents regarding their impressions of behavior analysts’ relationship and compassionate care skills indicated that behavior analysts show some core deficits, such as compromising, inquiring about satisfaction, and not being too authoritarian (Taylor et al., 2018). A follow-up survey by LeBlanc and colleagues (2020) found that most behavior analysts view skills in compassionate care as very or extremely important but received little or no training in these areas. Given the gaps in training and the difficulties behavior analysts have reported in their delivery of parent coaching, practical guidance can help practitioners develop and implement a successful PII.

A Potential Solution

One potential avenue to assist behavioral practitioners is to provide them with a practical, systematic tool to complement their PII. For example, the BTPS (Kazdin et al., 1997) was developed to measure potential barriers to treatment on a 5-point scale. This scale helps researchers and practitioners identify potential barriers but does not provide guidance in how to address them. Another tool, the Parent and Caregiver Active Participation Toolkit (PACT; Haine-Schlagel & Bustos, 2013), focuses on teaching evidence-based strategies to build a positive therapeutic relationship and increase engagement, such as focusing on strengths and effort and jointly identifying and problem-solving barriers. This toolkit effectively provides guidance on general engagement strategies, but does not focus on identifying unique family factors that impact engagement. A gap persists for a tool that helps practitioners identify barriers and facilitators to engagement, while also providing strategies to improve engagement by systematically adapting a PII to fit an individual family’s needs.

Thus, the purpose of the current study is to develop the Parent-coaching Assessment, Individualization, and Response to Stressors (PAIRS), a tool to help behavioral practitioners identify and address barriers and facilitators to intervention engagement using a contextual, functional approach within a compassionate care framework. The current study sought to develop the PAIRS in three stages and answer the following research questions:

Stage 1: What are potential solutions to common barriers to parent engagement?

Stage 2: Are the solutions to these barriers relevant, effective, and appropriate, as rated by a panel of BCBAs?

Stage 3: Do BCBAs think the PAIRS tool would be useful for implementing PIIs?

Method

The process of developing the PAIRS was adapted from the approaches outlined by Kassam-Adams and colleagues (2015) and Halek and colleagues (2017), who developed an e-health intervention for teaching coping skills and an assessment system for challenging behavior in residents with dementia, respectively. To the authors’ knowledge, this approach to tool development is novel in the field of behavior analysis. The development of the PAIRS consisted of three stages (Fig. 1): Stage 1 consisted of a non-systematic collation of barriers and facilitators from the literature and the proposal of function-based solutions to common barriers; Stage 2 consisted of an evaluation by an expert panel of the content validity of the proposed solutions, i.e., relevance of solution to barrier, likely effectiveness of solution to address barrier, and appropriateness for BCBAs to implement solution; and Stage 3 consisted of a follow-up evaluation by a smaller subset of the original panel on the structure of the tool and its potential utility for decision-making and day-to-day work with families. The first evaluation of the PAIRS focused on barrier-solution pairings at the item level using a content validity survey, while the second evaluation examined the tool as a whole by presenting a workshop on the PAIRS and then engaging in discussion and completing a questionnaire.

Fig. 1
figure 1

Adapted from Halek et al., 2017

Stages of the development and evaluation of the PAIRS.

Full size image

Parents were not consulted during this early stage of tool development because (1) the barriers, facilitators, and potential solutions were drawn from studies that showcased parent perspectives, allowing the research team to draw from parent experiences to develop the tool without recruiting new parent participants; (2) parent evaluation of the tool is likely to be most valuable and representative when recruited at a later stage of tool development, during real-world clinical practice when they are engaged in a PII; and (3) participation in a later stage of tool development allows the research team to ensure parent participants receive an improved version of the tool, after it has been refined through multiple evaluations.

Stage 1: Development of the PAIRS

First, common barriers and facilitators were listed, using information obtained from a collation of relevant research. This was not a systematic review of the literature on this topic; the first author identified relevant articles by searching databases using combinations of terms such as “barriers”, “facilitators”, “autism”, “behavior”, “intervention”, and “parents”, and recorded the barriers and facilitators reported. Searches concluded when no new barriers and facilitators were identified through additional searches. Findings from over sixty relevant articles informed the initial list of barriers and facilitators (see Table 1 for a sample). This list was condensed by grouping similar facilitators and barriers together, and then further collapsed by the second author. This was done to allow for enough differentiation to propose distinct function-based solutions, while avoiding an unnecessarily lengthy list of barriers that could impact the practicality and utility of the tool for everyday clinical use.

Table 1 Definitions of facilitators and barriers and examples of research
Full size table

Specific barrier-solution pairings were then delineated by proposing potential function-based solutions to address each identified barrier. Solutions were developed based on findings drawn from the literature, including identified facilitators to engagement (see Table 1 for a list of common facilitators) and specific recommendations provided by parents when discussing their barriers and experiences with PIIs (e.g., Amsbary et al., 2020; Stahmer et al., 2011). Solutions were also informed by the collective professional experience of the research team, which comprises two BCBAs and two BCBA-Ds. This led to the first version of the PAIRS (version 1.0), which comprised a table of barriers and potential solutions.

Stage 2: Evaluation of Content Validity

This stage of evaluation focused on the content validity (i.e., relevance, likely effectiveness, and appropriateness) of each item (barrier-solution pairing). The purpose of this stage was to investigate which solutions needed to be revised or discarded, and to obtain suggestions from an expert panel on additional function-based solutions to address each barrier. This stage of tool development focused on recruiting feedback from BCBAs as the intended administrators of the tool. The research team intends to gain feedback from parents in future studies through the use of the tool in real-world settings, after the tool has been refined through the evaluations and revisions presented here.

Participants and Recruitment

Twenty-eight BCBAs were invited to participate, with a response and completion rate of 53.6% (15 participants). This is consistent with recommendations for the first iteration of content validation, which suggests a panel of 8–12 experts (Lynn, 1986; Polit et al., 2007). The research team recruited BCBAs who work regularly with parents and caretakers by sending an email invitation to participate in an expert panel (i.e., convenience sampling). Eligibility requirements for participation included certification with the Behavior Analyst Certification Board (BACB) and professional experience delivering parent training interventions. BCBAs were recruited from several countries, including the United Kingdom, Ireland, Australia, and the United Arab Emirates. The majority of participants reported 11–20 years of experience in behavior analysis and served clients with ASD aged 3–12 (see Table 2 for expert panel demographics). Only responses from BCBAs who completed the survey in full were included in the analysis.

Table 2 Demographics of expert panel
Full size table

Instrument and Distribution

The Content Validity Survey Tool (CVST; Kassam-Adams et al., 2015) was adapted for the current study and the three dimensions of content validity were revised to apply to barriers and proposed solutions: (1) relevance, i.e., the extent to which a solution is pertinent to its barrier; (2) likely effectiveness, i.e., the extent to which the solution would successfully address its barrier; and (3) appropriateness of the solution for BCBAs delivering parent training services. The proposed solutions from PAIRS version 1.0 were divided into two groups: intervention adaptations and additional services or referrals. This was done to streamline the content validity evaluation by only presenting the intervention adaptations in the CVST. The adapted CVST was thus populated with 45 barrier-solution pairings, consisting of eleven barriers and 2–6 proposed solutions per barrier. Response options were on a four-point scale ranging from “not at all” to “very”. The CVST also contained a suggestion box for reviewer comments after each barrier and a final section for comments on the PAIRS as a whole. The CVST was uploaded to Qualtrics, an online survey tool, and a link to the survey was emailed to all invited BCBAs.

Analysis

Results of the CVST were analyzed quantitatively (via calculation of content validity indices; CVI) and qualitatively (via examination of narrative comments). The CVI was calculated for each scale of item (I-CVI), i.e., the relevance, effectiveness, and appropriateness of each barrier-solution pairing, and each scale overall (S-CVI/Ave), i.e., the solutions’ overall relevance, effectiveness, and appropriateness. For each item, the I-CVI was computed as the number of experts giving a rating of either 3 or 4 divided by the number of experts. The recommended minimum I-CVI value for five or more experts is 0.78 (Polit et al., 2007). The average agreement for each scale overall (S-CVI/Ave) was calculated by calculating the average I-CVI across all items. The recommended minimum value for S-CVI/Ave is 0.9 (Polit et al., 2007). Qualitative comments were categorized into suggestions and comments, and the suggestions were grouped into similar factors within each barrier to allow for narrative examination. The PAIRS table of barriers and solutions was then refined based on the combined quantitative and qualitative findings. To better support practitioners for whom the PAIRS is intended, a more comprehensive tool was developed with additional components derived from this table that corresponded with the typical stages of implementing a PII (i.e., developing and planning, conducting an intake assessment, and finally implementing the intervention). This expanded tool was labelled PAIRS version 2.0.

Stage 3: Evaluation of Utility

While the first iteration of evaluation focused on barrier-solution pairings at the item level, this second iteration sought feedback on the revised tool as a whole, specifically its structural aspects and potential benefits for decision-making and day-to-day work with families. This was done through a workshop presenting PAIRS version 2.0 to a subset of the original panel, followed by a discussion and questionnaire.

Participants and Recruitment

In line with previous recommendations, a subset of the same experts who participated in the primary evaluation of an instrument re-evaluated the revised version (Halek et al., 2017; Lynn, 1986). All the members of the original panel were invited via email to attend the PAIRS workshop and re-evaluation (n = 15). Six members of the panel were unable to attend due to scheduling conflicts, two members registered for the workshop but did not attend, and three members did not respond to the invitation. Four members of the panel attended the workshop and participated in the re-evaluation. Caution should be taken when evaluating the results of the second evaluation due to the relatively small number of participants, although the size of the panel is consistent with the recommendation of recruiting 3–5 reviewers for a secondary review (Polit et al., 2007).

Workshop

The workshop took place over Zoom, an online meeting platform. The first author delivered the one-hour workshop to the subpanel. At the beginning of the workshop, participants were asked to describe a family they had worked with who they found difficult to engage, and the circumstances that made it challenging. Participants were then asked to keep this family in mind as the PAIRS was introduced, and to consider how having access to the PAIRS may have impacted how they worked with the family. An introduction to the background and rationale of the PAIRS was presented, including the functional, contextual approach to engagement and the role of concordance. This was followed by an overview of the results from the first evaluation (i.e., CVI, narrative comments, and item revisions), which they had participated in. Finally, the PAIRS tool was presented and explained in full.

Evaluation and Analysis

At the end of the workshop, participants engaged in a brief discussion and were then provided a link to a questionnaire. The questionnaire had four sections: (1) Structure; (2) Benefits of the tool for planning and decision-making; (3) Benefits of the tool for day-to-day work with families; and (4) Overall impression of the tool. Response options were on a four-point scale ranging from “strongly disagree” to “strongly agree.” Comments and suggestions were requested at the end of each section. The discussion portion of the workshop was also transcribed and qualitatively examined. Questionnaire data (quantitative and qualitative) were reviewed. The PAIRS was then further revised based on this feedback (PAIRS version 3.0).

Results

Stage 1 results included the collation of common barriers and facilitators and the development of potential solutions, Stage 2 results included the quantitative (i.e., content validity indices) and qualitative (i.e., examination of narrative comments) findings from the content validity evaluation, and Stage 3 results included descriptive statistics from the questionnaire and revisions based on the post-workshop discussion and written comments.

Stage 1: Development

Forty-eight barriers and 41 facilitators were identified. After grouping similar factors together, 11 barriers and six facilitators remained (Table 1). Multiple solutions were proposed for each barrier (Table 3). For example, for the barrier “Difficulties with access”, four solutions were proposed, including connecting families with charities, insurance, funding supports, etc., where appropriate, providing Telehealth or home-based training, providing flexible training days/times, and developing explicit strategies for involving the child and/or sibling. Thus, PAIRS Version 1.0 consisted of a list of 6 facilitators and a table of barriers and proposed solutions. This table comprised 11 barriers and 2–6 solutions per barrier, for a total of 45 barrier-solution pairings.

Table 3 Barriers with original and revised solutions based on I-CVI values and narrative suggestions
Full size table

Stage 2: Evaluation of Content Validity

The 45 barrier-solution pairings were reviewed by fifteen BCBAs for content validity (i.e., relevance, effectiveness, and appropriateness) (see Table 3). I-CVI values ranged from 0.53 (N = 1) to 1 (N = 35), which is the highest possible I-CVI value. The recommended minimum I-CVI value for five or more experts is 0.78 (Polit et al., 2007). Content validity indices across barrier-solution pairings are presented in Table 3. Thirty-three (73.3%) items had I-CVI values at or above the recommended minimum value across all three scales. Eight items (17.8%) had one scale below the recommended value, while four items (8.9%) had two scales below the recommended value. Overall scale values were above the recommended minimum value of 0.9 for relevance (S-CVI/Ave = 0.92) and appropriateness (S-CVI/Ave = 0.91), but slightly below the recommended minimum value for effectiveness (S-CVI/Ave = 0.85).

CVI for Logistical Factors

For the barrier “Difficulties with access”, two solutions (50%) had I-CVI values above the recommended value for all scales. The proposed solution to connect families with charities, insurance, funding supports, etc. was below the recommended value for effectiveness (I-CVI = 0.73) and the solution to include explicit strategies for involving child/sibling was below the recommended value for relevance (I-CVI = 0.73) and effectiveness (I-CVI = 0.67). For the barrier “Administrative difficulties”, one solution (33.3%) had I-CVI values above the recommended value across all scales. One solution (Provide or reduce need for additional therapy materials) had was below the recommended value for appropriateness (I-CVI = 0.67), and one solution (Telehealth or clinic-based training) was below the recommended value for relevance (I-CVI = 0.73) and effectiveness (I-CVI = 0.73).

CVI for Child and Sibling Factors

For the barrier “Complex child profile”, two solutions (66.7%) had I-CVI values of 1 across all scales, which represents the maximum value. One solution (Multidisciplinary collaboration) was below the recommended value for effectiveness (I-CVI = 0.73). For the barrier “Difficulties with siblings”, both solutions (100%) had I-CVI values above the recommended value across all scales.

CVI for Parent Factors

For the barrier “Individual or cultural concerns”, five solutions (83.3%) had I-CVI values above the recommended value across all scales. One solution (Shape successive approximations and establish intermediate outcomes as reinforcers) was below the recommended value for relevance (I-CVI = 0.67) and effectiveness (I-CVI = 0.73). For the barrier “Difficult living circumstances”, three solutions (75%) had I-CVI values above the recommended value across all scales. One solution (Reduce treatment burden and/or complexity) was below the recommended value for effectiveness (I-CVI = 0.73). For the barrier “Treatment burden and stress”, five solutions (83.3%) had I-CVI values above the recommended value across all scales. One solution (Distance-learning options) had was below the recommended value for effectiveness (I-CVI = 0.6). For the barrier “Training not the right fit”, two solutions (66.7%) had I-CVI values above the recommended value across all scales. One solution (Provide different training options) was below the recommended value for effectiveness (I-CVI = 0.73). Finally, for the barrier “Low motivation/belief in effectiveness”, four solutions (80%) had I-CVI values above the recommended value across all scales. One solution (Share parent-friendly data sheets) had was below the recommended value for effectiveness (I-CVI = 0.67).

CVI for Intervention Factors

For the barrier “Variations in treatment efficacy”, three solutions (75%) had I-CVI values above the recommended value across all scales. One solution (Explicitly acknowledge that strategies may not always be effective or that there may be setbacks and plan for this early on) was below the recommended value for effectiveness (I-CVI = 0.53). For the barrier “Difficulties with generalization”, four solutions (80%) had I-CVI values above the recommended value across all scales. One solution (Connect with peer support) had was below the recommended value for effectiveness (I-CVI = 0.73) and appropriateness (I-CVI = 0.73).

Narrative Suggestions

Narrative suggestions were examined for overarching factors leading to the identification of six common factors: Flexibility and accommodation, coordination between services, addressing additional needs/providing additional services, incorporating Acceptance and Commitment Therapy (ACT), parent-practitioner collaboration, and proactive planning and setting of initial expectations. These factors were consistent with identified facilitators from the literature, except for incorporating ACT and proactive planning and setting of initial expectations. These two suggestions were novel findings and were incorporated into the proposed solutions. The narrative comments were then divided into comments and specific suggestions. For each barrier, similar suggestions were grouped together and summarized for examination. Individual solutions were then revised based on a combination of the quantitative and qualitative feedback (see Table 3 for revised solutions and the relevant comments/suggestions).

Revision

The table of barrier-solution pairings from PAIRS Version 1.0 was updated with the revised solutions. To support practitioners using the tool, additional components of the PAIRS were derived (see Table 4), and the table of barrier-solution pairings was named the PAIRS Table of Function-Based Solutions. As using a strengths-based approach was identified as a facilitator, facilitators are always presented before barriers within each component of the tool.

Table 4 Comprehensive PAIRS components
Full size table

Stage 3: Evaluation of Utility

The four members of the sub-panel responded “agree” or “strongly agree” to all items on the questionnaire, including items evaluating the structure of the PAIRS (i.e., clarity, meaningfulness, completeness, length and scope, and comprehensibility), its potential benefits as for day-to-day work with families (i.e., time required and utility as an information aid, communication aid, planning aid, and evaluation aid), potential benefits as a tool for assessment and decision-making (see Table 5), and overall assessment of the PAIRS (i.e., usefulness and practicability, use in day-to-day work, recommendation to other BCBAs).

Table 5 Questionnaire results for PAIRS benefits for day-to-day work with families
Full size table

Comments were also received within each of the four sections. Based on a combination of the verbal (during discussion) and written (from questionnaire) comments and suggestions, the following revisions were made: (1) Background and rationale were added to the tool, as a participant noted that concepts introduced during this section of the presentation were impactful; (2) Contents of the appendix (definitions of barriers and facilitators and a list of terms) were moved to the front of the tool to improve clarity; (3) Practitioner skills and behaviors (i.e., using a non-directive collaborative approach, using a strengths-based approach) were operationalized more clearly within the body of the tool, as suggested by a participant; (4) Additional readings were added to the list of terms, including specific suggestions from participants; (5) Motivational Interviewing was added as a potential solution alongside ACT, as suggested by a participant; (6) A section of “Additional Considerations and Limitations” was added with advice on administration, adaptation, and limits of the tool, based on comments from multiple participants during the discussion.

Discussion

The PAIRS was developed grounded in the functional contextual approach to engagement (Allen & Warzak, 2000; Fryling, 2014; Moore & Amado, 2021) and in response to recent calls for increased attention to compassionate care for behavior analysts (Callahan et al., 2019; Leblanc et al., 2020; Taylor et al., 2018). The PAIRS was designed to collate the information drawn from a growing body of literature examining common barriers and facilitators to engagement and make it accessible to practitioners delivering PIIs, and help them tailor their intervention to individual families. In the first evaluation of the PAIRS by a panel of fifteen BCBAs, the overall average rating of the relevance and appropriateness of all proposed solutions were each over 0.9, indicating that on average, over 90% of ratings agreed that the proposed solutions were relevant to the barriers and appropriate for use by BCBAs. The average rating of the effectiveness of the solutions was slightly lower, at 0.85. This indicates that on average, 85% of ratings agreed that the proposed solutions were likely to be effective at addressing the barriers. One potential explanation for this slightly lower rating for effectiveness is that family barriers are complex and are unlikely to be easily addressed with a single solution. Thus, the individual barrier-solution pairings presented in the CVST may have been rated as less effective at the item level since multiple solutions were proposed for each barrier. Nevertheless, it is worthwhile to note that 85% is still a high level of agreement.

While the first evaluation of the PAIRS focused on evaluating the tool at the item level, the second evaluation examined the tool as a whole. A subset of the original panel attended a workshop which introduced them to the comprehensive PAIRS tool. It should be noted that the second evaluation was conducted with only four participants. While this is in line with recommendations for studies of content validity (e.g., Polit et al., 2007), their evaluation may not fully generalize to the wider behavior analytic community. Nevertheless, in a discussion at the end of the workshop, all participants noted the lack of training and guidance they had received on how to work with parents. One participant noted, “A lot of the things you’ve gone through in the last hour [of the workshop], it just made me think, where have all these things been over the last few years?” This is consistent with Ingersoll and colleagues’ (2020) findings that the majority of behavior analysts have not received adequate support in how to effectively conduct parent training. All participants highly rated the PAIRS’ structure and benefits for decision-making and day-to-day work with families. One participant commented, “I see it becoming part of the essential guides required to implementing successful parent training. Its comprehensive and user-friendly layout is exactly what is needed for behavior analysts to effectively support and train parents.”

The PAIRS was designed to be complementary and collaborative. That is, the PAIRS is intended to complement an evidence-based intervention; it is not a standalone program. The use of a tool to enhance and complement existing interventions is preceded by PACT, a toolkit for promoting parent engagement (Haine-Schlagel & Bustos, 2013). For instance, PACT has been integrated into Project ImPACT (Ingersoll & Dvortcsak, 2010), an evidence-based PII for improving social communication skills in children with ASD. The engagement strategies are incorporated into the parent training lessons, e.g., when teaching parents of children with ASD how to make play interactive, practitioners are encouraged to make suggestions rather than giving directions (Rieth et al., 2018). While PACT provides good-practice strategies that are helpful for practitioner interactions with most families, the PAIRS provides strategies for individualizing interventions based on specific family variables. Thus, the PAIRS can similarly be incorporated into a PII, where it can provide systematic guidance to practitioners on jointly identifying barriers and facilitators and problem-solving challenges unique to each family, consistent with building a positive and empowering therapeutic relationship. Furthermore, the PAIRS is complementary in that it provides information and suggestions for further reading and training for skills and concepts that may be unfamiliar to practicing behavior analysts (e.g., therapeutic relationship skills), but may be required to produce the greatest benefit.

The PAIRS is also intended to be used collaboratively. The function-based solutions presented are intended to guide the collaborative problem-solving process with families, not to prescribe rigid responses to complex situations. The PAIRS thus encourages concordance rather than strict adherence and emphasizes open communication and joint problem-solving. This is supported by the narrative comments received. For example, one reviewer noted “…the importance of including the parents and facilitating them to elicit the solutions for change, as then there is more buy in, and it reduces the ‘expert model’ of the parents relying on you to solve this problem.” Another reviewer commented, “Adopting a collaborative approach, which listens carefully to the concerns of the family, is vital. This is an ongoing ‘soft’ skill for BCBAs to develop and requires ongoing work.” During the second evaluation discussion, several members of the panel noted how going through the PAIRS with a family could be a way to connect and foster concordance by helping practitioners better understand the family’s context.

This is in line with improving relationship and compassionate care skills for behavior analysts. In the second evaluation of the PAIRS, the four members of the secondary panel strongly agreed that the PAIRS encouraged them to think more intensively about the family’s contextual barriers and see the family’s challenges from another perspective. One participant commented, “The categories of barriers and facilitators help me to think through other idiosyncratic variables specific to the family and how they might fit within the PAIRS framework.” By illustrating the contextual variables that commonly impact families of children with ASD and providing guidance on potential areas of professional development, the PAIRS seeks to help practitioners build positive therapeutic alliance with families. The feedback received from the panel is encouraging in this regard, however further evaluation by both practitioners and families in real-world clinical settings is needed.

When considering the barriers and facilitators to engagement within a family system, the PAIRS takes a multi-level approach, with the chosen level of analysis defined by its pragmatic purpose (Hayes et al., 2021). That is, the PAIRS expands the traditional behavioral paradigm from the level of the individual to the level of the family, and from immediate environmental contingencies to nonlinear contextual variables, including private events (e.g., values and beliefs) on one end of the spectrum and large-scale external factors (e.g., socioeconomic disadvantage, cultural concerns) on the other. This breadth of scope is necessary because the barriers and facilitators to engagement identified in the literature cover this continuum. The PAIRS recognizes that a single practitioner cannot address all the potential challenges a family may experience. Thus, the PAIRS encourages practitioners to focus on adaptations that they can make within their program, and advocates for appropriate referrals to other services as well as positive and coordinated multi-disciplinary collaboration. It is important to note that any given analytic focus should not diminish the relevance of other levels of analysis (Hayes et al., 2021). Thus, the PAIRS does not eschew the importance of careful analysis of behavior using information gathered around proximal antecedent and consequent variables. Instead, the PAIRS aims to expand on this information to provide practitioners with a systematic and practical way of identifying and addressing a wide range of difficulties, utilizing the level of analysis that is pertinent and helpful to a given situation.

Strengths, Limitations, and Future Directions

Within this research, a systematic literature review of the barriers and facilitators to parent engagement was not conducted. However, as previously outlined, a growing literature base exists on this topic, and collation of common barriers and facilitators from over sixty research articles was possible without utilising a systematic review methodology. The eleven barriers and six facilitators represent a synthesis of the specific factors found in a sample of the literature, which had numerous areas of overlap. It is recognised that this is not necessarily a definitive list, and the PAIRS is designed to be a flexible and adaptive tool that can continue to be updated with new literature as well as individualized to a specific family’s unique strengths and needs.

The present study utilizes a methodology for content validation that is novel in the field of behavior analysis. The CVST was adapted from a medical context to fit the parameters of the PAIRS. By recruiting an expert panel to evaluate the content validity of each barrier-solution pairing and consulting a subset of that panel to evaluate the tool as a whole, the PAIRS is thus developed with the expertise of the professionals it is designed for. While the present study focused on developing the PAIRS based on the available literature and conducting two rounds of expert review to evaluate its content validity, it is recognised that the sample of fifteen BCBAs, and especially the smaller subset of four BCBAs who participated in the second evaluation, may not fully represent the professional experiences of behavioral practitioners working in a variety of contexts. Therefore, future research will examine the use of the PAIRS with larger samples within real-world clinical practice. The PAIRS is a tool developed by BCBAs for BCBAs and related practitioners (e.g. behavioral psychologists, assistant behavior analysts). In this way, the current intended use of the PAIRS is limited to behavioral PIIs. Given the novelty of the PAIRS, the research team wishes to avoid overextending its reach at the outset. As further research is conducted using the tool to complement evidence-based PIIs, there may be scope to adapt and expand its utility for professionals from other disciplines (e.g., early childhood educators, early interventionists) who likewise work with families of children with autism.

Finally, the present study focused on evaluation of the tool by a panel of professionals. Direct parent input was not sought at this stage of tool development for a few reasons. First, the barriers, facilitators, and solutions in the PAIRS were drawn from the literature presenting parents’ views in the form of interviews (e.g., Amsbary et al., 2020; Botterill et al., 2019; Freuler et al., 2014) or surveys (e.g., Bowker et al., 2011; DuBay et al., 2018; Rovane et al., 2020). Thus, the tool is created from the experiences and recommendations of the parents it is intended to serve, using research that is already available. Second, parent feedback would likely be most valuable and representative when the PAIRS is used in practice to address facilitators and barriers that are relevant to their unique situation. Finally, parent participation at a later stage of tool development is also likely to benefit families, as the PAIRS will have gone through the present evaluation and piloted before it is administered. Therefore, future research will focus on the family’s evaluation of the revised tool’s goals, procedures, and outcomes (i.e., social validity; Wolf, 1978) when it is used in clinical practice.

Summary

The PAIRS was developed and revised in conjunction with an expert panel of BCBAs through two rounds of content evaluation to fill a gap in parent training literature and practice. It is a systematic, practical tool designed to complement evidence-based PIIs. Parent engagement is an essential component of PIIs, but can be impacted by numerous barriers. The PAIRS is the first tool designed to help practitioner assess barriers and facilitators to engagement, individualize their parent training approach, and respond to stressors using a contextual, functional approach.