Introduction

Staff in intellectual disability services may experience significant work stress (Skirrow and Hatton 2007). Prolonged exposure to high levels of such stress may lead to burnout, which is characterized by emotional exhaustion, impersonal attitudes toward clients, and a decreased sense of accomplishment at work (Maslach et al. 1996). Heightened levels of stress and burnout may contribute to high rates of staff turnover and absenteeism (Hastings et al. 2004), and staff who are experiencing high levels of burnout may interact less frequently with clients and engage in fewer positive interactions with them (Lawson and O'Brien 1994; Rose et al. 1998a).

Research suggests that coping strategies may be an important factor in determining whether staff experience negative psychological outcomes related to work stress (Devereux et al. 2009). For example, Devereux et al. (2009) found that wishful thinking coping, a strategy based on emotional avoidance, partially mediated the relationship between work stressors and burnout in support staff, such that staff who engaged in wishful thinking reported higher levels of burnout. Hastings and Brown (2002) also found that special education staff who were exposed to high levels of challenging client behavior and adopted avoidant coping strategies reported higher levels of burnout than staff who were exposed to high levels of challenging behavior but did not adopt such strategies. Conversely, Noone and Hastings (2011) found that psychological acceptance, a strategy characterized by a willingness to experience difficult thoughts and emotions, was negatively correlated with emotional exhaustion burnout in a sample of intellectual disability support staff.

Given the importance of effectively coping with work stressors, surprisingly few data are available on work stress interventions for intellectual disability staff. A handful of studies have demonstrated decreases on measures of work stress (e.g., Gardner et al. 2005; Keyes and Dean 1988; Noone and Hastings 2009; Rose et al. 1998b). Even fewer intervention studies have demonstrated evidence for important behavioral outcomes such as decreased use of restraint in response to challenging behavior (Keyes and Dean 1988) and increased frequency and quality of interactions with clients (Rose et al. 1998b). In one such study, Singh and colleagues (2006) examined the effect of augmenting behavior analysis training for staff with mindfulness training, which makes use of meditation practice to facilitate acceptance of difficult thoughts and emotions. Results indicated substantial decreases in challenging client behavior and increased client progress on learning objectives after the addition of mindfulness training for staff.

One intervention that has been tested in the area of work stress is Acceptance and Commitment Therapy (ACT; Hayes et al. 1999), which is also known as Acceptance and Commitment Training when applied to non-clinical populations. ACT attempts to foster psychological flexibility, which may be described as the ability to experience both pleasant and unpleasant thoughts and emotions without engaging in maladaptive avoidance behaviors. When psychological flexibility is increased, individuals may behave in a manner that is more consistent with personal goals and values (Bond et al. 2006).

One of the means by which ACT may increase psychological flexibility is through the use of cognitive defusion techniques. These techniques resemble mindfulness skills in that they teach individuals to discriminate themselves from their thoughts and foster nonjudgmental acceptance of one's experience (Hayes et al. 2006). The effects of defusion are typically demonstrated by a decrease in the believability of thoughts (Hayes et al. 2006), although believability is not directly targeted.

Noone and Hastings (2009, 2010) have examined the effect of an ACT-based stress management intervention for intellectual disability staff. The intervention group reported a reduction in general psychological distress from pretest to 6-week follow-up, while their perception of work stressors did not change significantly over time. There were no changes in distress or perception of work stressors for the wait-list control group. These findings are consistent with the ACT model, which suggests that psychological flexibility may promote well-being even as staff continue to perceive stressors in their work environments (Hayes et al. 2006). However, the authors noted that researchers could improve further evaluations of ACT interventions in this population by including an active control group, measures of change processes (Noone and Hastings 2010), social validity data to assess the acceptability of ACT interventions, and consideration of the extent to which participants apply the techniques taught in the intervention (Noone and Hastings 2009).

The aim of the present study was to evaluate the effects of a workshop for intellectual disabilities staff that combined ACT with training in applied behavior analysis (ACT + ABA). This combined intervention was compared with a control condition in which staff were trained in ABA only. We predicted that participants in the ACT + ABA condition would report decreases in burnout and psychological distress over time. No such changes were expected to occur in the ABA control condition. We also predicted that the participants in the ACT + ABA condition would exhibit a concurrent decrease in their ratings of the believability of thoughts related to burnout. A social validity survey was also administered to assess staff perceptions of the workshops.

Method

Participants

Participants were recruited from a large, state-funded residential facility for individuals with intellectual disability in the USA. The facility provides 24-h care to several hundred individuals for whom placement in a less restrictive setting is often precluded by challenging behavior, severe adaptive deficits, serious medical conditions, or some combination thereof. The participants came from various professional and paraprofessional departments involved in direct service provision, including psychologists and psychological technicians (n = 14), special education teachers and assistant teachers (n = 11), direct care staff (n = 10), nurses (n = 2), and social workers (n = 1). The staff were offered continuing education credit in exchange for their participation.

Originally, 38 participants consented to participate in the present study, with 20 randomly assigned to the ACT + ABA condition and 18 assigned to the ABA control condition. Two participants from each condition did not complete the study. One of the individuals in the ACT + ABA condition had to leave the study due to illness in the immediate family, while the other left for unknown reasons. One individual in the ABA condition resigned from the sponsoring agency while the other dropped out for unknown reasons. This led to a final total of 18 participants in the ACT + ABA condition and 16 in the ABA control condition.

Of these 34 participants, 17 identified themselves as Caucasian and 17 identified themselves as African American. Participants included 26 women (76.5 %) and 8 men (23.5 %). Twelve (35.3 %) participants had completed some college coursework, 9 (26.5 %) had bachelor's degrees, 11 (32.4 %) had master's degrees, and 2 (5.9 %) had doctoral degrees. The mean age of the participants was 38 years (range, 22–60), and the mean length of time employed at the facility was 7.6 years (range, 0.17–25.75).

One-way ANOVAs comparing the two groups on age and length of employment revealed no significant differences between the groups. Chi-square analyses also indicated no significant differences in gender, ethnicity, and level of education between the groups.

Outcome Measures

General Health Questionnaire-12 (Goldberg 1978)

The General Health Questionnaire (GHQ) is a 12-item scale that is typically used as a measure of general psychological distress. Items are scored on a 4-point Likert-type scale ranging from 0 (not at all) to 3 (much more than usual). High scores indicate poorer mental health, with scores equal to or greater than 11 predicting the presence of a psychological disorder with 78.9 % sensitivity and 77.4 % specificity (Goldberg et al. 1997). Bond and Bunce (2003) reported alpha coefficients of .84 and .85 in a non-clinical sample, indicating good internal consistency.

Maslach Burnout Inventory, Human Services Survey (Maslach et al. 1996)

The Maslach Burnout Inventory (MBI) is a widely used measure of burnout. Its reliability, validity, and factor structure have received support across a number of occupational settings, including direct care staff working in intellectual disabilities services (Hastings et al. 2004). Each of the 22 items is rated on a 7-point Likert-type scale to indicate how often respondents experience certain feelings related to their work. The inventory yields three subscale scores: emotional exhaustion (feelings of being emotionally drained and overwhelmed by one's work), depersonalization (impersonal attitudes toward service users), and personal accomplishment (feelings of achievement and competence).

Process Measure

Burnout Believability Scale

This 13-item measure was adapted from selected items from the emotional exhaustion and depersonalization subscales of the MBI, such that respondents were asked to rate the believability of the items, rather than their frequency. For example, respondents were asked to imagine that they were having a particular thought, such as “Working with people all day is a real strain.” or “I don't care what happens to some clients.” and were asked to rate the believability of that thought on a 7-point Likert-type scale ranging from 1 (not at all believable) to 7 (completely believable). The purpose of this instrument was to measure cognitive defusion. Similar adapted measures of cognitive defusion have been demonstrated to mediate change in previous ACT outcome studies (e.g., Bach and Hayes 2002; Hayes et al. 2004). A reliability analysis of participants' responses at pretest on the Burnout Believability Scale (BBS) yielded an alpha coefficient of .89, indicating good internal consistency for this measure in the present study.

Social Validity Measure

Social Validity Survey

This six-item survey was developed to assess staff perceptions of the workshops and the extent to which they had been practicing the techniques they had learned. Staff indicated the extent to which they agreed with each item on a 7-point Likert-type scale ranging from 1 (completely disagree) to 7 (completely agree).

Procedure

Participants were recruited via general announcements and flyers posted at their workplace. Once participants gave informed consent to participate in the study, they were randomly assigned to one of two conditions: ACT + ABA or ABA. The ACT + ABA condition involved 6 h of Acceptance and Commitment Training combined with 3 h of training in the principles of applied behavior analysis. The ABA condition, which served as the control condition, consisted of 9 h of didactic training in principles of applied behavior analysis. Each of these interventions consisted of three 3-h group sessions that were administered at 1-week intervals for 3 weeks. To accommodate the participants' schedules, they were offered three weekly times in which they could attend their assigned training group. Participants were asked to attend the groups at the same time each week in order to foster group cohesion. Each session was attended by four to eight participants.

Participants completed all outcome and process measures immediately before the first session of their respective workshop series, following the conclusion of the final workshop session, and 3 months after the final workshop session. The social validity survey was completed at 3-month follow-up. A brief description of each intervention follows.

Acceptance and Commitment Training and Applied Behavior Analysis

This intervention was constructed based upon examination of ACT treatment protocols relevant to worksite stress (Bond and Hayes 2002), burnout among substance abuse counselors (Hayes et al. 2004), and stress management for parents of autistic children (Blackledge and Hayes 2006). Treatment components were adapted to address the particular difficulties encountered by intellectual disabilities staff, such as dealing with emotional reactions to challenging behaviors, as well as perceived lack of support from and cooperation among coworkers. The principal investigator, who was then an advanced graduate student with 1 year of training and supervision in ACT, conducted the ACT + ABA workshops.

The first session of the workshop began with brief didactics about stress in the workplace and the ACT model. Participants were then asked to identify and share examples of work stressors and to identify thoughts, emotions, and coping strategies associated with the stressors.

The workability of these coping strategies was then examined, and the concepts of willingness and acceptance were introduced via metaphors and experiential exercises. The first day concluded with guided practice of the “Just Noticing/Leaves on a Stream” exercise (Bond and Hayes 2002, pp. 126–127), which is a beginning mindfulness exercise that is commonly employed in ACT. For “homework,” participants were asked to continue to practice this exercise, as well as an additional “Being in the Present Moment” exercise (Bethay et al. 2009, pp. 236–237) designed to promote the application of mindfulness during interactions with clients.

The second workshop session began with a review of the homework assignment from the previous session. A continued discussion of defusion ensued, with an emphasis on perspective-taking skills in difficult work situations. Staff then engaged in guided practice of the “Observer” (Bond and Hayes 2002, pp. 131–132) exercise, which attempts to foster contact with a transcendent sense of self and to encourage nonjudgmental acceptance of internal events. Staff were then asked to engage in a writing exercise in which they described a stressful work experience. After completion of this exercise, they practiced the “Tin Can Monster” (Hayes et al. 1999, pp. 171–174) exercise, in which they were asked to attend to various bodily sensations, emotions, and thoughts that emerged in response to the memory of the event that they had just written about. Staff then engaged in another experiential exercise, the “Retirement Party” (Bethay et al. 2009, pp. 240–241), a guided meditation designed to facilitate experiential contact with work-related values. Staff were then asked to list some of their values and to identify goals related to those values. Finally, participants were asked to make behavioral commitments to practice valued actions. Barriers to effective action and techniques for building larger patterns of committed action were discussed. For homework, the participants were encouraged to continue to practice the “Tin Can Monster” as well as the other two exercises from the previous homework assignment. They were also encouraged to follow through on their commitments to engage in valued action in the workplace.

The final session consisted of a 3-h lecture about the principles of applied behavior analysis derived from the Miller (2005) text, with emphasis on how participants can use ACT techniques to enhance their behavior management skills. Participants were also encouraged to continue to practice the mindfulness skills that they had learned in the previous two sessions. Table 1 provides an outline of the content of each workshop.

Table 1 Outline of workshop content
Full size table

Applied Behavior Analysis

The control condition consisted of three 3-h lectures about the principles of applied behavior analysis that were derived from the Miller (2005) text. Advanced graduate students in clinical psychology who had completed coursework in applied behavior analysis served as instructors for the control condition. The training sessions consisted of programmed instruction in which the lectures were divided into brief (15–20 min) lessons, each followed by a brief quiz which the participants completed in small groups. The instructors provided corrective feedback on the quizzes before advancing to the next lesson. Throughout the training, participants were encouraged to relate the course content to their job duties and to practice the principles and techniques they were learning between sessions.

Results

Exploratory data analysis indicated that the data from the outcome and process measures were not uniformly normally distributed for each group at every time point. However, group variances were found to be homogenous, and the sample sizes were roughly equal. Since parametric tests are known to be relatively robust to violations of normality assumptions (Tabachnick and Fidel 1996), we decided it was appropriate to analyze these data using parametric methods. While the study is small and statistical power is limited, it could potentially provide sufficient pilot data on both efficacy and feasibility to support a more substantial trial.

One-way ANOVAs revealed no significant between-group differences at pretest for any of the outcome or process variables. Next, 2 × 3 repeated-measures ANOVAs were conducted to examine differences between pretest, posttest, and 3-month follow-up scores for all outcome and process measures across the two conditions. If significant results were obtained in this first step of the analysis, then between-group t tests were used to assess differences in change scores from pretest to posttest and from pretest to follow-up. Mean ratings for each item on the social validity survey were compared via independent-samples t tests to examine between-group differences at follow-up. Table 2 lists means and standard deviations for all measures for all participants.

Table 2 Means (standard deviations) for outcome and process measures for all participants
Full size table

When the data for all participants were examined, no significant group × time interactions were observed for the GHQ-12, F(2, 64) = 2.48, p = .092. Likewise, no significant group × time interactions were observed for the subscales of the MBI: emotional exhaustion, F(2, 64) = .137, p = .873; depersonalization, F(2, 64) = 2.31, p = .107; and personal accomplishment, F(2, 64) = 1.32, p = .274. There was no analysis of the BBS for all participants due to the lack of significant treatment effects on the outcome variables.

Regarding the social validity data, participants in the ACT + ABA group were significantly more likely to agree that the workshop improved interactions with their coworkers (item 3), t(32) = 2.19, p = .04, when compared to participants in the ABA control group. They also were significantly more likely than the control group to agree that they experienced a reduction in work stress as a result of the techniques they had learned (item 4), t(32) = 2.71, p = .01. Participants in the ACT + ABA group were also more likely to find the workshop interesting and enjoyable (item 5), t(32) = 2.56, p = .02, and were more likely to report that they would recommend the workshop to their coworkers (item 6), t(32) = 2.83, p = .008. Table 3 provides the means and standard deviations of each social validity survey item for all participants.

Table 3 Social validity survey for ACT + ABA (n = 18) and ABA (n = 16) groups
Full size table

Given the lack of significant findings on the primary outcome measures, we conducted additional exploratory analyses in an effort to identify factors which may have influenced our results. In the first of these, we wished to explore whether participants who reported that they had been practicing the techniques and principles they had learned in the workshops differed in outcome from participants who reported that they had not been practicing what they had learned. Second, we examined the effect of the interventions on only those participants who reported clinically significant psychological distress on the GHQ-12 at pretest.

In order to discriminate participants who may not have practiced what they had learned in the workshops from those who may have been practicing what they had learned, we divided the participants into two groups based on responses to item 1 of the social validity survey, “I have been consistently practicing the techniques and principles that I learned in the workshop.” Scores of 1 to 3 (ranging from “disagree” to “completely disagree”) on this item were taken to reflect disagreement, and scores of 4 or more were taken to reflect agreement.

Six participants indicated disagreement, four in the ACT + ABA group and two in the ABA group. Due to the small size of this group, nonparametric tests (Mann–Whitney U) were used to compare them to the other participants. Participants who reported that they had not been practicing what they had learned exhibited significantly different pretest scores on the GHQ-12, U = 135.5, p = .019, r = 0.40, as well as on the emotional exhaustion, U = 130.5, p = .035, r = 0.36, and depersonalization, U = 134.5, p = .022, r = 0.39, subscales of the MBI when compared to participants who reported practicing what they learned. In particular, the non-practicers displayed lower GHQ scores (Mdn = 8.00) than practicers (Mdn = 10.50), lower emotional exhaustion (Mdn = 9.00) than practicers (Mdn = 17.50), and lower depersonalization scores (Mdn = 2.00) than practicers (Mdn = 5.00). The two groups did not differ significantly on any of the demographic variables.

The 28 participants who reported practicing the workshop content were retained for this exploratory analysis. The 28 participants were evenly distributed between the two groups. Means and standard deviations on the outcome and process measures for these participants are displayed in Table 4. A repeated-measures ANOVA on GHQ-12 scores yielded a significant group × time interaction, F(2, 52) = 3.279, p = .046, partial η 2 = .112. A between-group t test (with equal variance not assumed) on change scores from pretest to posttest indicated that the ACT + ABA group exhibited a significantly greater reduction in GHQ-12 scores (M = 3.64, SD = 6.49) than did the ABA (M = −.786, SD = 3.17) group, t(18.86) = 2.295, p = .033, d = 0.87. However, this difference was not maintained at follow-up, t(22.20) = .885, p = .386. A repeated-measures ANOVA on BBS scores did not yield a significant group × time interaction, F(2, 52) = .331, p = 0.720. No significant group × time interactions were observed for the subscales of the MBI: emotional exhaustion, F(2, 52) = .334, p = .710, depersonalization, F(2, 52) = 2.29, p = .111, or personal accomplishment, F(2, 52) = 1.317, p = .277.

Table 4 Means (standard deviations) for outcome and process measures for participants who reported practicing workshop content
Full size table

The previous analyses included many individuals who did not exhibit clinically significant levels of psychological distress at pretest. In order to examine the effect of the interventions for individuals who exhibited higher levels of distress, we analyzed outcomes for participants who reported a GHQ-12 score greater than or equal to 11 at pretest, which is the established cutoff score for psychiatric distress (Goldberg et al. 1997). Five participants in the ACT + ABA condition and nine participants in the control condition scored above this cut point. Due to the small number of participants reporting clinically significant distress, nonparametric tests were used. Mann–Whitney U tests revealed no significant between-group differences at pretest in age, length of employment, or for any of the outcome or process variables for these participants. Chi-square analyses also indicated that participants did not differ in ethnicity, gender, or level of education. Table 5 lists medians and ranges on the outcome and process measures for the 14 participants scoring in the clinical range on the GHQ-12.

Table 5 Medians (ranges) for outcome and process measures for participants with GHQ-12 ≥ 11 at pretest
Full size table

For participants scoring at or above the cutoff at pretest, between-group outcomes were analyzed by performing Mann–Whitney U tests on the change scores from pretest to posttest and from pretest to follow-up. Results for the GHQ-12 suggested that the ACT + ABA group (Mdn = 4) exhibited a significantly greater decrease in distress from pretest to posttest than did the ABA group (Mdn = 0) U = 4.00, p = .012, r = 0.67. These results were maintained at follow-up, with the ACT + ABA group (Mdn = 9) showing decreased GHQ-12 scores from pretest to follow-up in comparison to the ABA group (Mdn = 2), U = 3.00, p = .009, r = 0.70. These results were accompanied by a concurrent decrease in BBS scores from pretest to follow-up in the ACT + ABA group (Mdn = 12) relative to the ABA group (Mdn = −2), U = 3.00, p = .009, r = 0.69. Four of the five participants (80 %) in the ACT group no longer met the GHQ criterion for significant distress at follow-up, compared with four of the nine participants (44 %) in the ABA group. No significant between-group differences in pretest to posttest change scores were observed for the subscales of the MBI: emotional exhaustion, U = 12.00, p = .161; depersonalization, U = 24.50, p = .786; or personal accomplishment, U = 26.00, p = .637. A similar pattern of non-significant results was observed for MBI subscale change scores from pretest to follow-up: emotional exhaustion, U = 8.50, p = .062; depersonalization, U = 33.50, p = .139; or personal accomplishment, U = 36.00, p = .070.

Discussion

We hypothesized that participants in the ACT + ABA condition would exhibit decreased general distress and burnout over time as compared to participants in the ABA condition. These hypotheses were not supported when all participant data were examined. However, exploratory analyses revealed that participants who reported that they had been practicing the ACT techniques did exhibit decreased general distress relative to controls at posttest, but not at follow-up. Additionally, the social validity data were favorable toward the ACT + ABA condition.

A different pattern of results emerged when only participants reporting significant psychological distress on the GHQ-12 at pretest were included in the analysis. Participants in the ACT + ABA group who reported significant psychological distress at the outset of the study exhibited significantly larger reductions in distress than their counterparts who received only ABA training, and participants in the ACT + ABA group reported a significant decrease in the believability of thoughts that are indicative of burnout when compared to the ABA group. The two groups did not differ in the reported frequency of thoughts and feelings that are indicative of burnout. These results are consistent with the ACT model, which emphasizes a reduction in the functional impact of thoughts rather than altering their form or frequency.

The present study has several limitations which merit discussion. To begin, the internal validity of the study is compromised by the small number of participants. While large effect sizes were observed for decreases in general distress in those participants who reported that they had been practicing the ACT techniques and in those who had GHQ-12 scores in the clinical range, only tentative conclusions can be made given the small number of participants. One way to increase the number of participants would be to offer briefer, more frequent training sessions. The current interventions were modeled after previous studies which employed a 3-h group format (e.g., Bond and Hayes 2002); however, there are no data as yet to suggest that this is the only group format that would provide favorable results. Briefer, more frequent sessions might be particularly appealing to direct care staff, who are often limited in the amount of time they can spend away from clients.

Second, the ACT + ABA intervention appears to have been most useful for individuals who exhibited higher levels of stress, which suggests the intervention may not need to be applied universally. Perhaps, workers who are experiencing higher levels of work stress could be offered the opportunity to participate in ACT-based stress management based on referral from their supervisors or from employee assistance programs. Another way to improve the utility of the interventions might be to spend more time focusing on personal values clarification and committed action tied to those values. Doing so might increase the personal relevance of the intervention for all participants, especially if the values-based exercises were expanded to include areas that may not be directly related to work.

Third, because the ACT condition contained elements of applied behavior analysis training, the current design does not allow for firm conclusions about whether changes in stress levels were produced solely by ACT or by the combination of ACT and ABA training. More importantly, it is unclear whether participants in either condition were likely to apply the ABA component of their training in a way that improved their interactions with difficult clients to foster more positive outcomes.

These concerns could be addressed via a crossover design similar to that employed by Singh et al. (2006) in their study of mindfulness training. For instance, staff in relatively independent units, such as group homes, could be administered either a program of ACT training or ABA, with both staff and client outcomes (such as use of restraint or mastery of adaptive skills) monitored for a set period of time after the intervention. The treatments could be reversed such that staff who had received behavior analysis training would then receive ACT training and vice versa. Such a design would allow for examination of treatment effects for each intervention both separately and in combination.

Fourth, another possible limitation is the lack of a control group consisting of another stress management intervention. For example, future studies could be improved by the addition of a control group consisting of cognitive behavior therapy, basic psycho-education and social support, or another such intervention that is not explicitly mindfulness based. This may allow for a more precise examination of change processes, as well as outcomes in reference to more established forms of stress management.

A fifth and related limitation is that the present study did not include measures of important ACT-related processes other than defusion. Future studies should particularly include measures of psychological flexibility and values-related processes. The Acceptance and Action Questionnaire-II (Bond et al. 2011) is a measure of psychological flexibility that has recently received some support for use with intellectual disability staff (Noone and Hastings 2011) and which may be particularly useful. Also, Noone and Hastings (2011) have recently developed a promising measure of values processes in intellectual disability staff which may also prove useful in measuring change processes.

The present study has several promising aspects. It expands upon previous work by Noone and Hastings (2009, 2010) by comparing an ACT stress management intervention with a control group consisting of a commonly employed staff training modality. In complement to Noone and Hastings' (2009, 2010) findings, the current study suggests that an ACT-based group intervention may decrease stress in staff who are experiencing significant work-related difficulties. Supplementary to Noone and Hastings' (2009) findings, social validity data from the present study indicate that the ACT workshop was well received by participants. Further, it is one of only a few studies to examine process variables in stress management interventions for mental health staff. As such, it adds to the growing body of literature supporting mindfulness and acceptance-based technologies and the theoretical models on which they are based.