The Impact of Respondent Type on Treatment Facility Performance Evaluations – Do Alumni and their Friends and Family Prioritize different Offerings?

C l i n M e d International Library Citation: Moler AK, Carlin A, Sanghani RM (2015) The Impact of Respondent Type on Treatment Facility Performance Evaluations – Do Alumni and their Friends and Family Prioritize different Offerings?. Int Arch Addict Res Med 1:008 Received: July 23, 2015: Accepted: August 14, 2015: Published: August 17, 2015 Copyright: © 2015 Moler AK. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Moler et al. Int Arch Addict Res Med 2015, 1:2

struggling with addiction, finding the appropriate facility that suits a client's need can be an arduous task; potential clients must navigate dozens of facility types (e.g. long-term residential, outpatient, etc.) and innumerable treatment strategies [3]. These different aspects of treatment vary in their relative importance depending on an individual's personal preferences or needs. Additionally, many individuals seeking treatment rely on their friends and/or family for emotional, logistical, and/or financial support; these loved ones may have their own preferences and opinions on treatment and treatment facilities.
Research highlights the importance of an individual's selfselection of treatment intervention in regards to the efficacy of that treatment. A component to self-selection is an individual's readiness for treatment, which may be affected by past treatment or non-treatment influences [4]. In addition to the importance of the individual's needs, opinions and readiness on the effectiveness of treatment is the familial involvement. Lander, Howsare and Bryn found that the family members' perceptions on substance use disorders (SUDs) influence individuals as they attempt to seek recovery and will in turn affect the outcome of treatment. Researchers in the addiction space have confirmed the reciprocal relationship between an individual's SUD and their environment [5]. Those struggling with an SUD may not be able to effectively obtain treatment without considering the impact on the whole family and their environment in addition to their own needs and desires in the self-selection process [4,5].
Due to the importance of both the individual and those in his surrounding environment, the presence of differences in preferences between the client himself and his friends and/or family could have profound impacts on treatment outcomes [6]. Successful treatment can partially rely on an individual's involvement with his treatment process, satisfaction with the treatment he received, and family involvement [7][8][9]. Therefore, if the client and those supporting him have differing expectations of treatment, intentions for treatment, or misunderstandings of the client's priorities [10], treatment outcomes could be negatively impacted.
This project explores potential differences in how alumni, or alumni's friends and/or family evaluate a facility's offerings. Specifically, this project investigates whether respondent type is a predictive factor of which aspects of a facility's offerings will be prioritized in performance evaluations. The authors hypothesize that respondent type is a predictive factor. To test this hypothesis, a mixed method approach was employed, utilizing quantitative analyses of open-ended string responses and quantitative analysis of response categories.

Survey design
A survey was created using SurveyMonkey's online survey design platform [11]. The survey targets either alumni or friends and/or family members (of alumni) of inpatient drug or alcohol rehabilitation facilities; the survey contains a qualifying question asking "Within the past 10 years, have you or a close loved one received treatment from a rehab facility for substance/alcohol addiction, behavioral problems, or similar disorders." Response options are, "Yes, I received treatment," "Yes, my loved one received treatment," and "No, neither I nor my loved ones have received treatment." Those who responded "No" were disqualified from the survey. The survey contains fifteen questions pertaining to facility evaluation, structured in a 5-point Likert matrix, as well as two openended questions related to facility performance. The two open-ended questions ask respondents to identify the strengths and weaknesses, respectively, of their chosen facility. Specifically, the two open-ended questions ask, "Please list the facility's strengths," and "Please list the facility's weaknesses." These two open-ended questions provide all of the data used in the analyses below.

Sample selection
Data collection occurred from May 1 st , 2015 to June 15 th , 2015; 3,500 completed responses were received. Respondents were recruited via Survey Monkey Audience, a feature through which survey creators are provided with targeted, representative sample populations [11]. Respondents opt in to participate in relevant surveys, based on their detailed demographic profile. Respondents are indirectly compensated; for each completed survey, SurveyMonkey makes a charitable micro-donation.
From these responses (n=3500), 500 reviews were randomly selected from each respondent category to comprise a sample size of 1,000 reviews. Random selection was completed with STATA's sample command.

Analytical approach
This project utilizes a mixed-methods, three step analytical process. First, the randomly selected 1,000 cases and the selected respective three variables were extracted from the survey's full results and loaded on to a spreadsheet. These variables are strengths -full string (S.FS), weaknesses -full string (W.FS), and respondent type (RT). These data were first loaded into KH Coder, software designed to quantitative analyze string-formatted data [12]. Data were preprocessed utilizing the Stanford POS Tagger. Following pre-processing, the first approach, correspondence analyses, was conducted on the strengths and weaknesses variables. Each correspondence analysis was conducted using the following parameters: Term Frequency > 3, POS=Noun and Proper Noun, Filter Chi Value top 35, Bubble Plots.
The second stage of the analysis consisted of constructing numerical categorical variables to classify the primary content focus of each response to the two open-ended survey questions. These two variables, strengths -primary content category (S.PCC) and weaknessesprimary content category (W.PCC) were constructed to provide clarity to results reporting by condensing results and to facilitate quantitative analysis. Thirteen primary content categories were created. The content categories are as follows: Activities, Amenities & Offerings, Counseling (Both), Family Involvement, Group Counseling, Individual Counseling, Location, Multiple (if respondent noted 'all'), Peers, Policies, Price, Staff, and Treatment Type. Each response was assigned a Primary Content Category for both strengths and weaknesses. Each category was assigned by identifying the primary content focus of the review. In the case of multiple foci, the first response was used for category identification. After these two variables were integrated into the dataset, pre-processing was rerun utilizing the same approach. Following preprocessing, correspondence analyses were conducted on the new variables. Each correspondence analysis was conducted using the following parameters: Term Frequency >1, POS = Noun and Proper Noun, No Filter, Bubble Plots.
The final stage consisted of a quantitative analysis utilizing STATA/ IC 13.1 [13]. This approach utilized multinomial logistic regression, due to the dependent variable(s) containing more than two discreet outcomes. Standard diagnostics were conducted to ensure appropriateness of the model. Each model with the predictor variable was tested against the naïve model and found to be significant. The models were constructed with the primary content category variable as the dependent variable, with 'staff' serving as the base, and respondent type serving as the predictor variable. .log files are available. Adjusted probability estimations for each outcome were then calculated.
All data were de-identified. Respondents explicitly consented to the use of their de-identified responses in public reporting and research.

Response distributions
The frequency distributions for each possible outcome for S.PCC and W.PCC are located in table 1. For alumni, and for friends and/ or family, aspects related to staff were the most frequently cited as the facility's greatest strength, at 44.34% of alumni (n=192) and 36.94% of friends and/or family (n=164), respectively, followed by treatment type at 15.94% (n=69) and 20.50% (n=91). Conversely, activity offerings were least frequently most infrequently considered a priority strength, at .069% (n=3) and 0.90% (n=4), respectively.
For both alumni and friends/and or family, aspects related to a facility's policies were the most frequently cited as the facility's greatest weakness, at 27.49% (n=102) and 28.29% (n=99), respectively, followed by aspects related to amenities and offerings, at 16.17% (n=60) and 18.00% (n=63). Conversely, peers were the least cited, at 0.00% (n=0) and 0.14% (n=1), respectively.

Strengths
The output ( Figure 1) of a correspondence analysis conducted on S.FS by RT demonstrates that distinct clusters exist for each variable; that is, being an alumni or a family and/or friend corresponds with unique response patterns in the Strengths reviews. Particularly, the aspects of treatment related to 'meal', 'food', 'class', 'campus', and treatment ([12]-step and AA) are distinct to alumni responses, where aspects related to 'cost', 'family', 'communication', 'interaction', and 'well-being', among others, are distinct to family and/or friend responses.
The output (Figure 2) from the correspondence analysis of S.PCC, by RT demonstrates that distinct clusters exist for each respondent type. This model suggests that aspects of treatment related to the categories 'peer' and 'group counseling' are distinct to alumni respondents, where ' family involvement', 'location, and 'price' are distinct to family and/or friend respondents. This model also suggests that certain categories are not distinct to the respondent type; these are 'individual counseling', 'staff', 'amenities & offerings', 'treatment type', and 'policy'. The categories not distinct to respondent type are plotted quite closely, suggesting that differences in distinctness may not be substantially significant, requiring further analysis.
The output from the S.PCC regression model is found in table 2. This model suggests that alumni are likely to prioritize the following treatment aspect categories in their performance evaluations: 'group counseling', 'counseling (both)', and 'peers'. Conversely, family and/ or friends are likely to prioritize 'location', 'policies', 'treatment type', 'price', and 'family involvement'. Additionally, no substantially significant difference exists between 'amenities & offerings', and 'staff'.

Weaknesses
As with Strengths, a correspondence analysis of the entire sample was also used as the first analytical approach to W. FS. These models used the same pre-processing and filter settings as the Strengths model. The output ( Figure 3) from this model demonstrated that distinct clusters exist for each variable; respondent type corresponds with a unique response pattern. Specifically, the aspects of treatment related to 'food', 'bathroom', 'exercise', 'activity', 'complaint', treatment, and 'room' are distinct to alumni responses, were aspects related to 'funding', 'distance', 'family', and 'insurance', among others, are distinct to family and/or friend responses. This output indicates that some commonalities exist; 'facility', 'people', and 'treatment' are not substantially distinct.
A correspondence analysis on the constructed content category variable (W.PCC) was also conducted for Weaknesses, using the same filtering as the corresponding model from Strengths. The output (Figure 4) demonstrated that distinct clusters exist for each respondent type. This model suggests that aspects of treatment related to the categories 'peer', 'group counseling', and to a lesser extent, 'staff' and 'individual counseling' are distinct to alumni respondents, where 'price and 'family involvement' are unique to friend and/or family respondents. This output also suggests that numerous categories are not distinct to either respondent; these categories are 'location', 'policy', 'amenities & offerings', 'counseling (both)' and 'treatment type'.
As with Strengths, the final stage of the Weaknesses analysis consisted of running multinomial logistic regression with post-model estimation on W.PCC. The construction and running of this model  1=Alumni / 2=Friends and/or Family followed the same procedure as described in Strengths. The output from this model is found in table 3. This model suggests that alumni are more likely to prioritize the following treatment aspect categories, compared to friends and/or family, when evaluating a facility's (negative) performance: 'individual counseling' and 'staff'. Friends and/or family were more likely to prioritize 'family involvement'. No substantial difference existed between respondent types for 'amenities & offerings' and 'policies'. The remaining categories were not statistically significant.

Discussion
Correspondence analyses and regression modeling on 1,000 surveys taken by addiction treatment alumni and loved ones of alumni strongly suggest that substantially and statistically significant differences exist across respondent types when evaluating an addiction treatment facility's performance. That is, when an alumnus of a treatment facility, or the (closely involved) friends and/or family members of an alumnus of a treatment facility evaluate their respective facility's performance, they prioritize different aspects of the facility's offerings. It follows, then, that as a result of different priorities respondent type will affect expectations for facilities and treatment, as well as measures of success. The potential implications of this relationship are discussed below.
Previous research has thoroughly investigated the importance of loved ones' participation and support in the effectiveness of treatment for SUDs [4,[7][8][9]. In addition, investigations on individuals' engagement and attitudes toward treatment, and the subsequent effects on treatment, have been performed [4]. Little is known, however, on how preferences of both the individual and his or her loved ones on the treatment facility compare and in turn  Flat or discontinuous region encountered Implementing well-rounded treatment that is conscious of and addresses the differences in priorities for treatment offerings could have profound impacts on treatment outcomes. First, acknowledging and addressing ranging priorities can result in increasing satisfaction with treatment; increasing satisfaction with services received can positively impact treatment outcome [15]. Addressing concerns (i.e. priorities in offerings) can also increase treatment plan buy-in, which could result in potentially increasing probability of treatment success [16,17]. Integrating friends and/or family into treatment, whether directly or indirectly, can benefit the individual receiving treatment [6]. To best integrate friends and/or family, facilities must ensure that their offerings are meeting these individuals' priorities.
In addition to the implications for well-rounded treatment implementation within facilities, these findings can also direct treatment facilities on how to aid the decision-making process for individuals and loved choosing treatment. Research on decisionmaking within the healthcare industry has shown that decision aids, such as pamphlets, videos or online tools can be used to help those deciding between options where there is no one clear advantageous choice. Decision aids have been shown to make the process of assessing options more effective by explicitly illustrating available options, and the benefits and harms of those options [18]. By taking into consideration the differing preferences of individuals and their loved ones when seeking SUD treatment, facilities may support the effect treatment. Therefore, this present study expands on existing literature to provide more knowledge on how respondent type effects the prioritization of treatment components and how that can guide treatment offerings, effectiveness and future research.
The two respondent types do not simply prioritize different treatment areas; they prioritize different types of treatment areas. For both Strengths and Weaknesses, alumni prioritize interpersonal aspects of treatment, such as their relationships with the staff, counselors and their fellow peers. Conversely, friends and/or family prioritize more logistical aspects of treatment -policies and prices, the location, family involvement, and treatment offerings. The alumni are concerned with day-to-day concerns, while the friends and/or family are more concerned with the broader picture, or administrative aspects of treatment.
This difference has two implications. First, it suggests that friends and/or family are heavily involved in the treatment process, particularly in administrative endeavors. That is, loved ones who support individuals seeking treatment are likely more concerned about the administrative aspects due to their investment; therefore, they want to ensure the facility's administrative aspects are in line with their ability to contribute and their value system. Alumni, however, focus more on the interpersonal and emotional aspects of treatment, which may be due to the positive influence of support systems and personal relationships on treatment outcomes [14]. Furthermore, the difference suggests that facilities must prioritize offering well-rounded treatment options. Loved ones support a large portion of individuals undergoing treatment, in many manners. As the above results suggest, these supporters may be handling crucial aspects of treatment, such as interacting with treatment providers decision-making process by providing all encompassing decision aids that elucidate their offerings preferred by both an individual and their loved ones.
Future research could take multiple directions. Comparing the two correspondence analysis models for each evaluation category (i.e. Strengths and Weaknesses) suggests that certain categories may have sub-aspects that are distinct to respondent type, which, when categorized together, causes the category to illustrate nondistinctness. Thus, in addition to refining categories, future efforts could focus on differences within categories, rather than across categories. Recognizing that differences exist, future efforts could examine the impact of these differences on treatment outcomes and facility operations; to what extent does differing expectations and notions of success impact these areas?
This study was limited by sample selection. In order to protect identity and anonymity, random samples were selected. Due to this, it is not likely that the loved ones of queried alumni were also queried. Therefore, there is an inability to investigate specific perceptions on a singular facility during a specific instance. Further studies may want to purposively sample alumni from select facilities, as well as these specific alumni friends and/or family. Additionally, further studies could include additional questions to increase internal consistency by limiting the subjective nature of each respondent defining "strength" and "weakness".

Conclusion
Tens of millions of individuals are affected by alcohol and substance use disorders every year and of those, multiple millions seek treatment. Treatment is incredibly diverse, with nearly innumerable permutations of facility type, facility offerings, and treatment type. Navigating these options can be a difficult process, and often treatment clients seek the assistance and support of close friends and/or family members. Both of these types of individuals prioritize different treatment offerings. This study suggests that respondent type acts as a predictive factor on which treatment areas are prioritized. The presence of these differences can influence treatment outcomes. Specifically, acknowledging and addressing these differences through providing well-rounded treatment provides the opportunity to potentially positively increase treatment outcomes.