Prospective validation of substance abuse severity measures from administrative data☆
Introduction
Severity measures pertaining to substance abuse have important administrative applications. For example, severity measures can be used to set capitated payment rates for pre-paid behavioral health care (McFarland et al., 1995). In addition, because client acuity affects choice of treatment setting (see, e.g. Duffy et al., 2004) and treatment outcomes (see, e.g., Alemi et al., 1995), severity measures are important components of risk adjustment strategies needed to compare client outcomes among providers (Shwartz et al., 1997). This latter consideration has become especially significant as the US federal government implements its “Performance Partnership Program” for financing public sector substance abuse treatment services (Gallant, 2003, SAMHSA, 2003a, SAMHSA, 2003b, SAMHSA, 2005). If providers are to be compared with one another based on performance, then it is essential that the comparisons account for variation in client acuity (Shwartz et al., 1997). Severity measures can also be used to examine claims that providers may selectively “cream skim” clients with relatively low acuity while avoiding provision of care to more impaired individuals (Deck and McFarland, 2002, Werner and Asch, 2005). These and other administrative applications generally require large sample sizes and usually make use of electronic administrative data systems (McCarty et al., 1998).
Of course, there are numerous structured interviews that generate indicators of client severity. For example, the Addiction Severity Index (ASI) (McLellan et al., 1992) and the Global Appraisal of Individual Needs (GAIN) (Dennis et al., 2004) are well known interview protocols that yield several measures pertaining to severity. On the other hand, instruments like the ASI and the GAIN require trained interviewers as well as considerable time to administer. In some sense, the American Society of Addiction Medicine (ASAM) Patient Placement Criteria (Mee-Lee et al., 2001) can be considered a severity measurement system. A computer program is available to assist clinicians in generating the ASAM measures (Turner et al., 1999). However, the ASAM data may or may not be found in administrative records. There are also instruments that focus on specific substances such as the Lifetime Severity Index for Cocaine Use Disorder (Hser et al., 1999). Here, too, the items used to create these substance-specific measures may or may not be routinely recorded in electronic data systems (McCarty et al., 1998). In addition, many self-administered questionnaires such as the Alcohol Use Disorders Identification Test (AUDIT, Allen et al., 1997) can be used to produce indicators of severity. Again, these standardized instruments are not necessarily found in typical administrative databases.
Recently, two substance abuse severity measures have been developed for use with information routinely found in large administrative databases. Caspi et al. (2001) designed measures based on administrative data that predicted concurrent Addiction Severity Index component scores for public sector substance abuse treatment clients in Massachusetts. Deck (Deck and McFarland, 2002, McFarland et al., 2005) constructed a severity measure from administrative data that could predict concurrent Addiction Severity Index total scores and American Society of Addiction Medicine Patient Placement Criteria scores. Both severity instruments are based on items generally available in administrative information systems pertaining to public sector substance abuse treatment clients. The Caspi et al. (2001) measures (henceforth called the Caspi severity measures) generate scores pertaining to alcohol, cocaine, and heroin, respectively, for adult clients. The Deck (Deck and McFarland, 2002, McFarland et al., 2005) measure (henceforth called the Deck severity measure) generates one overall severity score.
These two measures showed good concurrent (criterion) validity (Nunnally and Bernstein, 1994) when they were compared with scores generated from the Addiction Severity Index and/or the American Society of Addiction Medicine Patient Placement Criteria. However, little information is available about the predictive (prospective) validity (Nunnally and Bernstein, 1994) of these severity measures. The purpose of the present project was to examine the prospective validity of the Caspi and Deck severity measures, respectively.
Section snippets
Methods
Information was obtained from large national databases in the US pertaining to substance abuse treatment. The databases were selected because they: (a) represented several sites across the country and (b) had baseline (typically substance abuse treatment intake) and follow-up data. The data sets are described briefly here.
Results
Table 1 summarizes the demographic and clinical characteristics of the adult subjects in the prospective validation data sets. The DATOS adults were slightly younger than the SAMHSA adults. There were more males than females in both data sets. Minorities were well represented with more African Americans in the DATOS data and more Hispanics in the SAMHSA data. The DATOS data had a higher percentage of people using cocaine compared with the SAMHSA data. In both data sets, the percentages
Discussion
This project examined the prospective (predictive) validity of substance abuse severity measures derived from readily available administrative data. The Caspi severity measures were highly predictive of substance use at follow-up 1 year after the end of treatment for the DATOS adult and adolescent samples, 1 year after entering treatment for the SAMHSA adult sample, and 6 months after entering treatment for the SAMHSA adolescent sample. With the exception of predicting alcohol use for the DATOS
Acknowledgements
Supported in part by National Institute on Alcohol Abuse and Alcoholism Grant number R21 AA 014050 and by Robert Wood Johnson Substance Abuse Policy Research Program Grant number 048278.
References (41)
Self-report among injecting drug users: a review
Drug Alcohol Depend.
(1998)- et al.
Drug-use initiation and conduct disorder among adolescents in drug treatment
J. Adolesc.
(2003) - et al.
The fifth edition of the Addiction Severity Index
J. Subst. Abuse Treat.
(1992) - et al.
The development of the Comprehensive Addiction Severity Index for Adolescents (CASI-A). An interview for assessing multiple problems of adolescents
J. Subst. Abuse Treat.
(1995) - et al.
Feasibility of multidimensional substance abuse treatment matching: automating the ASAM Patient Placement Criteria
Drug Alcohol Depend.
(1999) - et al.
A review of factors affecting treatment outcomes: expected treatment outcomes scale
Am. J. Drug Alcohol Abuse
(1995) - et al.
A review of research on the alcohol use disorders identification test (AUDIT)
Alcohol. Clin. Exp. Res.
(1997) - et al.
Talk is cheap: measuring drinking outcomes in clinical trials
J. Stud. Alcohol
(2000) - et al.
Patient satisfaction, use of services, and one-year outcomes in publicly funded substance abuse treatment
Psychiatr. Serv.
(2001) Methodological issues and problems in the assessment of substance abuse
Psychol. Assess.
(1995)
The severity index: an indicator of alcohol and drug dependence using administrative data
Alcohol. Treat. Quart.
Use of substance abuse treatment services before and after Medicaid managed care
Psychiatr. Serv.
Truth or consequences: the validity of self-report data in health services research on addictions
Addiction
Cross-validation of the alcohol and cannabis use measures in the Global Appraisal of Individual Needs (GAIN) and Timeline Followback (TLFB; Form 90) among adolescents in substance abuse treatment
Addiction
The drug abuse treatment outcome study (DATOS): treatment evaluation research in the United States
Psychol. Addict. Behav.
Methodological overview and research design for the Drug Abuse Treatment Outcome Study (DATOS)
Psychol. Addict. Behav.
Drug treatment outcomes for adolescents with comorbid mental and substance use disorders
J. Nervous Mental Dis.
Cited by (11)
Association of facility characteristics and substance use disorder outcomes at discharge from residential treatment
2022, Journal of Substance Abuse TreatmentCitation Excerpt :We also controlled for socioeconomic characteristics at intake: marital status (married, partnered vs. divorced, separated, widowed, never married), education level (less than high school diploma, high school diploma/GED, more than high school), receiving income from any source, health insurance (public, private, none), housing status, and employment status. The study adjusted for SUD severity and history using the Deck Severity Index (McCamant et al., 2007; McFarland et al., 2005), age of first use, and primary/secondary substance at intake. We also controlled for co-occurring mental health disorder (i.e., having any prior mental health service), prior SUD treatment engagement (e.g., any prior SUD service, any prior residential treatment), referral source (e.g., SUD treatment provider, criminal justice system, or other), and receipt of any other state services at intake.
Incentives in a public addiction treatment system: Effects on waiting time and selection
2018, Journal of Substance Abuse TreatmentCitation Excerpt :Mental illness/disorder history was operationalized as admissions that identified a co-occurring DSM-IV mental illness diagnosis, any outpatient mental health visit in the past 12 months, or any psychiatric hospitalization the past two years. The substance use severity measure is a validated composite index that captures multiple dimensions of substance use that are readily available in administrative data, including frequency and duration of substance use, use of “hard” drugs (e.g., cocaine, heroin, methamphetamine), intravenous drug use, employability, employment status, and income (McCamant, Zani, McFarland, & Gabriel, 2007; McFarland, Deck, McCamant, Gabriel, & Bigelow, 2005). Each dimension is given equal weight in calculating the substance use severity score, which ranges between zero and one, with higher scores denoting greater severity; specific severity levels were not constructed.
Residential substance abuse treatment for Urban American Indians and Alaska Natives
2017, American Indian and Alaska Native Mental Health ResearchAdmissions to detoxification after treatment: Does engagement make a difference?
2016, Substance Abuse
- ☆
Supplementary material for this article can be viewed by accessing the online version of this paper at http://dx.doi.org.