Skip to main content
SpringerLink
Log in

Preventing Substance Use in High Risk Youth: Evaluation Challenges and Solutions

  • Published:
Journal of Primary Prevention Aims and scope Submit manuscript

Abstract

Programs to prevent substance use among high risk youth can pose numerous challenges for program evaluators including FEASIBILITY ISSUES such as Participant Recruitment and Retention, Identifying High Risk Youth, and Obtaining a Control/Comparison Group; MEASUREMENT ISSUES such as Social Desirability Bias and Instrument Reliability; METHODOLOGIC ISSUES such as Attrition (both selective and differential), Inadequate Implementation and Variable Dose, Low Statistical Power, Contamination of Comparison Groups, and Low Literacy Skills of Participants. For each of these challenges specific solutions are offered for researchers and practitioners. In general the solutions relate to three themes; 1) incorporate evaluation early into program design; 2) develop a strong partnership with program staff; and 3) be flexible. Also addressed are the need to acknowledge possible bias toward documenting positive outcomes as well as the need to customize evaluation designs for different settings and populations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aguinis, H., Pierce, C. A., & Quigley, B. M. (1993). Conditions under which a bogus pipeline procedure enhances the validity of self-reported cigarette smoking: A meta-analytic review. Journal of Applied Social Psychology, 23 (5), 352-373.

    Google Scholar 

  • Airhihenbuwa, C. (1992). Health promotion and disease prevention strategies for African Americans: A conceptual model. In R. Braithwaite & S. Taylor (Eds.), Health issues in the Black community (pp. 267-277). San Francisco, CA: Jossey-Bass Publishers.

    Google Scholar 

  • Airhihenbuwa, C. O., DiClemente, R. J., Wingood, G. M., & Lowe, A. (1992). HIV/AIDS education and prevention among African-Americans: a focus on culture. AIDS Education & Prevention, 4 (3), 267-76.

    Google Scholar 

  • Bachman, J. G., Johnston, L. D., & O'Malley, P. M. (1996). Monitoring the Future: A continuing study of the lifestyles and values of youth, 1994 (Computer file distributed through the Inter-university Consortium for Political and Social Research ). Ann Arbor, MI: University of Michigan, Survey Research Center.

    Google Scholar 

  • Baranowski, T., Lin, L., Wetter, D.W., Resnicow, K., & Davis, M. (1997). Theory as mediating variables: Why aren't community interventions working as desired? Annals of Epidemiology, 7 (S7), 89-95.

    Google Scholar 

  • Baron, R. M., & Kenny, D. A. (1986). The Moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51 (6).

  • Basch, C. E., Sliepcevich, E. M., Gold, R. S., Duncan, D., & Kolbe, L. (1985). Avoiding type III errors in health education program evaluations: A case study. Health Education Quarterly, 12, 315-31.

    Google Scholar 

  • Basen-Engquist, K., Tompkins, N., Lovato, C., Lewis, M., Parcel, G., & Gingiss, P. (1994). The effect of two types of teacher training on implementation of Smart Choices: A tobacco prevention curriculum. Journal of School Health, 64, 334-339.

    Google Scholar 

  • Bentler, P.M. (1987). Drug use and personality in adolescence and young adulthood: Structural models with nonnormal variables. Child Development, 58 (1), 65-79.

    Google Scholar 

  • Brick, J. M., & Kalton, G. (1996). Handling missing data in survey research. Statistical Methods in Medical Research, 5 (3), 215-38.

    Google Scholar 

  • Brown, M. (1992). A test for the difference between two treatments in a continuous measure of outcome when there are dropouts. Controlled Clinical Trials, 13 (3), 213-225.

    Google Scholar 

  • Cook, T., & Campbell, D. (1979). Quasi-experimentation: design & analysis issues for field settings. Boston: Houghton Mifflin.

    Google Scholar 

  • Dignan, M., & Carr, P. (1992). Program Planning for Health Education and Promotion. (2nd ed.). Philadelphia: Lea and Febiger.

    Google Scholar 

  • DiIorio, C., Soet, J., Van Marter, D., Woodring, T., & Dudley, W. N. (2000). An evaluation of a self-generated identification code. Research in Nursing & Health, 23, 167-174.

    Google Scholar 

  • Donner, A. (1987). Statistical methodology for paired cluster designs. American Journal of Epidemiology, 126, 972-979.

    Google Scholar 

  • Donner, A., & Donald, A. (1987). Analysis of data arising from a stratified design with the cluster as unit of randomization. Statistics in Medicine, 41, 43-52.

    Google Scholar 

  • Edmundson, E., Luton, S., McGraw, S., Kelder, S., Layman, A., Smyth, M., Bachman, K., Perdersen, S., & Stone, E. (1994). CATCH: Classroom process evaluation in a multicenter trial. Health Education Quarterly, Supplement 2, S27-S50.

    Google Scholar 

  • el-Sadr, W., & Capps, L. (1992). The challenge of minority recruitment in clinical trials for AIDS. Jama, 267 (7), 954-7.

    Google Scholar 

  • Feldman, H. A., & McKinlay, S. M. (1994). Cohort versus cross-section design in large field trials: Precision, sample size, and a unifying model. Statistics in Medicine, 13, 61-78.

    Google Scholar 

  • Finkelstein, M. O., Levin, B., & Robbins, H. (1996). Clinical and prophylactic trials with assured new treatment for those at greater risk: II. Examples. American Journal of Public Health, 86 (5), 696-705.

    Google Scholar 

  • Fitzpatrick, J. L., & Morris, M. (1999). Current and emerging ethical challenges in evaluation. San Francisco, CA, USA: Jossey Bass Inc, Publishers, 103.

    Google Scholar 

  • General Accounting Office. (1993). Drug Use Measurement. Strengths, Limitations, and Recommendations for Improvement. (BAO/T-PEMD-94-4.) Washington, D.C.

  • Graham, J.W., Hofer, S. M., Donaldson, S. I., MacKinnon, D. P., & Schafer, J. L. (1997). Anaysis with missing data in prevention research. In K. J. a. W. Bryant, Michael (Eds) (Ed.), The science of prevention: Methodological advances from alcohol and substance abuse research (pp. 325-366): American Psychological Association.

  • Graham, J. W., Hofer, S. M., & MacKinnon, D. P. (1996). Maximizing the usefulness of data obtained with planned missing value patterns: An application of maximum likelihood procedures. Multivariate Behavioral Research, 31 (2), 197-218.

    Google Scholar 

  • Graham, J. W., Hofer, S. M., & Piccinin, A. M. (1994). Analysis with missing data in drug prevention research. NIDA Research Monograph, 142, 13-63.

    Google Scholar 

  • Harris, Y., Gorelick, P., Samuels, P., & Bempong, I. (1996). Why African Americans May Not Be Participating in Clinical Trials. Journal of the National Medical Association, 88, 630-634.

    Google Scholar 

  • Jemmott, L., & Jemmott, J. (1992). Increasing condom-use intentions among sexually active Black adolescent women. Nursing Research, 41, 73-279.

    Google Scholar 

  • Johnston, L., O'Malley, P., & Bachman, J. (1995). National survey results on drug use from the monitoring the future study, 1975–1994 (Volume 1). Rockville: US Department of Health and Human Services.

    Google Scholar 

  • Kann, L., Warren, C.W., Harris, W. A., Collins, J. L., Williams, B. I., Ross, J. G., & Kolbe, L. J. (1996). Youth Risk Behavior Surveillance—United States, 1995. MMWR CDC Surveillance Summaries, 45 (4), 1-84.

    Google Scholar 

  • Klonoff, E., & Landrine, H. (1997). Distrust of whites, acculturation, and AIDS knowledge among African Americans. Journal of Black Psychology, 23 (1), 50-57.

    Google Scholar 

  • MacKinnon, D. P., & Dwyer, J.H. (1993). Estimating mediated effects in prevention studies. Evaluation Review, 17 (2), 144-158.

    Google Scholar 

  • McCaul, K., & Glasgow, R. (1985). Preventing adolescent smoking: What have we learned about treatment construct validity? Health Psychology, 4, 361-387.

    Google Scholar 

  • McCormick, L. K., Steckler, A. B., & McLeroy, K. R. (1995). Diffusion of Innovations in Schools: A Study of Adoption and Implementation of School-based Tobacco Prevention Curricula. American Journal of Health Promotion, 9 (3), 210-219.

    Google Scholar 

  • McGraw, S., Stone, E., Osganian, S., Elder, J., Perry, C., Johnson, C., Parcel, G., Webber, L., & Luepker, R. (1994). Design of process evaluation within the Child and Adolescent Trial for Cardiovascular Health (CATCH). Health Education Quarterly, Supplement 2, S5-S26.

    Google Scholar 

  • Murray, D. (1998). Design and Analysis of Group Randomized Trials. (Vol. 27). New York: Oxford University Press.

    Google Scholar 

  • Murray, D., & Wolfinger, R. (1994). Analysis issues in the evaluation of community trials: Progress toward solutions in SAS/STAT MIXED. Journal of Community Psychology, 22, 140-145.

    Google Scholar 

  • Murray, D. M., & Perry, C. L. (1987). The measurement of substance use among adolescents: When is the “bogus pipeline” method needed? Addictive Behaviors, 12 (3), 225-233.

    Google Scholar 

  • Nelson, G. D., Poehler, D., & Johnson, L. (1988). Implementation of the teenage health teaching modules: a case study. Health Education, 19 (3), 14-8.

    Google Scholar 

  • Ness, R. B., Nelson, D. B., Kumanyika, S. K., & Grisso, J. A. (1997). Evaluating minority recruitment into clinical studies: how good are the data? Annals of Epidemiology, 7 (7), 472-8.

    Google Scholar 

  • Newcomb, M., & Bentler, P. (1989). Substance use and abuse among children and teenagers. American Psychologist, 4 (2), 242-248.

    Google Scholar 

  • Newman, D. L., & Brown, R. D. (1996). Applied ethics for program evaluation. Thousand Oaks, CA, USA: Sage Publications, Inc, 228.

    Google Scholar 

  • Resnicow, K., & Botvin, G. (1993). On the effects of school health education programs: Why do they decay? Preventive Medicine, 22 (4), 484-90.

    Google Scholar 

  • Resnicow, K., Cohn, L., Reinhardt, J., Cross, D., Futterman, R., Kirschner, E., Wynder, E., & Allegrante, J. (1992). A three-year evaluation of the Know Your Body program in minority schoolchildren. Health Education Quarterly, 19, 463-480.

    Google Scholar 

  • Resnicow, K., Davis, M., Smith, M., Lazarus, A., Baranowski, T., Baranowski, J., Doyle, C., & Wang, D. T. (1998). How Best to Measure Implementation of School Health Curricula: A comparison of three measures. Health Education Research, 13 (2), 239-250.

    Google Scholar 

  • Resnicow, K., & Kirby, J. (1997). Finding Common Ground: On the marriage of evaluation and demonstration programs. Journal of Early Adolescence, 17, 8-18.

    Google Scholar 

  • Rohrbach, L., Graham, J., & Hansen, W. (1993). Diffusion of a school-based substance abuse prevention program: Predictors of program implementation. Preventive Medicine, 22 (2), 237-60.

    Google Scholar 

  • Schafer, J. L., & Olsen, M. K. (1998). Multiple imputation for multivariate missing-data problems: A data analyst's perspective. Multivariate Behavioral Research, 33 (4), 545-571.

    Google Scholar 

  • Short, L. M., & Hennessy, M. (1994). Using structural equations to estimate effects of behavioral interventions. Structural Equation Modeling, 1 (1), 68-81.

    Google Scholar 

  • Smith, D. W., McCormick, L. K., Steckler, A. B., & McLeroy, K. R. (1993). Teachers' use of health curricula: implementation of growing healthy, project SMART, and the teenage health teaching modules. Journal of School Health, 63 (8), 349-54.

    Google Scholar 

  • Sussman, S., Dent, C., Flay, B., Hansen, W., & Johnson, C. (1987). Psychosocial predictors of cigarette smoking onset by White, Black, Hispanic, and Asian adolescents in southern California. Morbidity and Mortality Weekly Report, 36 (4s), 11s-16s.

    Google Scholar 

  • Thomas, C. R., Jr., Pinto, H. A., Roach, M., 3rd, & Vaughn, C. B. (1994). Participation in clinical trials: is it state-of-the-art treatment for African Americans and other people of color? [editorial]. Journal of the National Medical Association, 86 (3), 177-82.

    Google Scholar 

  • Thomas, S., & Quinn, S. (1991). The Tuskegee Syphilis Study 1932–1972: Implications for HIV education and AIDS risk reduction programs in the Black community. American Journal of Public Health, 81, 1498-1504.

    Google Scholar 

  • Turner, C. F., Ku, L., Rogers, S. M., Lindberg, L. D., Pleck, J. H., & Sonenstein, F. L. (1998). Adolescent sexual behavior, drug use, and violence: increased reporting with computer survey technology. Science, 280 (5365), 867-73.

    Google Scholar 

  • USDHHS/SAMHSA. (1993). National Household Survey on Drug Abuse: Population Estimates, 1992. (Vol. SMA 93-2053).

  • Vaughan, R., McCarthy, J., Walter, H., Resnicow, K., Waterman, P., Armstrong, B., & Tiezzi, L. (1996). Development, Reliability, and Validity of a Risk Factor Screening Survey for Urban Minority Junior High School Students. Journal of Adolescent Health., 17, 1-13.

    Google Scholar 

  • Walter, H. J., & Vaughan, R. D. (1993). AIDS risk reduction among a multiethnic sample of urban high school students. JAMA, 270 (6), 725-30.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Rolllins School of Public Health, Emory University, 1518 Clifton Road, Atlanta, Georgia, 30322

    Ken Resnicow

  2. Rollins School of Public Health, Emory University, Atlanta, Georgia

    Ronald Braithwaite & Colleen Dilorio

  3. Center for Population and Family Health, Columbia University, New York

    Roger Vaughan

  4. Development Services Group, Inc., Bethesda, Maryland

    Marcia I. Cohen

  5. Program Evaluation Research Branch, Centers for Disease Control and Prevention, Atlanta, Georgia

    Gary A. Uhl

Authors
  1. Ken Resnicow
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ronald Braithwaite
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Colleen Dilorio
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Roger Vaughan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marcia I. Cohen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Gary A. Uhl
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Resnicow, K., Braithwaite, R., Dilorio, C. et al. Preventing Substance Use in High Risk Youth: Evaluation Challenges and Solutions. The Journal of Primary Prevention 21, 399–415 (2001). https://doi.org/10.1023/A:1007029910822

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1007029910822

Search

Navigation