Skip to main content
SpringerLink
Log in

Measuring Youths’ Perceptions of Counseling Impact: Description, Psychometric Evaluation, and Longitudinal Examination of the Youth Counseling Impact Scale v.2

  • Original Paper
  • Published:
Administration and Policy in Mental Health and Mental Health Services Research Aims and scope Submit manuscript

Abstract

The Youth Counseling Impact Scale (YCIS) is an empirically validated treatment progress measure that assesses youths’ perceptions of the short term effectiveness of therapy. Since its initial publication, the original 10-item measure has been shortened to ease measurement burden and revised to include a question about a youth’s insight into his or her strengths. The current study describes the development of the revised YCIS (v.2) and evaluates its psychometric properties. Additionally, this study examines whether the YCIS (v.2) total score or subscale scores change over time and investigates whether there are gender or age differences for youths’ perceptions of the impact of therapy. Results found the revised version obtained comparable information to that of the original measure, and that the revised version retained the factor structure of the original model with one primary general factor of Counseling Impact and two secondary factors (Insight and Change). Results also suggested that while the YCIS (v.2) total score and Change subscale score did not change linearly over the course of treatment, the Insight subscale score showed a small but significant linear increase over time. No significant differences in YCIS scores based on youth age or gender were found. The implication of these findings, the clinical and empirical utility of this measure, and its limitations are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Addis, M. E., & Jacobson, N. S. (1996). Reasons for depression and the process and outcome of cognitive—Behavioral psychotherapies. Journal of Consulting and Clinical Psychology, 64, 1417–1424.

    Article  PubMed  CAS  Google Scholar 

  • Alfano, C. A., Pina, A. A., Villalta, I. K., Beidel, D. C., Ammerman, R. T., & Crosby, L. E. (2009). Mediators and moderators of outcome in the behavioral treatment of childhood social phobia. Journal of the American Academy of Child and Adolescent Psychiatry, 48, 945–953.

    Article  PubMed  Google Scholar 

  • American Psychological Association, & Presidential Task Force on Evidence-Based Practice. (2006). Evidence-based practice in psychology. American Psychologist, 61, 271–285.

    Article  Google Scholar 

  • Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 561–573.

    Article  Google Scholar 

  • Athay, M. M., Riemer, M., & Bickman, L. (2012). The Symptoms and Functioning Severity Scale (SFSS): Psychometric evaluation and differences among youth, caregiver, and clinician ratings over time. Administration and Policy in Mental Health and Mental Health Services Research. doi:10.1007/s10488-012-0403-2.

  • Bickman, L., Kelley, S., Breda, C., Vides de Andrade, A. R., & Riemer, M. (2011). Effects of routine feedback to clinicians on youth mental health outcomes: A randomized cluster design. Psychiatric Services, 62, 1423–1429.

    Article  PubMed  Google Scholar 

  • Bickman, L., Riemer, M., Kelley, S., Tempesti, T., Brannan, A., Athay, M., et al. (2010). Manual of the peabody treatment progress battery. Nashville, TN: Vanderbilt University.

    Google Scholar 

  • Bickman, L., Riemer, M., Lambert, E., Kelley, S., Breda, C., Dew, S., et al. (2007). Manual of the Peabody treatment progress battery [Electronic version]. Nashville, TN: Vanderbilt University.

    Google Scholar 

  • Bockting, C. L. H., Spinhoven, P., Koeter, M. W. J., Wouters, L. F., Visser, I., & Schene, A. H. (2006). Differential predictors of response to preventive cognitive therapy in recurrent depression: A 2-year prospective study. Psychotherapy and Psychosomatics, 75, 229–236.

    Article  PubMed  Google Scholar 

  • Browne, M. W. C., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136–162). Newbury Park: Sage.

  • Bussell, J. R. (2001). Exploring the role of therapy process and outcome in interventions that target adolescent identity and intimacy. Dissertation Abstracts International: Section B: The Sciences and Engineering, 61(10-B), 5553.

    Google Scholar 

  • Clarkin, J., & Levy, K. (2004). The influence of client variables on psychotherapy. In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (5th ed., pp. 194–226). New York: Wiley.

    Google Scholar 

  • Curry, J., Rohde, P., Simons, A., Silva, S., Vitiello, B., Kratochvil, C., et al. (2006). Predictors and moderators of acute outcome in the Treatment for Adolescents with Depression Study (TADS). Journal of the American Academy of Child and Adolescent Psychiatry, 45, 1427–1439.

    Article  PubMed  Google Scholar 

  • Dunne, A., Thompson, W., & Leitch, R. (2000). Adolescent males’ experience of the counselling process. Journal of Adolescence, 23, 79–93.

    Article  PubMed  CAS  Google Scholar 

  • Durlak, J. A., Fuhrman, T., & Lampman, C. (1991). Effectiveness of cognitive-behavior therapy for maladapting children: A meta-analysis. Psychological Bulletin, 110, 204–214.

    Article  PubMed  CAS  Google Scholar 

  • Elliott, R. (1985). Helpful and nonhelpful events in brief counseling interviews: An empirical taxonomy. Journal of Counseling Psychology, 32, 307–322.

    Article  Google Scholar 

  • Elliott, R., & Wexler, M. (1994). Measuring the impact of sessions in process-experiential therapy of depression: The session impacts scale. Journal of Counseling Psychology, 41, 166–174.

    Article  Google Scholar 

  • Embretson, S. E. (1996). The new rules of measurement. Psychological Assessment, 8, 341–349.

    Article  Google Scholar 

  • Eugster, S. L., & Wampold, B. E. (1996). Systematic effects of participant role on evaluation of the psychotherapy session. Journal of Consulting and Clinical Psychology, 64, 1020–1028.

    Article  PubMed  CAS  Google Scholar 

  • Forgatch, M. S., & Patterson, G. R. (2010). Parent Management Training—Oregon Model: An intervention for antisocial behavior in children and adolescents. In J. R. Weisz & A. E. Kazdin (Eds.), Evidence-based psychotherapies for children and adolescents (2nd ed., pp. 159–178). New York: Guilford.

    Google Scholar 

  • Garcia, A. M., Sapyta, J. J., Moore, P. S., Freeman, J. B., Franklin, M. E., March, J. S., et al. (2010). Predictors and moderators of treatment outcome in the Pediatric Obsessive Compulsive Treatment Study (POTS I). Journal of the American Academy of Child and Adolescent Psychiatry, 49, 1024–1033.

    Article  PubMed  Google Scholar 

  • Garland, A. F., Hawley, K. M., Brookman-Frazee, L., & Hurlburt, M. S. (2008). Identifying common elements of evidence-based psychosocial treatments for children’’s disruptive behavior problems. Journal of the American Academy of Child and Adolescent Psychiatry, 47, 505–514.

    Article  PubMed  Google Scholar 

  • Gaynor, S. T., Weersing, V. R., Kolko, D. J., Birmaher, B., Heo, J., & Brent, D. A. (2003). The prevalence and impact of large sudden improvements during adolescent therapy for depression: A comparison across cognitive-behavioral, family, and supportive therapy. Journal of Consulting and Clinical Psychology, 71, 386–393.

    Article  PubMed  Google Scholar 

  • Handwerk, M. L., Huefner, J. C., Smith, G. L., Clopton, K., Hoff, K. E., & Lucas, C. P. (2006). Gender differences in adolescents in residential treatment. American Journal of Orthopsychiatry, 76, 312–324.

    Article  PubMed  Google Scholar 

  • Hansen, N. B., & Lambert, M. J. (2003). An evaluation of the dose–response relationship in naturalistic treatment settings using survival analysis. Mental Health Services Research, 5, 1–12.

    Article  PubMed  Google Scholar 

  • Harkness, A., & Lillienfeld, S. O. (1997). Individual differences science for treatment planning: Personality traits. Psychological Assessment, 9, 349–360.

    Article  Google Scholar 

  • Harmon, S. C., Lambert, M. J., Smart, D. M., Hawkins, E., Nielsen, S. L., Slade, K., et al. (2007). Enhancing outcome for potential treatment failures: Therapist-client feedback and clinical support tools. Psychotherapy Research, 17, 379–392.

    Article  Google Scholar 

  • Hayes, A. M., Laurenceau, J. P., Feldman, G., Strauss, J. L., & Cardaciotto, L. A. (2007). Change is not always linear: The study of nonlinear and discontinuous patterns of change in psychotherapy. Clinical Psychology Review, 27, 715–723.

    Article  PubMed  Google Scholar 

  • Henggeler, S. W. (1999). Multisystemic therapy: An overview of clinical procedures, outcomes, and policy implications. Child and Adolescent Mental Health, 4, 2–10.

    Article  Google Scholar 

  • Horvath, A. O., & Luborsky, L. (1993). The role of the therapeutic alliance in psychotherapy. Journal of Consulting and Clinical Psychology, 61(4), 561–573.

    Article  PubMed  CAS  Google Scholar 

  • Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.

    Article  Google Scholar 

  • Janzen, J., Fitzpatrick, M., & Drapeau, M. (2008). Processes involved in client-nominated relationship building incidents: Client attachment, attachment to therapist, and session impact. Psychotherapy: Theory, Research, Practice, Training, 45, 377–390.

    Article  Google Scholar 

  • Ji, P. Y. (2002). The role of attachment status in predicting longitudinal relationships between session-impact events and the working alliance within an adolescent client population. Dissertation Abstracts International: Section B: The Sciences and Engineering, 62(12-B), 5966.

    Google Scholar 

  • Kazdin, A. E. (2008). Evidence-based treatment and practice: New opportunities to bridge clinical research and practice, enhance the knowledge base, and improve patient care. American Psychologist, 63, 146–159.

    Article  PubMed  Google Scholar 

  • Kellett, S., Clarke, S., & Matthews, L. (2006). Session impact and outcome in group psychoeducative cognitive behavioural therapy. Behavioural and Cognitive Psychotherapy, 35, 335–342.

    Article  Google Scholar 

  • King, R., Bambling, M., Reid, W., & Thomas, I. (2006). Telephone and online counselling for young people: A naturalistic comparison of session outcome, session impact and therapeutic alliance. Counselling & Psychotherapy Research, 6, 175–181.

    Article  Google Scholar 

  • Kolden, G. G. (1996). Change in early sessions of dynamic therapy: Universal processes and the generic model of psychotherapy. Journal of Consulting and Clinical Psychology, 64, 489–496.

    Article  PubMed  CAS  Google Scholar 

  • Kolden, G. G., Chisholm-Stockard, S. M., Strauman, T. J., Tierney, S. C., Mullen, E. A., & Schneider, K. L. (2006). Universal session-level change processes in an early session of psychotherapy: Path models. Journal of Consulting and Clinical Psychology, 74, 327–336.

    Article  PubMed  Google Scholar 

  • Kolden, G. G., & Howard, K. I. (1992). An empirical test of the generic model of psychotherapy. Journal of Psychotherapy Practice & Research, 1, 225–236.

    Google Scholar 

  • Kolden, G. G., Strauman, T. J., Gittleman, M., Halverson, J. L., Heerey, E., & Schneider, K. L. (2000). The Therapeutic Realizations Scale—Revised (TRS-R): Psychometric characteristics and relationship to treatment process and outcome. Journal of Clinical Psychology, 56(9), 1207–1220.

    Article  PubMed  CAS  Google Scholar 

  • Lambert, M. J., & Barley, D. E. (2001). Research summary on the therapeutic relationship and psychotherapy outcome. Psychotherapy: Theory, Research, Practice, Training, 38, 357–361.

    Article  Google Scholar 

  • Lambert, M. J., Whipple, J. L., Vermeersch, D. A., Smart, D. W., Hawkins, E. J., Nielsen, S. L., et al. (2002). Enhancing psychotherapy outcomes via providing feedback on client progress: A replication. Clinical Psychology & Psychotherapy, 9, 91–103.

    Article  Google Scholar 

  • Laurenceau, J. P., Hayes, A. M., & Feldman, G. C. (2007). Some methodological and statistical issues in the study of change processes in psychotherapy. Clinical Psychology Review, 27, 682–695.

    Article  PubMed  Google Scholar 

  • Le Grange, D., Lock, J., Agras, W. S., Moye, A., Bryson, S. W., Jo, B., et al. (2012). Moderators and mediators of remission in family-based treatment and adolescent focused therapy for Anorexia Nervosa. Behaviour Research and Therapy, 50, 85–92.

    Article  PubMed  Google Scholar 

  • Linacre, J. (2007). Winsteps Rasch measurement (Version 3.64.2). Chicago, IL: MESA Press.

    Google Scholar 

  • Mallinckrodt, B. (1994). Session impact in counseling process research: Comment on Elliott and Wexler (1994) and Stiles et al. (1994). Journal of Counseling Psychology, 41, 186–190.

    Article  Google Scholar 

  • Martin, D. J., Garske, J. P., & Davis, M. K. (2000). Relation of the therapeutic alliance with outcome and other variables: A meta-analytic review. Journal of Consulting and Clinical Psychology, 68, 438–450.

    Article  PubMed  CAS  Google Scholar 

  • Maton, K. I., Schellenbach, C. J., Leadbeater, B. J., & Solarz, A. L. (Eds.). (2004). Investing in children, youth, families, and communities: Strengths-based research and policy. Washington, D.C.: American Psychological Association.

    Google Scholar 

  • Ogles, B. M., Lambert, M. J., & Fields, S. A. (2002). Essentials of outcome assessment. New York: Wiley.

    Google Scholar 

  • Ogrodniczuk, J. S., Piper, W. E., Joyce, A. S., & McCallum, M. (2001). Effect of patient gender on outcome in two forms of short-term individual psychotherapy. Journal of Psychotherapy Practice Research, 10, 69–78.

    CAS  Google Scholar 

  • Orlinsky, D. E., & Howard, K. I. (1967). The good therapy hour: Experiential correlates of patients’ and therapists’ evaluations of therapy sessions. Archives of General Psychiatry, 16, 621–632.

    Article  PubMed  CAS  Google Scholar 

  • Orlinsky, D. E., Roonestad, M., & Willutzki, U. (2004). Fifty years of psychotherapy process-outcome research: Continuity and change. In M. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (5th ed., pp. 307–389). New York: Wiley.

    Google Scholar 

  • Owens, E. B., Hinshaw, S. P., Kraemer, H. C., Arnold, L. E., Abikoff, H. B., Cantwell, D. P., et al. (2003). Which treatment for whom for ADHD? Moderators of treatment response in the MTA. Journal of Consulting and Clinical Psychology, 71, 540–552.

    Article  PubMed  Google Scholar 

  • Prochaska, J. O., DiClemente, C. C., & Norcross, J. C. (1992). In search of how people change: Applications to addictive behaviors. American Psychologist, 47, 1102–1114.

    Article  PubMed  CAS  Google Scholar 

  • Reise, S. P., & Waller, N. G. (2009). Item response theory and clinical measurement. Annual Review of Clinical Psychology, 5, 27–48.

    Article  PubMed  Google Scholar 

  • Reynolds, S., Stiles, W. B., Barkham, M., Shapiro, D. A., Hardy, G. E., & Rees, A. (1996). Acceleration of changes in session impact during contrasting time-limited psychotherapies. Journal of Consulting and Clinical Psychology, 64, 577–586.

    Article  PubMed  CAS  Google Scholar 

  • Riemer, M., Williams, N. J., & Kearns, M. A. (unpublished manuscript). Caregiver reports on youth counseling impact: Description and psychometric evaluation of the Youth Counseling Impact Scale—Caregiver Version.

  • Riemer, M., Athay, M. M., Bickman, L., Breda, C., Kelley, S. D., & Vides de Andrade, A. (2012). The Peabody Treatment Progress Battery: History and methods for developing a comprehensive measurement battery for youth mental health. Administration and Policy in Mental Health and Mental Health Services Research. doi:10.1007/s10488-012-0404-1.

  • Riemer, M., & Kearns, M. A. (2010). Description and psychometric evaluation of the Youth Counseling Impact Scale. Psychological Assessment, 22, 259–268.

    Article  PubMed  Google Scholar 

  • Schmitt, J. S., & Di Fabio, R. P. (2004). Reliable change and minimum important difference (MID) proportions facilitated group responsiveness comparisons using individual threshold criteria. Journal of Clinical Epidemiology, 57, 1008–1018.

    Article  PubMed  Google Scholar 

  • Shirk, S. R., & Karver, M. (2003). Prediction of treatment outcome from relationship variables in child and adolescent therapy: A meta-analytic review. Journal of Consulting and Clinical Psychology, 71, 452–464.

    Article  PubMed  Google Scholar 

  • Shirk, S. R., Karver, M. S., & Brown, R. (2011). The alliance in child and adolescent psychotherapy. Psychotherapy, 48, 17–24.

    Article  PubMed  Google Scholar 

  • Southam-Gerow, M. A., Kendall, P. C., & Weersing, V. R. (2001). Examining outcome variability: Correlates of treatment response in a child and adolescent anxiety clinic. Journal of Clinical Child Psychology, 30, 422–436.

    Article  PubMed  CAS  Google Scholar 

  • Spek, V., Nyklícek, I., Cuijpers, P., & Pop, V. (2008). Predictors of outcome of group and internet-based cognitive behavior therapy. Journal of Affective Disorders, 105, 137–145.

    Article  PubMed  Google Scholar 

  • Stiles, W. B. (1980). Measurement of the impact of psychotherapy sessions. Journal of Consulting and Clinical Psychology, 48, 176–185.

    Article  PubMed  CAS  Google Scholar 

  • Stiles, W. B., Reynolds, S., Hardy, G. E., Rees, A., Barkham, M., & Shapiro, D. A. (1994). Evaluation and description of psychotherapy sessions by clients using the Session Evaluation Questionnaire and the Session Impacts Scale. Journal of Counseling Psychology, 41, 175–185.

    Article  Google Scholar 

  • Tang, T. Z., & DeRubeis, R. J. (1999). Sudden gains and critical sessions in cognitive-behavioral therapy for depression. Journal of Consulting and Clinical Psychology, 67, 894–904.

    Article  PubMed  CAS  Google Scholar 

  • Tang, T. Z., DeRubeis, R. J., Beberman, R., & Pham, T. (2005). Cognitive changes, critical sessions, and sudden gains in cognitive-behavioral therapy for depression. Journal of Consulting and Clinical Psychology, 73, 168–172.

    Article  PubMed  Google Scholar 

  • Tang, T. Z., Luborsky, L., & Andrusyna, T. (2002). Sudden gains in recovering from depression: Are they also found in psychotherapies other than cognitive-behavioral therapy? Journal of Consulting and Clinical Psychology, 70, 444–447.

    Article  PubMed  Google Scholar 

  • Thomas, J. M., & Guskin, K. A. (2001). Disruptive behavior in young children: What does it mean? Journal of the American Academy of Child and Adolescent Psychiatry, 40, 44–51.

    Article  PubMed  CAS  Google Scholar 

  • Timulak, L. (2010). Significant events in psychotherapy: An update of research findings. Psychology and Psychotherapy: Theory, Research and Practice, 83, 421–447.

    Article  Google Scholar 

  • Weisz, J. R., Weiss, B., Han, S. S., Granger, D. A., & Morton, T. (1995). Effects of psychotherapy with children and adolescents revisited: A meta-analysis of treatment outcome studies. Psychological Bulletin, 117, 450–468.

    Article  PubMed  CAS  Google Scholar 

  • Wright, B. D., & Linacre, J. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8, 370.

    Google Scholar 

  • Zahn-Waxler, C., Shirtcliff, E. A., & Marceau, K. (2008). Disorders of childhood and adolescence: Gender and psychopathology. Annual Review of Clinical Psychology, 4, 275–303.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

This research was supported by NIMH grants R01-MH068589 and 4264600201 awarded to Leonard Bickman.

Author information

Authors and Affiliations

  1. Department of Psychological Sciences, University of Missouri, 210 McAlester Hall, Columbia, MO, 65211-2500, USA

    Marcia A. Kearns

  2. Vanderbilt University, Nashville, TN, USA

    M. Michele Athay

  3. Wilfrid Laurier University, Waterloo, ON, Canada

    Manuel Riemer

Authors
  1. Marcia A. Kearns
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Michele Athay
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Manuel Riemer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marcia A. Kearns.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kearns, M.A., Athay, M.M. & Riemer, M. Measuring Youths’ Perceptions of Counseling Impact: Description, Psychometric Evaluation, and Longitudinal Examination of the Youth Counseling Impact Scale v.2. Adm Policy Ment Health 39, 104–117 (2012). https://doi.org/10.1007/s10488-012-0414-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10488-012-0414-z

Keywords

Search

Navigation