Skip to main content
Log in

Psychometric Evaluation of the Symptoms and Functioning Severity Scale (SFSS) Short Forms with Out-of-Home Care Youth

  • Original Paper
  • Published:
Child & Youth Care Forum Aims and scope Submit manuscript

Abstract

Background

There is a need for brief progress monitoring measures of behavioral and emotional symptoms for youth in out-of-home care. The Symptoms and Functioning Severity Scale (SFSS; Bickman et al. in Manual of the peabody treatment progress battery. Vanderbilt University, Nashville, 2010) is one measure that has clinician and youth short forms (SFSS-SFs); however, the psychometric soundness of the SFSS-SFs with youth in out-of-home care has yet to be examined.

Objective

The objective was to determine if the psychometric characteristics of the clinician and youth SFSS-SFs are viable for use in out-of-home care programs.

Methods

The participants included 143 youth receiving residential treatment and 52 direct care residential staff. The current study assessed internal consistency and alternate forms reliability for SFSS-SFs for youth in a residential care setting. Further, a binary classification test was completed to determine if the SFSS-SFs similarly classified youth as the SFSS full version for low- and elevated-severity.

Results

The internal consistency for the clinician and youth SFSS-SFs was adequate (α = .75–.82) as was the parallel forms reliability (r = .85–.97). The sensitivity (0.80–0.95), specificity (0.88–0.97), and overall accuracy (0.89–0.93) for differentiating low and elevated symptom severity was acceptable.

Conclusions

The clinician and youth SFSS-SFs have acceptable psychometrics and may be beneficial for progress monitoring; however, more research is needed to assess their sensitivity to change over time in out-of-home programs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Achenbach, T. M., & Rescorla, L. A. (2001). Manual for ASEBA school-age forms & profiles. Burlington: University of Vermont, Research Center for Children, Youth, & Families.

    Google Scholar 

  • American Educational Research Association, American Psychological Association, National Council on Measurement in Education, Joint Committee on Standards for Educational, & Psychological Testing (US). (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

  • Andrews, G., & Page, A. C. (2005). Outcome measurement, outcome management and monitoring. Australian and New Zealand Journal of Psychiatry, 39, 649–651.

    Article  PubMed  Google Scholar 

  • Athay, M. M., Riemer, M., & Bickman, L. (2012). The Symptoms and Functioning Severity Scale (SFSS): Psychometric evaluation and discrepancies among youth, caregiver, and clinician ratings over time. Administration and Policy in Mental Health and Mental Health Services Research, 39, 13–29.

    Article  PubMed Central  PubMed  Google Scholar 

  • Baker, A. J., Archer, M., & Curtis, P. (2007). Youth characteristics associated with behavioral and mental health problems during the transition to residential treatment centers: The Odyssey Project population. Child welfare, 86, 5–29.

    Google Scholar 

  • Bickman, L., Athay, M. M., Riemer, M., Lambert, E. W., Kelley, S. D., Breda, C., et al. (Eds). (2010). Manual of the peabody treatment progress battery (2nd ed.) [Electronic version]. Nashville, TN: Vanderbilt University. http://peabody.vanderbilt.edu/ptpb/.

  • Brown, J. D., Hamilton, M., Natzke, B., Ireys, H. T., & Gillingham, M. (2011). Use of out-of-home care among a statewide population of children and youth enrolled in Medicaid. Journal of Child and Family Studies, 20, 48–56.

    Article  Google Scholar 

  • Butler, L. S., & Richard, M. (2013). Evaluating components of residential treatment success for the most complex youth. Residential Treatment for Children & Youth, 30, 119–130.

    Article  Google Scholar 

  • Chambers, M. F., Saunders, A. M., New, B. D., Williams, C. L., & Stachurska, A. (2010). Assessment of children coming into care: Processes, pitfalls and partnerships. Clinical Child Psychology and Psychiatry, 15, 511–527.

    Article  PubMed  Google Scholar 

  • Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290.

    Article  Google Scholar 

  • Connor, D. F., Doerfler, L. A., Toscano, P. F, Jr, Volungis, A. M., & Steingard, R. J. (2004). Characteristics of children and adolescents admitted to a residential treatment center. Journal of Child and Family Studies, 13, 497–510.

    Article  Google Scholar 

  • Cuthbert, R., Pierre, J. S., Stewart, S. L., Cook, S., Johnson, A. M., & Leschied, A. W. (2011). Symptom persistence in seriously emotionally disordered children: Findings of a two-year follow-up after residential treatment. Child & Youth Care Forum, 40, 267–280.

    Article  Google Scholar 

  • Davis, J., & Daly, D. L. (2003). Long-term residential program training manual (4th ed.). Boys Town, NE: Father Flanagan’s Boy’s Home.

    Google Scholar 

  • Duppong Hurley, K., Lambert, M. C., & Stevens, A. S. (2014). Psychometrics of the Symptoms and Functioning Severity Scale for high-risk youth. Journal of Emotional and Behavioral Disorders. Advance online publication. doi:10.1177/1063426614535809.

  • Duppong Hurley, K., Lambert, M. C., Van Ryzin, M., Sullivan, J., & Stevens, A. (2013). Therapeutic alliance between youth and staff in residential group care: Psychometrics of the therapeutic alliance quality scale. Children and Youth Services Review, 35, 56–64.

    Article  PubMed Central  PubMed  Google Scholar 

  • Feldt, L. S. (1969). A test of the hypothesis that Cronbach’s alpha or Kuder–Richardson coefficient twenty is the same for two tests. Psychometrika, 34, 363–373.

    Article  Google Scholar 

  • Fixsen, D. L., & Blasé, K. A. (1993). Creating new realities: Program development and dissemination. Journal of Applied Behavior Analysis, 26, 597–615.

    Article  PubMed Central  PubMed  Google Scholar 

  • Goodman, R. (1997). The Strengths and Difficulties Questionnaire: A research note. Journal of Child Psychology and Psychiatry, 38, 581–586.

    Article  PubMed  Google Scholar 

  • Gresham, F. M., Cook, C. R., Collins, T., Dart, E., Rasetshwane, K., Truelson, E., et al. (2010). Developing a change-sensitive brief behavior rating scale as a progress monitoring tool for social behavior: An example using the Social Skills Rating System-Teacher form. School Psychology Review, 39, 364–379.

    Google Scholar 

  • Henson, R. K. (2001). Understanding internal consistency reliability estimates: A conceptual primer on coefficient alpha. Measurement and Evaluation in Counseling and Development, 34, 177–189.

    Google Scholar 

  • Johnson, E. S., Jenkins, J. R., Petscher, Y., & Catts, H. W. (2009). How can we improve the accuracy of screening instruments? Learning Disabilities Research & Practice, 24, 174–185. doi:10.1111/j.1540-5826.2009.00291.x.

    Article  Google Scholar 

  • Kazdin, A. E. (1994). Methodology, design and evaluation in psychotherapy research. In A. E. Bergin & S. L. Garfield (Eds.), Handbook of psychotherapy and behaviour change (4th ed., pp. 19–71). New York, NY: Wiley.

    Google Scholar 

  • Kazdin, A. E. (2005). Evidence-based assessment for children and adolescents: Issues in measurement development and clinical application. Journal of Clinical Child and Adolescent Psychology, 34, 548–558.

    Article  PubMed  Google Scholar 

  • Kirigin, K. A. (2001). The teaching-family model: A replicable system of care. Residential Treatment for Children and Youth, 18, 9–110.

    Article  Google Scholar 

  • Kish, L. (1965). Survey sampling. New York, NY: Wiley.

    Google Scholar 

  • Lambert, M., Duppong Hurley, K., Gross, T., Epstein, M., & Stevens, A. (2014). Validation of the Symptoms and Functioning Severity Scale in residential group care. Administration and Policy in Mental Health and Mental Health Services Research. Advance online publication. doi:10.1007/s10488-014-0575-z.

  • Larzelere, R. E., Daly, D. L., Davis, J. L., Chmelka, M. B., & Handwerk, M. L. (2004). Outcome evaluation of Girls and Boy’s Town family home program. Education and Treatment of Children, 27, 130–149.

    Google Scholar 

  • McKinney, C., & Morse, M. (2012). Assessment of disruptive behavior disorders: Tools and recommendations. Professional Psychology: Research and Practice, 43, 641–649.

    Article  Google Scholar 

  • Muthén, B., & Satorra, A. (1995). Complex sample data in structural equation modeling. In P. V. Marsden (Ed.), Sociological methodology (pp. 267–316). Washington, DC: American Sociological Association.

    Google Scholar 

  • Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York, NY: McGraw-Hill.

    Google Scholar 

  • Overington, L., & Ionita, G. (2012). Progress monitoring measures: A brief guide. Canadian Psychology/Psychologie Canadienne, 53, 82–92.

    Article  Google Scholar 

  • Robst, J., Armstrong, M., & Dollard, N. (2011). Comparing outcomes for youth served in treatment foster care and treatment group care. Journal of Child and Family Studies, 20, 696–705.

    Article  Google Scholar 

  • Shaffer, D., Fisher, P., Lucas, C., Dulcan, M., & Schwab-Stone, M. (2000). NIMH Diagnostic Interview Schedule for Children version IV (NIMH DISC-IV): Description, differences from previous versions, and reliability of some common diagnoses. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 28–38.

    Article  PubMed  Google Scholar 

  • Straus, M. A., & Douglas, E. M. (2004). A short form of the Revised Conflict Tactics Scales, and typologies for severity and mutuality. Violence Victims, 19, 507–520.

    Article  PubMed  Google Scholar 

  • Tarren-Sweeney, M. (2007). The Assessment Checklist for children—ACC: A behavioral rating scale for children in foster, kinship and residential care. Children and Youth Services Review, 29, 672–691.

    Article  Google Scholar 

  • Tarren-Sweeney, M. (2013a). The Assessment Checklist for adolescents—ACA: A scale for measuring the mental health of young people in foster, kinship, residential and adoptive care. Children and Youth Services Review, 35, 384–393.

    Article  Google Scholar 

  • Tarren-Sweeney, M. (2013b). The Brief Assessment Checklists (BAC-C, BAC-A): Mental health screening measures for school-aged children and adolescents in foster, kinship, residential and adoptive care. Children and Youth Services Review, 35, 771–779.

    Article  Google Scholar 

  • Turner, W., & Macdonald, G. (2011). Treatment foster care for improving outcomes in children and young people: A systematic review. Research on Social Work Practice, 21, 501–527.

    Article  Google Scholar 

  • Volpe, R. J., & Gadow, K. D. (2010). Creating abbreviated rating scales to monitor classroom inattention-overactivity, aggression, and peer conflict: Reliability, validity, and treatment sensitivity. School Psychology Review, 39, 350–363.

    Google Scholar 

  • Wolf, M., Kirigin, K., Fixsen, D., Blasé, K., & Braukmann, C. (1995). The teaching-family model: A case study in data-based program development and refinement (and dragon wrestling). Journal of Organizational Behavior Management, 15, 11–68.

    Article  Google Scholar 

  • Wolf, M., Phillips, E., Fixsen, D., Beukmann, C., Kirigin, K., Wilner, A., et al. (1976). Achievement place: The teaching-family model. Child Care Quarterly, 5, 92–103.

    Article  Google Scholar 

  • Wood, F., Flowers, L., Meyer, M., & Hill, D. (2002). How to evaluate and compare screening tests: Principles of science and good sense. Paper presented at the meeting of the International Dyslexia Association, Atlanta, GA.

Download references

Acknowledgments

The research reported herein was supported, in part, by the National Institute of Mental Health through Grant R34MH080941 and by the Institute of Education Sciences, U.S. Department of Education, through Grant R324B110001 to the University of Nebraska-Lincoln. The opinions expressed are those of the authors and do not represent views of the National Institute of Mental Health or the U.S. Department of Education.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Thomas J. Gross.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gross, T.J., Duppong Hurley, K., Lambert, M.C. et al. Psychometric Evaluation of the Symptoms and Functioning Severity Scale (SFSS) Short Forms with Out-of-Home Care Youth. Child Youth Care Forum 44, 239–249 (2015). https://doi.org/10.1007/s10566-014-9280-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10566-014-9280-z

Keywords

Navigation