Skip to main content

Advertisement

SpringerLink
Log in

Development and Piloting of a Classroom-Focused Measurement Feedback System

  • Original Paper
  • Published:
Administration and Policy in Mental Health and Mental Health Services Research Aims and scope Submit manuscript

Abstract

The present study used a community partnered research method to develop and pilot a classroom-focused measurement feedback system (MFS) for school mental health providers to support teachers’ use of effective universal and target classroom practices related to student emotional and behavioral issues. School personnel from seven urban elementary and middle school classrooms participated. Phase I involved development and refinement of the system through a baseline needs assessment and rapid-cycle feedback. Phase II involved detailed case study analysis of pre-to-post quantitative and implementation process data. Results suggest that teachers who used the dashboard along with consultation showed improvement in observed classroom organization and emotional support. Results also suggest that MFS use was tied closely to consultation dose, and that broader support at the school level was critical. Classroom-focused MFSs are a promising tool to support classroom improvement, and warrant future research focused on their effectiveness and broad applicability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Aarons, G. A., Hurlburt, M., & Horwitz, S. M. C. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4–23.

    Article  PubMed  PubMed Central  Google Scholar 

  • APA Presidential Task Force on Evidence-Based Practice. (2006). Evidence-based practice in psychology. American Psychologist, 61(4), 271–285.

    Article  Google Scholar 

  • Bambrick-Santoyo, B. (2010). Driven by Data: A practical guide to improve instruction. San Francisco: Jossey-Bass.

    Google Scholar 

  • Becker, K. D., Bradshaw, C. P., Domitrovich, C., & Ialongo, N. S. (2013a). Coaching teachers to improve implementation of the good behavior game. Administration and Policy in Mental Health and Mental Health Services Research, 40(6), 482–493.

    Article  PubMed  Google Scholar 

  • Becker, K. D., Darney, D., Domitrovich, C., Keperling, J. P., & Ialongo, N. S. (2013b). Supporting universal prevention programs: A two-phased coaching model. Clinical Child and Family Psychology Review, 16(2), 213–228.

    Article  PubMed  PubMed Central  Google Scholar 

  • Berwick, D. M. (1989). Continuous improvement as an ideal in health care. New England Journal of Medicine, 320(1), 53–56.

    Article  CAS  PubMed  Google Scholar 

  • Bickman, L., Kelley, S. D., Breda, C., de Andrade, A. R., & Riemer, M. (2011). Effects of routine feedback to clinicians on mental health outcomes of youths: results of a randomized trial. Psychiatric Services, 62(12), 1423–1429.

    Article  PubMed  Google Scholar 

  • Borntrager, C., & Lyon, A. R. (2014). Client progress monitoring and feedback in school-based mental health. Cognitive and Behavioral Practice, 22, 74–88.

    Article  PubMed Central  Google Scholar 

  • Bourdon, K. H., Goodman, R., Rae, D. S., Simpson, G., & Koretz, D. S. (2005). The Strengths and Difficulties Questionnaire: U.S. normative data and psychometric properties. Journal of the American Academy of Child and Adolescent Psychiatry, 44, 557–564.

    Article  PubMed  Google Scholar 

  • Brown, J. L., Jones, S. M., LaRusso, M. D., & Aber, J. L. (2010). Improving classroom quality: Teacher influences and experimental impacts of the 4Rs program. Journal of Educational Psychology, 102(1), 153.

    Article  Google Scholar 

  • Bruns, E. J., & Hoagwood, K. E. (2008). State implementation of evidence-based practice for youths, Part I: Responses to the state of the evidence. Journal of the American Academy of Child and Adolescent Psychiatry, 47(4), 369–373.

    Article  PubMed  Google Scholar 

  • Bruns, E. J., Hoagwood, K. E., & Hamilton, J. D. (2008). State implementation of evidence-based practice for youths, part I: Responses to the state of the evidence. Journal of the Academy of Child and Adolescent Psychiatry, 47(4), 369–373.

    Article  Google Scholar 

  • Bryk, A. S., Gomez, L., & Grunow, A. (2010). Getting ideas into action: Building networked improvement communities in education. Retrieved August 21, 2013, from http://www.carnegiefoundation.org/spotlight/webinar-bryk-gomez-building-networkedimprovement-communities-in-education.

  • Bryk, A. S., & Schneider, B. L. (2002). Trust in schools: A core resource for improvement. New York: Russell Sage.

    Google Scholar 

  • Buntin, M. B., Burke, M. F., Hoaglin, M. C., & Blumenthal, D. (2011). The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Affairs, 30(3), 464–471.

    Article  PubMed  Google Scholar 

  • Burchinal, M., Vandergrift, N., Pianta, R., & Mashburn, A. (2010). Threshold analysis of association between child care quality and child outcomes for low-income children in pre-kindergarten programs. Early Childhood Research Quarterly, 25(2), 166–176.

    Article  Google Scholar 

  • Cappella, E., Hamre, B. K., Kim, H. Y., Henry, D. B., Frazier, S. L., Atkins, M. S., & Schoenwald, S. K. (2012). Teacher consultation and coaching within mental health practice: Classroom and child effects in urban elementary schools. Journal of Consulting and Clinical Psychology, 80(4), 597–610.

    Article  PubMed  PubMed Central  Google Scholar 

  • Chorpita, B. F., Bernstein, A., & Daleiden, E. L. (2008). Driving with roadmaps and dashboards: Using information resources to structure the decision models in service organizations. Administration and Policy in Mental Health and Mental Health Services Research, 35(1–2), 114–123.

    Article  PubMed  Google Scholar 

  • Chorpita, B. F., & Daleiden, E. L. (2009). Mapping evidence-based treatments for children and adolescents: Application of the distillation and matching model to 615 treatments from 322 randomized trials. Journal of Consulting and Clinical Psychology, 77(3), 566.

    Article  PubMed  Google Scholar 

  • Cresswell, K., & Sheikh, A. (2013). Organizational issues in the implementation and adoption of health information technology innovations: An interpretative review. International journal of medical informatics, 82(5), e73–e86.

  • Deming, W. E. (1986). Out of the crisis. Cambridge: MIT Press.

    Google Scholar 

  • DiGennaro, F. D., Martens, B. K., & Kleinmann, A. E. (2007). A comparison of performance feedback procedures on teachers’ treatment implementton integrity and students’ inappropriate behavior in special education classrooms. Journal of Applied Behavior Analysis, 40(3), 447–461.

    Article  PubMed  PubMed Central  Google Scholar 

  • Domitrovich, C. E., Bradshaw, C. P., Poduska, J. M., Hoagwood, K., Buckley, J. A., Olin, S., & Ialongo, N. S. (2008). Maximizing the implementation quality of evidence-based preventive interventions in schools: A conceptual framework. Advances in School Mental Health Promotion, 1(3), 6–27.

    Article  Google Scholar 

  • Domitrovich, C. E., Cortes, R. C., & Greenberg, M. T. (2007). Improving young children’s social and emotional competence: A randomized trial of the preschool “PATHS” curriculum. The Journal of Primary Prevention, 28(2), 67–91.

    Article  PubMed  Google Scholar 

  • Duchaine, E. L., Jolivette, K., & Fredrick, L. D. (2011). The effect of teacher coaching with performance feedback on behavior-specific praise in inclusion classrooms. Education and Treatment of Children, 34(2), 209–227.

    Article  Google Scholar 

  • Duhon, G. J., Mesmer, E. M., Gregerson, L., & Witt, J. C. (2009). Effects of public feedback during RTI team meetings on teacher implementation integrity and student academic performance. Journal of School Psychology, 47(1), 19–37.

    Article  Google Scholar 

  • Duncan, K., & Pozehl, B. (2000). Effects of performance feedback on patient pain outcomes. Clinical Nursing Research, 9(4), 379–397.

    Article  CAS  PubMed  Google Scholar 

  • Embry, D. D. (2002). The Good Behavior Game: A best practice candidate as a universal behavioral vaccine. Clinical Child and Family Psychology Review, 5(4), 273–297.

    Article  PubMed  Google Scholar 

  • Evertson, C. M., & Weinstein, C. S. (2006). Handbook of classroom management: Research, practice, and contemporary issues. Mahwah: Lawrence Erlbaum Associates Publishers.

    Google Scholar 

  • Fixsen, D., Blase, K., Metz, A., & Van Dyke, M. (2013). Statewide implementation of evidence-based programs. Exceptional Children, 79(2), 213–230.

    Google Scholar 

  • Gamoran, A., Secada, W. G., & Marrett, C. B. (2000). The organizational context of teaching and learning. In M. Hallinan (Ed.), Handbook of the Sociology of Education (pp. 37–63). USA: Springer.

    Google Scholar 

  • Goodman, R. (1999). The extended version of the strengths and difficulties questionnaire as a guide to child psychiatric caseness and consequent burden. Journal of Child Psychology and Psychiatry, 40, 791–801.

    Article  CAS  PubMed  Google Scholar 

  • Goodman, R., Meltzer, H., & Bailey, V. (1998). The strengths and difficulties questionnaire: A pilot study on the validity of the self-report version. European Child and Adolescent Psychiatry, 7, 125–130.

    Article  CAS  PubMed  Google Scholar 

  • Goodman, R., & Scott, S. (1999). Comparing the Strengths and Difficulties Questionnaire and the Child Behavior Checklist: Is small beautiful? Journal of Abnormal Child Psychology, 27, 17–24.

    Article  CAS  PubMed  Google Scholar 

  • Hamre, B. K., Pianta, R. C., Downer, J. T., DeCoster, J., Mashburn, A. J., Jones, S. M., & Hamagami, A. (2013). Teaching though interactions: Testing a developmental framework of teacher effectiveness across 4000 classrooms. Elementary School Journal, 113(4), 461–487.

    Article  Google Scholar 

  • Hawkins, S. M., & Heflin, L. J. (2010). Increasing secondary teachers’ behavior-specific praise using a video self-modeling and visual performance feedback intervention. Journal of Positive Behavior Interventions, 13(2), 97–108.

    Article  Google Scholar 

  • Jensen-Doss, A., & Hawley, K. M. (2010). Understanding barriers to evidence-based assessment: Clinician attitudes toward standardized assessment tools. Journal of Clinical Child & Adolescent Psychology, 39(6), 885–896.

    Article  Google Scholar 

  • Juran, J. M. (1951). Quality control handbook. New York: McGraw-Hill.

    Google Scholar 

  • Kellermann, A. L., & Jones, S. S. (2013). What it will take to achieve the as-yet-unfulfilled promises of health information technology. Health Affairs, 32(1), 63–68.

    Article  PubMed  Google Scholar 

  • Lambert, M. J., Hansen, N. B., & Finch, A. E. (2001a). Patient-focused research: Using patient outcome data to enhance treatment effects. Journal of Consulting and Clinical Psychology, 69(2), 159.

    Article  CAS  PubMed  Google Scholar 

  • Lambert, M. J., Harmon, C., Slade, K., Whipple, J. L., & Hawkins, E. J. (2005). Providing feedback to psychotherapists on their patients’ progress: clinical results and practice suggestions. Journal of Clinical Psychology, 61(2), 165–174.

    Article  PubMed  Google Scholar 

  • Lambert, M. J., Whipple, J. L., Smart, D. W., Vermeersch, D. A., & Nielsen, S. L. (2001b). The effects of providing therapists with feedback on patient progress during psychotherapy: Are outcomes enhanced? Psychotherapy Research, 11(1), 49–68.

    Article  CAS  PubMed  Google Scholar 

  • Mashburn, A. J., Pianta, R. C., Hamre, B. K., Downer, J. T., Barbarin, O. A., Bryant, D., & Howes, C. (2008). Measures of classroom quality in prekindergarten and children’s development of academic, language, and social skills. Child Development, 79(3), 732–749.

    Article  PubMed  Google Scholar 

  • National Council on Teacher Quality. (2012). State of the States 2012: Teacher Effectiveness Policies. NCTQ State Teacher Policy Yearbook Brief. Washington, DC: National Council on Teacher Quality.

  • Office of Head Start. (2014). USe of the Classrom Assessment Scoring System (CLASS) in Head Start Retrieved October 1, 2014, from http://eclkc.ohs.acf.hhs.gov/hslc/tta-system/teaching/eecd/Assessment/Classroom.Assessment.Scoring.System%28CLASS%29/ClassroomAssessm.htm.

  • Owens, J. S., Lyon, A. R., Brandt, N. E., Warner, C. M., Nadeem, E., Spiel, C., & Wagner, M. (2014). Implementation science in school mental health: Key constructs in a developing research Agenda. School Mental Health, 6(2), 99–111.

    Article  PubMed  PubMed Central  Google Scholar 

  • Park, S., Hironaka, S., Carver, P., & Nordstrum, L. (2013). Continuous improvement in education. Carnegie Foundation for the Advancement of Teaching.

  • Pianta, R. C., La Paro, K. M., & Hamre, B. K. (2008a). Classroom assessment scoring system. Paul H. Brookes Publishing Company.

  • Pianta, R. C., Mashburn, A. J., Downer, J. T., Hamre, B. K., & Justice, L. (2008b). Effects of web-mediated professional development resources on teacher–child interactions in pre-kindergarten classrooms. Early Childhood Research Quarterly, 23(4), 431–451.

    Article  PubMed  PubMed Central  Google Scholar 

  • PracticeWise. (2014). PracticeWise. Retrieved September 15, 2014, from http://www.practicewise.com.

  • Reddy, L. A., Newman, E., De Thomas, C. A., & Chun, V. (2009). Effectiveness of school-based prevention and intervention programs for children and adolescents with emotional disturbance: A meta-analysis. Journal of School Psychology, 47(2), 77–99.

    Article  PubMed  Google Scholar 

  • Reinke, W. M., Lewis-Palmer, T., & Martin, E. (2007). The effect of visual performance feedback on teacher use of behavior-specific praise. Behavior Modification, 31(3), 247–263.

    Article  PubMed  Google Scholar 

  • Reinke, W. M., Stormont, M., Webster-Stratton, C., Newcomer, L. L., & Herman, K. C. (2012). The Incredible Years teacher classroom management program: Using coaching to support generalization to real-world classroom settings. Psychology in the Schools, 49, 416–428.

    Article  Google Scholar 

  • Rose, D. J., & Church, R. J. (1998). Learning to teach: The acquisition and maintenance of teaching skills. Journal of Behavioral Education, 8(1), 5–35.

    Article  Google Scholar 

  • Singer, M. I., Anglin, T. M., Song, L., & Lunghofer, L. (1995). Adolescents’ exposure to violence and associated symptoms of psychological trauma. JAMA, 273(6), 477–482

    Article  CAS  PubMed  Google Scholar 

  • Stamm, B. H. (2010). The concise ProQOL manual (Vol. 2007). Pocatello: ProQOL. org.

    Google Scholar 

  • State of New Jersey Department of Education. (2013). AchieveNJ Retrieved August 21, 2013, from http://www.state.nj.us/education/AchieveNJ/.

  • Stecker, P. M., Lembke, E. S., & Foegen, A. (2008). Using progress-monitoring data to improve instructional decision making. Preventing School Failure: Alternative Education for Children and Youth, 52(2), 48–58.

    Article  Google Scholar 

  • Tuckman, B. W., & Yates, D. (1980). Evaluating the student feedback strategy for changing teacher style. The Journal of Educational Research, 74(2), 74–77.

    Article  Google Scholar 

  • U.S. Department of Health and Human Services. (2003). New Freedom Commission on Mental Health: Achieving the Promise: Transforming Mental Health Care in America. Final Report. In DHHS (Ed.). Rockville, MD: Department of Health and Human Services.

  • Wells, K. B., Staunton, A., Norris, K. C., Bluthenthal, R., Chung, B., Gelberg, L., & Wong, M. (2006). Building an academic-community partnered network for clinical services research: the Community Health Improvement Collaborative (CHIC). Ethnicity and Disease, 16(1 Suppl 1), S3–S17.

    PubMed  Google Scholar 

  • Yin, R. K. (2009). Case Study Research: Design and Methods (4th ed.). Thousand Oaks: Sage Publications.

    Google Scholar 

  • Yoon, J. S. (2002). Teacher characteristics as predictors of teacher-student relationships: Stress, negative affect, and self-efficacy. Social Behavior and Personality: An International Journal, 30(5), 485–493.

    Article  Google Scholar 

  • Ysseldyke, J., & Tardrew, S. (2007). Use of a progress monitoring system to enable teachers to differentiate mathematics instruction. Journal of Applied School Psychology, 24(1), 1–28.

    Article  Google Scholar 

Download references

Acknowledgments

Writing of this paper was supported by NIMH 5K01MH83694 (EN). The authors would like to acknowledge the contributions of all of our school district collaborators on this project, and the support of the PracticeWise, LLC team in the development of this manuscript.

Author information

Authors and Affiliations

  1. Department of Child and Adolescent Psychiatry, New York University, One Park Ave, 7th Floor, NYU Child Study Center, New York, NY, 10016, USA

    Erum Nadeem

  2. Department of Applied Psychology, Steinhardt School of New York University, New York, NY, USA

    Elise Cappella

  3. Graduate School of Education, Harvard University, Cambridge, MA, USA

    Sibyl Holland

  4. Jersey City Public Schools, Jersey City, NJ, USA

    Candace Coccaro & Gerard Crisonino

Authors
  1. Erum Nadeem
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Elise Cappella
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sibyl Holland
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Candace Coccaro
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gerard Crisonino
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Erum Nadeem.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nadeem, E., Cappella, E., Holland, S. et al. Development and Piloting of a Classroom-Focused Measurement Feedback System. Adm Policy Ment Health 43, 379–393 (2016). https://doi.org/10.1007/s10488-015-0651-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10488-015-0651-z

Keywords

Search

Navigation