Skip to main content
SpringerLink
Log in

Leveraging Technology to Facilitate Teachers’ Use of a Targeted Classroom Intervention: Evaluation of the Daily Report Card.Online (DRC.O) System

  • Original Paper
  • Published:
School Mental Health Aims and scope Submit manuscript

Abstract

Historically, teachers’ uptake and implementation of empirically supported classroom interventions have involved substantial face-to-face consultation. However, most schools do not have the resources to provide this intensive level of support and many teachers may not need it. Thus, evaluation of alternative supports is warranted. In this pilot study, we evaluated the feasibility and effectiveness of an interactive web-based technology [i.e., the Daily Report Card.Online (DRC.O)] designed to facilitate teachers’ use of the DRC with minimal external support and examined individual teacher characteristics associated with DRC use. Elementary school teachers (N = 54) were given access to the DRC.O Web site and asked to use it to implement a DRC with one child. With regard to implementation, 16% were short-term adopters (less than 1 month) and 39% were long-term adopters (1–7 months of use). On average, short-term adopters adhered to 37% of data entry procedures, whereas long-term adopters adhered to 74% of data entry procedures. Higher teacher stress was associated with shorter use and lower adherence. Web site analytics revealed that, on average, long-term adopters completed all steps of DRC development in less than 1 h and spent only 3 min per day engaged in data entry for progress monitoring. The magnitude of change in student target behaviors and Strengths and Difficulties Questionnaire scores was moderate to large. These results reveal the feasibility and promise of the DRC.O and generate hypotheses for future research. Implications for additional evaluation of technology-driven implementation supports for teachers are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research,38(1), 4–23.

    Article  PubMed  Google Scholar 

  • Alberta Government. (2013). A transformation in progress: Alberta’s K-12 education workforce 2011–2012. Retrieved at https://open.alberta.ca/publications/9781460106884 on December 8, 2018.

  • Atkins, M. S., Frazier, S. L., Leathers, S. J., Graczyk, P. A., Talbott, E., Jakobsons, L., et al. (2008). Teacher key opinion leaders and mental health consultation in low-income urban schools. Journal of Consulting and Clinical Psychology,76(5), 905–908.

    Article  PubMed  Google Scholar 

  • Becker, K. D., Bohnenkamp, J., Domitrovich, C., Keperling, J. P., & Ialongo, N. S. (2014). Online training for teachers delivering evidence-based preventative interventions. School Mental Health,6(4), 225–236.

    Article  Google Scholar 

  • Becker, K. D., Bradshaw, C. P., Domitrovich, C., & Ialongo, N. S. (2013). Coaching teachers to improve implementation of the good behavior game. Administration and Policy in Mental Health and Mental Health Services Research,40, 482–493.

    Article  PubMed  Google Scholar 

  • Benner, G. J., Kutash, K., Nelson, J. R., & Fisher, M. B. (2013). Closing the achievement gap of youth with emotional and behavioral disorders through multi-tiered systems of support. Education and Treatment of Children,36, 15–29.

    Article  Google Scholar 

  • Bishop, D. C., Giles, S. M., & Bryant, K. S. (2005). Teacher receptiveness toward web-based training and support. Teaching and Teacher Education,21(1), 3–14.

    Article  Google Scholar 

  • Canadian Teachers’ Federation. (2018). Teacher voices on work-life balance. Available from https://www.ctf-fce.ca/en/Pages/Issues/Teachers-Survey-Infographic.aspx. Retrieved on September 27, 2018.

  • Carey, R., Kleiman, G., Russell, M., Douglas Venable, J. D., & Louie, J. (2008). Online courses for math teachers: Comparing self-paced and facilitated cohort approaches. The Journal of Technology, Learning and Assessment,7(3), 4–35.

    Google Scholar 

  • Coles, E. K., Owens, J. S., Serrano, V. J., Slavec, J., & Evans, S. W. (2015). From consultation to student outcomes: The role of teacher knowledge, skills, and beliefs in increasing integrity in classroom management strategies. School Mental Health,7(1), 34–48.

    Article  Google Scholar 

  • Collier-Meek, M. A., Sanetti, L. M., & Boyle, A. M. (2018). Barriers to implementing classroom management and behavior support plans: An exploratory investigation. Psychology in the Schools,56, 5–17.

    Article  Google Scholar 

  • Conroy, M. A., Sutherland, K. S., Algina, J. J., Wilson, R. E., Martinez, J. R., & Whalon, K. J. (2015). Measuring teacher implementation of the BEST in CLASS intervention program and corollary child outcomes. Journal of Emotional and Behavioral Disorders,23, 144–155.

    Article  Google Scholar 

  • Domitrovich, C. E., Pas, E. T., Bradshaw, C. P., Becker, K. D., Keperling, J. P., Embry, D. D., et al. (2015). Individual and school organizational factors that influence implementation of the PAX good behavior game intervention. Prevention Science,16(8), 1064–1074.

    Article  PubMed  Google Scholar 

  • Epstein, M., Atkins, M., Culinan, D., Kutash, K., & Weaver, R. (2008). Reducing behavior problems in the elementary school classroom. IES Practice Guide (NCEE 2008-012). U.S. Department of Education, National Center for Education Statistics. Washington, DC: U.S. Government Printing Office.

  • Fabiano, G. A., Vujnovic, R., Pelham, W. E., Waschbusch, D. A., Massetti, G. M., Yu, J., et al. (2010). Enhancing the effectiveness of special education programming for children with ADHD using a daily report card. School Psychology Review,39, 219–239.

    Google Scholar 

  • Farley-Ripple, E., & Buttram, J. (2015). The development of capacity for data use: The role of teacher networks in an elementary school. Teachers College Record,117(4), 1–34.

    Google Scholar 

  • Fimian, M. J. (1988). Teacher stress inventory. Brandon, VT: Clinical Psychology Publishing Company.

    Google Scholar 

  • Fimian, M. J., & Krupicka, W. M. (1987). Occupational stress and receipt of professional counseling in special education. Perceptual and Motor Skills,65, 995–999.

    Article  PubMed  Google Scholar 

  • Forness, S. R., Freeman, S. F., Paparella, T., Kauffman, J. M., & Walker, H. M. (2012). Special education implications of point and cumulative prevalence for children with emotional or behavioral disorders. Journal of Emotional and Behavioral Disorders,20(1), 4–18.

    Article  Google Scholar 

  • Frank, J. L., & Kratochwill, T. R. (2014). School-based problem-solving consultation. In W. P. Erchul & S. M. Sheridan (Eds.), Handbook of research in school consultation (2nd ed., pp. 18–39). New York: Routledge.

    Google Scholar 

  • Girio, E. L., & Owens, J. S. (2009). Teacher acceptability of evidence-based and promising treatments for children with attention-deficit/hyperactivity disorder. School Mental Health,1, 16–25. https://doi.org/10.1007/s12310-008-9001-6.

    Article  Google Scholar 

  • Goodman, R. (2001). Psychometric properties of the strengths and difficulties questionnaire. Journal of the American Academy of Child & Adolescent Psychiatry,40(11), 1337–1345.

    Article  Google Scholar 

  • Hart, K. C., Fabiano, G. A., Evans, S. W., Manos, M. J., Hannah, J. N., & Vujnovic, R. K. (2017). Elementary and middle school teachers’ self-reported use of positive behavioral supports for children with ADHD: A national survey. Journal of Emotional and Behavioral Disorders,25(4), 246–256.

    Article  Google Scholar 

  • Holdaway, A. S., Owens, J. S., Evans, S. W., Coles, E. K., Egan, T. E., & Himawan, L. K. (2018, October). Incremental benefits of a daily report card over time for youth with or at-risk for ADHD: Replication and extension. Poster presented at the Annual Conference on Advancing School Mental Health, Las Vegas, NV.

  • Ingersoll, R. M. (2001). Teacher turnover and teacher shortages: An organizational analysis. American Educational Research Journal,38(3), 499–534.

    Article  Google Scholar 

  • Kelly, M. S., & Lueck, C. (2011). Adopting a data-driven public health framework in schools: Results from a multi-disciplinary survey on school-based mental health practice. Advances in School Mental Health Promotion,4(4), 5–12.

    Article  Google Scholar 

  • Kratochwill, T. R., Elliott, S. N., & Busse, R. T. (1995). Behavior consultation: A five-year evaluation of consultant and client outcomes. School Psychology Quarterly,10(2), 87–117.

    Article  Google Scholar 

  • Long, A. C., Sanetti, L. M. H., Collier-Meek, M. A., Gallucci, J., Altschaefl, M., & Kratochwill, T. R. (2016). An exploratory investigation of teachers’ intervention planning and perceived implementation barriers. Journal of School Psychology,55, 1–26.

    Article  PubMed  Google Scholar 

  • Martinussen, R., Tannock, R., & Chaban, P. (2011). Teachers’ reported use of instructional and behavior management practices for students with behavior problems: Relationship to role and level of training in ADHD. Child & Youth Care Forum,40(3), 193–210.

    Article  Google Scholar 

  • McLean, L., & Connor, C. M. (2015). Depressive symptoms in third-grade teachers: Relations to classroom quality and student achievement. Child Development,86(3), 945–954.

    Article  PubMed  PubMed Central  Google Scholar 

  • McLeod, B. D., Sutherland, K. S., Martinez, R. G., Conroy, M. A., Snyder, P. A., & Southam-Gerow, M. A. (2017). Identifying common practice elements to improve social, emotional, and behavioral outcomes of young children in early childhood classrooms. Prevention Science,18(2), 204–213.

    Article  PubMed  Google Scholar 

  • Mixon, C. S., Owens, J. S., Hustus, C., Serrano, V. J., & Holdaway, A. S. (2019). Evaluating the impact of online professional development on teachers’ use of a targeted behavioral classroom intervention. School Mental Health,11(1), 115–128.

    Article  Google Scholar 

  • Owens, J. S., Coles, E. K., & Evans, S. W. (2014, September). The role of teacher knowledge, skills, and beliefs in the implementation of classroom management skills. Presentation at the Annual School Mental Health Research Summit, Pittsburgh, PA.

  • Owens, J. S., Coles, E. K., Evans, S. W., Himawan, L. K., Girio-Herrera, E., Holdaway, A. S., et al. (2017). Using multi-component consultation to increase the integrity with which teachers implement behavioral classroom interventions: A pilot study. School Mental Health,9, 218–234.

    Article  Google Scholar 

  • Owens, J. S., Evans, S. W., Coles, E. K., Himawan, L. K., Holdaway, A. S., Mixon, C., et al. (2018). Consultation for classroom management and targeted interventions: examining benchmarks for teacher practices that produce desired change in student behavior. Journal of Emotional and Behavioral Disorders. https://doi.org/10.1177/106342661879544.

    Article  Google Scholar 

  • Owens, J. S., Holdaway, A. S., Zoromski, A. K., Evans, S. W., Himawan, L. K., Girio-Herrera, E., et al. (2012). Incremental benefits of a daily report card intervention over time for youth with disruptive behavior. Behavior Therapy,43, 848–861.

    Article  PubMed  Google Scholar 

  • Owens, J. S., Murphy, C. E., Richerson, L., Girio, E. L., & Himawan, L. K. (2008). Science to practice in underserved communities: The effectiveness of school mental health programming. Journal of Clinical Child and Adolescent Psychology,37, 434–447.

    Article  PubMed  Google Scholar 

  • Parker, R. I., Vannest, K. J., Davis, J. L., & Sauber, S. B. (2011). Combining nonoverlap and trend for single-case research: Tau-U. Behavior Therapy,42(2), 284–299.

    Article  PubMed  Google Scholar 

  • Pianta, R. C., Mashburn, A. J., Downer, J. T., Hamre, B. K., & Justice, L. (2008). Effects of web-mediated professional development resources on teacher-child interactions in pre-kindergarten classrooms. Early Childhood Research Quarterly,23, 431–451.

    Article  PubMed  PubMed Central  Google Scholar 

  • Pyle, K., & Fabiano, G. A. (2017). Daily report card intervention and attention deficit hyperactivity disorder: A meta-analysis of single-case studies. Exceptional Children,83(4), 378–395.

    Google Scholar 

  • Russell, M., Carey, R., Kleiman, G., & Venable, J. D. (2009). Face-to-face and online professional development for mathematics teachers: A comparative study. Journal of Asynchronous Learning Networks,13(2), 71–87.

    Google Scholar 

  • Sanetti, L. M. H., & Kratochwill, T. R. (2009). Toward developing a science of treatment integrity: Introduction to the special series. School Psychology Review,38(4), 445–459.

    Google Scholar 

  • Solomon, B. G., Klein, S. A., & Politylo, B. C. (2012). The effect of performance feedback on teachers’ treatment integrity: A meta-analysis of the single-case literature. School Psychology Review,41(2), 160–175.

    Google Scholar 

  • Troutman, A. C., Owens, J. S., Monopoli, W. J., Evans, S. W., & Coles, E. C. (2018, October). School, teacher, and student factors that predict use of effective classroom management practices. Poster presented at the Annual Conference on Advancing School Mental Health, Las Vegas, NV.

  • Vannest, K. J., Davis, J. L., Davis, C. R., Mason, B. A., & Burke, M. D. (2010). Effective intervention for behavior with a daily behavior report card: A meta-analysis. School Psychology Review,39(4), 654–672.

    Google Scholar 

  • Vannest, K. J., & Ninci, J. (2015). Evaluating intervention effects in single-case research designs. Journal of Counseling & Development,93(4), 403–411.

    Article  Google Scholar 

  • Wehby, J. H., Maggin, D. M., Partin, T. C. M., & Robertson, R. (2012). The impact of working alliance, social validity, and teacher burnout on implementation fidelity of the good behavior game. School Mental Health,4(1), 22–33.

    Article  Google Scholar 

Download references

Acknowledgements

The authors express their gratitude to the educators, administrators, and families in the participating school districts for their collaboration.

Funding

This study was funded by the PolicyWise for Children & Families, Alberta, Canada (Grant No. 1609SM). The opinions expressed are those of the authors and do not represent views of PolicyWise.

Author information

Author notes

  1. Clifton S. Mixon

    Present address: Ochsner Hospital for Children, 1514 Jefferson Hwy, Jefferson, LA, USA

Authors and Affiliations

  1. Ohio University, Porter Hall 200, Athens, OH, 45701, USA

    Julie Sarno Owens, Chelsea L. Hustus & Clifton S. Mixon

  2. CHEO-RI, University of Calgary, 401 Smyth Road, Ottawa, ON, K1H 8L1, Canada

    John D. McLennan

  3. University of Calgary, 3280 Hospital Drive NW, Calgary, AB, T2N 4Z6, Canada

    Rebecca Haines-Saah

  4. University of Calgary, 2500 University Drive NW, Calgary, AB, T2N 1N4, Canada

    Sarah Mitchell

  5. Spelman College, Atlanta, GA, USA

    Ayanna Troutman

Authors
  1. Julie Sarno Owens
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. John D. McLennan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chelsea L. Hustus
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rebecca Haines-Saah
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sarah Mitchell
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Clifton S. Mixon
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ayanna Troutman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Julie Sarno Owens.

Ethics declarations

Conflict of interest

The first author received no financial benefit from the DRC.O program, but is the primary developer of the Web site. Other authors have no conflicts of interest to disclose.

Ethical Standard

Procedures were approved by the Institutional Review Boards at both universities and within all school districts. All procedures were performed in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Owens, J.S., McLennan, J.D., Hustus, C.L. et al. Leveraging Technology to Facilitate Teachers’ Use of a Targeted Classroom Intervention: Evaluation of the Daily Report Card.Online (DRC.O) System. School Mental Health 11, 665–677 (2019). https://doi.org/10.1007/s12310-019-09320-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12310-019-09320-6

Keywords

Search

Navigation