Abstract
Background
Part C of the Individuals with Disabilities Education Act mandates that eligible children under age three receive free access to early intervention (EI) through state-based programs. Despite a robust “Child Find” system, enrollment remains lower than expected. Community and medical professionals often rely on parent report of service utilization to gauge whether children are receiving developmental services. Understanding the reliability of parent report of EI can guide strategies to increase EI participation.
Objective
To assess the extent to which parental reports of Part C EI services agree with state-based service records.
Method
We address this research objective among 162 children and their parents from marginalized communities in Boston, MA and New Haven, CT who were identified as having a higher likelihood of autism and participated in a randomized controlled trial to facilitate access to autism evaluation and treatment. Parents reported whether their child was currently receiving EI services at four time points and released their child’s EI service records from state agencies.
Results
Overall agreement between parent and service records was 70% and ranged from fair to moderate and service-specific agreement ranged from 46% to 81.6%. Sensitivity, specificity, and positive/negative predictive values ranged widely for both overall and service-specific EI use. We found no differences in agreement by sociodemographic characteristics, but agreement was higher when children received EI services at home.
Conclusions
Our findings support community and medical professionals’ elicitation of children’s EI service use from parents, especially for participation in EI overall and for physical and occupational therapy.
Trial Registration Number
Clinical Trials.gov Identifier: NCT02359084
Date of Trial Registration
February 9, 2015
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
American Academy of Pediatrics. (2014). AAP publications reaffirmed or retired. Pediatrics, 134(5), e1520–e1520. https://doi.org/10.1542/peds.2014-2679
American Academy of Pediatrics. (2020). AAP publications reaffirmed or retired. Pediatrics, 145(3), e20193991. https://doi.org/10.1542/peds.2019-3991
Atkins, K. L., Duvall, S. W., Dolata, J. K., Blasco, P. M., & Saxton, S. N. (2016). Part C early intervention enrollment in low birth weight infants at-risk for developmental delays. Maternal and Child Health Journal, 21(2), 290–296. https://doi.org/10.1007/S10995-016-2113-Y
Beacham, C., Reid, M., Bradshaw, J., Lambha, M., Evans, L., Gillespie, S., Klaiman, C., & Richardson, S. S. (2018). Screening for autism spectrum disorder. Journal of Developmental and Behavioral Pediatrics, 39(9), 673–682. https://doi.org/10.1097/DBP.0000000000000607
Braddock, B., & Twyman, K. (2014). Access to treatment for toddlers with autism spectrum disorders. Clinical Pediatrics, 53(3), 225–229. https://doi.org/10.1177/0009922814521284
Broder-Fingert, S., Walls, M., Augustyn, M., Beidas, R., Mandell, D., Wiltsey-Stirman, S., Silverstein, M., & Feinberg, E. (2018). A hybrid type I randomized effectiveness-implementation trial of patient navigation to improve access to services for children with autism spectrum disorder. BMC Psychiatry. https://doi.org/10.1186/s12888-018-1661-7
Brown, D. C. M., Copeland, D. K. A., Sucharew, D. H., & Kahn, D. R. S. (2012). Social-emotional problems in preschool-aged children: Opportunities for prevention and early intervention. Archives of Pediatrics and Adolescent Medicine, 166(10), 926. https://doi.org/10.1001/ARCHPEDIATRICS.2012.793
Clements, K. M., Barfield, W. D., Kotelchuck, M., Lee, K. G., Wilber, N., Clements, K. M., Barfield, W. D., Wilber, N., Kotelchuck, M., & Lee, K. G. (2006). Birth characteristics associated with early intervention referral, evaluation for eligibility, and program eligibility in the first year of life multivariate logistic regressions identified independent. Maternal and Child Health Journal, 10, 433–441. https://doi.org/10.1007/s10995-006-0080-4
Boston Public Health Commission. (2021). What we do by topic. Boston Public Health Commission. https://www.bphc.org/whatwedo/Pages/What-We-Do-by-Topic.aspx.
Commonwealth of Massachusetts. (2022). EI program contact information. https://www.mass.gov/info-details/ei-program-contact-information.
Connecticut Office of Early Childhood. (2016). Connecticut birth to three programs. https://www.birth23.org/programs/.
Council on Children with Disabilities, Section on Developmental Behavioral, Bright Futures Steering Committee, & Medical Home Initiatives for Children with Special Needs Project. (2006). Identifying infants and young children with developmental disorders in the medical home: An algorithm for developmental surveillance and screening. Pediatrics, 118(1), 405–420. https://doi.org/10.1542/peds.2006-1231
D’Souza-Vazirani, D., Minkovitz, C. S., & Strobino, D. M. (2005). Validity of maternal report of acute health care use for children younger than 3 years. Archives of Pediatrics and Adolescent Medicine, 159(2), 167–172. https://doi.org/10.1001/ARCHPEDI.159.2.167
Dettori, J. R., & Norvell, D. C. (2020). Kappa and beyond: Is there agreement? Global Spine Journal, 10(4), 499–501. https://doi.org/10.1177/2192568220911648
Doyle, O., Harmon, C. P., Heckman, J. J., & Tremblay, R. E. (2009). Investing in early human development: Timing and economic efficiency. Economics and Human Biology, 7, 1–6. https://doi.org/10.1016/j.ehb.2009.01.002
Dragojlovic, N., Kim, E., Elliott, A. M., Friedman, J. M., & Lynd, L. D. (2018). Evaluating the use of parental reports to estimate health care resource utilization in children with suspected genetic disorders. Journal of Evaluation in Clinical Practice, 24(2), 416–422. https://doi.org/10.1111/JEP.12876
Feinberg, E., Augustyn, M., Broder-Fingert, S., Bennett, A., Weitzman, C., Kuhn, J., Hickey, E., Chu, A., Levinson, J., Sandler Eilenberg, J., Silverstein, M., Cabral, H. J., Patts, G., Diaz-Linhart, Y., Rosenberg, J., Miller, J. S., Guevara, J. P., Fenick, A. M., & Blum, N. J. (2021). Effect of family navigation on diagnostic ascertainment among children at risk for autism: A randomized clinical trial from DBPNet. JAMA Pediatrics. https://doi.org/10.1001/jamapediatrics.2020.5218
Feinberg, E., Stransky, M. L., Augustyn, M., Broder-Fingert, S., Bennett, A., Weitzman, C., Kuhn, J., Chu, A., Cabral, H. J., Fenick, A. M., & Blum, N. J. (2023). Effect of family navigation on participation in part C early intervention. Academic Pediatrics. https://doi.org/10.1016/j.acap.2023.03.013
Gwet, K. L. (2008). Computing inter-rater reliability and its variance in the presence of high agreement. The British Journal of Mathematical and Statistical Psychology, 61(Pt 1), 29–48. https://doi.org/10.1348/000711006X126600
Hebbeler, K., Spiker, D., Bailey, D., Scarborough, A., Mallik, S., Simeonsson, R., Singer, M., Nelson, L., Casanueva, C., Chambers, J., Lam, I., Levin, J., Mcwilliam, R., Morrison, K., Perez, M., Raspa, M., Robinson, J., Schuster, J., Snyder, E., & Zercher, C. (2007). Early intervention for infants and toddlers with disabilities and their families: participants, services, and outcomes.
King, T. M., Tandon, S. D., Macias, M. M., Healy, J. A., Duncan, P. M., Swigonski, N. L., Skipper, S. M., & Lipkin, P. H. (2010). Implementing developmental screening and referrals: Lessons learned from a national project. Pediatrics, 125(2), 350–360. https://doi.org/10.1542/peds.2009-0388
Klein, D. (2018). Implementing a general framework for assessing interrater agreement in Stata. The Stata Journal, 18(4), 871–901.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159. https://doi.org/10.2307/2529310
Lipkin, P. H., & Macias, M. M. (2020). Promoting optimal development: Identifying infants and young children with developmental disorders through developmental surveillance and screening. Pediatrics. https://doi.org/10.1542/peds.2019-3449
Lipkin, P. H., & Okamoto, J. (2015). The individuals with disabilities education act (IDEA) for children with special educational needs. Pediatrics, 136(6), e1650–e1662. https://doi.org/10.1542/peds.2015-3409
Litt, J. S., Glymour, M. M., Hauser-Cram, P., Hehir, T., & Mccormick, M. C. (2018). Early intervention services improve school-age functional outcome among neonatal intensive care unit graduates. Academic Pediatrics, 18(4), 468–474.
McManus, B. M., Robert, S., Albanese, A., Sadek-Badawi, M., & Palta, M. (2013). Predictors of receiving therapy among very low birth weight 2 year olds eligible for Part C early intervention in Wisconsin. BMC Pediatrics. https://doi.org/10.1186/1471-2431-13-106
McManus, B. M., McCormick, M. C., Acevedo-Garcia, D., Ganz, M., & Hauser-Cram, P. (2009). The effect of state early intervention eligibility policy on participation among a cohort of young CSHCN. Pediatrics, 124(Supplement 4), S368–S374. https://doi.org/10.1542/PEDS.2009-1255G
Murray, K. D., El-Mohandes, A. A. E., El-Khorazaty, M. N., & Kiely, M. (2007). Low-income minority mothers’ reports of infant health care utilisation compared with medical records. Paediatric and Perinatal Epidemiology, 21(3), 274–283. https://doi.org/10.1111/J.1365-3016.2007.00800.X
Mussatto, K. A., Hollenbeck-Pringle, D., Trachtenberg, F., Sood, E., Sananes, R., Pike, N. A., Lambert, L. M., Mahle, W. T., Goldberg, D. J., Goldberg, C. S., Dunbar-Masterson, C., Otto, M., Marino, B. S., Bartle, B. H., Williams, I. A., Jacobs, J. P., Zyblewski, S. C., & Pemberton, V. L. (2018). Utilisation of early intervention services in young children with hypoplastic left heart syndrome. Cardiology in the Young, 28(1), 126. https://doi.org/10.1017/S104795111700169X
Palmer, L., Johnston, S. S., Rousculp, M. D., Chu, B. C., Nichol, K. L., & Mahadevia, P. J. (2012). Agreement between internet-based self- and proxy-reported health care resource utilization and administrative health care claims. Value in Health, 15(3), 458–465. https://doi.org/10.1016/J.JVAL.2011.12.015
Payakachat, N. (2018). Estimating health utility scores for children with autism using a mapping algorithm: Secondary analysis using NDAR.
Payakachat, N., Tilford, J. M., & Kuhlthau, K. A. (2018). Parent-reported use of interventions by toddlers and preschoolers with autism spectrum disorder. Psychiatric Services, 69(2), 186–194. https://doi.org/10.1176/APPI.PS.201600524
Pless, C. E., & Pless, I. B. (1995). How well they remember: the accuracy of parent reports. Archives of Pediatrics and Adolescent Medicine, 149(5), 553–558. https://doi.org/10.1001/ARCHPEDI.1995.02170180083016
Porter, S. C., Forbes, P., Manzi, S., & Kalish, L. A. (2010). Patients providing the answers: Narrowing the gap in data quality for emergency care. Quality and Safety in Health Care, 19, e34. https://doi.org/10.1136/qshc.2009.032540
Reid, G. J., Tobon, J. I., & Shanley, D. C. (2008). What is a mental health clinic? How to ask parents about help-seeking contacts within the mental health system. Administration and Policy in Mental Health and Mental Health Services Research, 35(4), 241–249. https://doi.org/10.1007/S10488-008-0165-Z
Robins, D. L., Casagrande, K., Barton, M., Chen, C. M. A., Dumont-Mathieu, T., & Fein, D. (2014). Validation of the modified checklist for autism in toddlers, revised with follow-up (M-CHAT-R/F). Pediatrics, 133(1), 37–45. https://doi.org/10.1542/peds.2013-1813
Rosenberg, S. A., Robinson, C. C., Shaw, E. F., & Ellison, M. C. (2013). Part C early intervention for infants and toddlers: Percentage eligible versus served. Pediatrics, 131(1), 38–46. https://doi.org/10.1542/PEDS.2012-1662
Rosenberg, S. A., Zhang, D., & Robinson, C. C. (2008). Prevalence of developmental delays and participation in early intervention services for young children. Pediatrics, 121(6), e1503–e1509. https://doi.org/10.1542/PEDS.2007-1680
Tang, B. G., Feldman, H. M., Huffman, L. C., Kagawa, K. J., & Gould, J. B. (2012). Missed opportunities in the referral of high-risk infants to early intervention. Pediatrics, 129(6), 1027–1034. https://doi.org/10.1542/peds.2011-2720
The City of New Haven. (n.d.). New Haven, CT - Maternal and Child Health. Retrieved December 2, 2021, from https://www.newhavenct.gov/gov/depts/health/divisions/maternal_child.htm.
Turner, B. E., Steinberg, J. R., Weeks, B. T., Rodriguez, F., & Cullen, M. R. (2022). Race/ethnicity reporting and representation in US clinical trials: A cohort study. The Lancet Regional Health–americas, 11, 100252. https://doi.org/10.1016/J.LANA.2022.100252
U.S. Department of Education, Office of Special and Rehabilitative Services, & Office of Special Education Programs. (2021). 42nd annual report to congress on the implementation of the individuals with disabilities education Act, 2020.
Ungar, W. J., Davidson-Grimwood, S. R., & Cousins, M. (2007). Parents were accurate proxy reporters of urgent pediatric asthma health services—A retrospective agreement analysis. Journal of Clinical Epidemiology, 60(11), 1176. https://doi.org/10.1016/J.JCLINEPI.2007.02.002
Vach, W., & Gerke, O. (2023). Gwet’s AC1 is not a substitute for Cohen’s kappa—A comparison of basic properties. MethodsX, 10, 102212. https://doi.org/10.1016/j.mex.2023.102212
Wang, C. J., Elliott, M. N., Rogowski, J., Lim, N., Ratner, J. A., Schuster, M. A., Health, R., Monica, S., Wang, C., & Wang, J. (2009). Factors influencing the enrollment of eligible extremely-low-birth-weight children in the part c early intervention program HHS public access. Academic Pediatric, 9(4), 283–287. https://doi.org/10.1016/j.acap.2009.04.001
Wongpakaran, N., Wongpakaran, T., Wedding, D., & Gwet, K. L. (2013). A comparison of cohen’s kappa and gwet’s AC1 when calculating inter-rater reliability coefficients: A study conducted with personality disorder samples. BMC Medical Research Methodology, 13(1), 1–7. https://doi.org/10.1186/1471-2288-13-61
Funding
This research was supported by grants from the National Institute of Mental Health (R01MH104355, NCT02359084) and Health Service Research Administration (UA3MC20218). Fellowship funding (MLS, JK) came from grant number T32HS022242 from the Agency for Healthcare Research and Quality (AHRQ), U.S. Department of Health and Human Services. The authors are solely responsible for this document’s contents, findings, and conclusions, which do not necessarily represent the views of AHRQ. Readers should not interpret any statement in this product as an official position of AHRQ or of the U.S. Department of Health and Human Services. None of the authors has any affiliation or financial involvement that conflicts with the material presented in this product.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors have no relevant financial or non-financial interests to disclose.
Human and Animal Rights
This research has been approved by the Institutional Review Board at our institution.
Informed Consent
Informed consent was obtained from all individual participants included in the study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Stransky, M.L., Kuhn, J. & Feinberg, E. Agreement Between Parental Reports of Part C Early Intervention Service Utilization and Part C Early Intervention Service Records. Child Youth Care Forum (2023). https://doi.org/10.1007/s10566-023-09773-6
Accepted:
Published:
DOI: https://doi.org/10.1007/s10566-023-09773-6