Skip to main content
SpringerLink
Log in

Informant Agreement for Youth with Autism Spectrum Disorder or Intellectual Disability: A Meta-analysis

  • Original Paper
  • Published:
Journal of Autism and Developmental Disorders Aims and scope Submit manuscript

Abstract

This study investigated informant agreement on emotional and behavior problems and social skills in youth with autism spectrum disorder or intellectual disability using meta-analytic methods. Forty-nine studies were included, consisting of 107 effect sizes. The mean weighted effect size across all raters and all behaviors was .36, reflecting moderate agreement. Consistent with meta-analyses in typically developing youth, pairs of similar informants (e.g., parent–parent) demonstrated higher agreement compared to pairs of different raters (e.g., parent–teacher). With all rater pairs combined, agreement was significantly higher for externalizing problems (\( \bar{r} \) = .42) than either internalizing problems (\( \bar{r} \) = .35) or social skills (\( \bar{r} \) = .30). Several factors appear to moderate the level of agreement among informants, including the youth’s diagnosis, age, and IQ.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

* Those references were included in the meta-analysis

  • Achenbach, T. M., McConaughy, S. H., & Howell, C. T. (1987). Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin, 101, 213–232.

    Article  PubMed  Google Scholar 

  • American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text revisions). Washington: American Psychiatric Association.

  • American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Publishing.

    Book  Google Scholar 

  • *Baker, J. K., Fenning, R. M., Crnic, K. A., Baker, B. L., & Blacher, J. (2007). Prediction of social skills in 6-year-old children with and without developmental delays: Contributions of early regulation and maternal scaffolding. American Journal on Mental Retardation, 112(5), 375–391.

    Article  PubMed  Google Scholar 

  • *Baker, B. L., McIntyre, L. L., Blacher, J., Crnic, K., Edelbrock, C., & Low, C. (2003). Pre-school children with and without developmental delay: Behaviour problems and parenting stress over time. Journal of Intellectual Disability Research, 47(4–5), 217–230.

    PubMed  Google Scholar 

  • *Barnhill, G. P., Hagiwara, T., Myles, B. S., Simpson, R. L., Brick, M. L., & Griswold, D. E. (2000). Parent, teacher, and self-report of problem and adaptive behaviors in children and adolescents with Asperger syndrome. Assessment for Effective Intervention, 25(2), 147–167.

    Article  Google Scholar 

  • *Blakeley-Smith, A., Reaven, J., Ridge, K., & Hepburn, S. (2012). Parent–child agreement of anxiety symptoms in youth with autism spectrum disorders. Research in Autism Spectrum Disorders, 6(2), 707–716.

    Article  Google Scholar 

  • *Butterworth, T. W., Hodge, M. A. R., Sofronoff, K., Beaumont, R., Gray, K. M., Roberts, J., et al. (2013). Validation of the emotion regulation and social skills questionnaire for young people with autism spectrum disorders. Journal of Autism and Developmental Disorders. doi:10.1007/s10803-013-2014-5.

  • Card, N. E. (2012). Applied meta-analysis for social science research. New York, NY: The Guilford Press.

    Google Scholar 

  • *Cohen, I. L., Schmidt-Lackner, S., Romanczyk, R., & Sudhalter, V. (2003). The PDD behavior inventory: A rating scale for assessing response to intervention in children with pervasive developmental disorder. Journal of Autism and Developmental Disorders, 33(1), 31–45.

    Article  PubMed  Google Scholar 

  • *Constantino, J. N., Przybeck, T., Friesen, D., & Todd, R. D. (2000). Reciprocal social behavior in children with and without pervasive developmental disorders. Journal of Developmental and Behavioral Pediatrics, 21(1), 2–11.

    Article  PubMed  Google Scholar 

  • de Bildt, A., Serra, M., Kraijer, D., Sytema, S., & Minderaa, R. (2005). Social skills in children with intellectual disabilities with and without autism. Journal of Intellectual Disability Research, 49, 317–328.

    Article  PubMed  Google Scholar 

  • De Los Reyes, A. (2011). Introduction to the special section: More than measurement error: Discovering meaning behind informant discrepancies in clinical assessments of children and adolescents. Journal of Clinical Child and Adolescent Psychology, 40, 1–9.

    Article  PubMed  Google Scholar 

  • De Los Reyes, A., Henry, D. B., Tolan, P. H., & Wakschlag, L. S. (2009). Linking informant discrepancies to observed variations in young children’s disruptive behavior. Journal of Abnormal Child Psychology, 37, 637–652.

    Article  PubMed Central  PubMed  Google Scholar 

  • De Los Reyes, A., & Kazdin, A. E. (2005). Informant discrepancies in the assessment of childhood psychopathology: A critical review, theoretical framework, and recommendations for further study. Psychological Bulletin, 131, 483–509.

    Article  PubMed  Google Scholar 

  • *Deb, S., Dhaliwal, A.-J., & Roy, M. (2008). The usefulness of Conners’ Rating Scales-Revised in screening for attention deficit hyperactivity disorder in children with intellectual disabilities and borderline intelligence. Journal of Intellectual Disability Research, 52(11), 950–965.

    Article  PubMed  Google Scholar 

  • *Dekker, M. C., Nunn, R., & Koot, H. M. (2002). Psychometric properties of the Revised Developmental Behaviour Checklist scales in Dutch children with intellectual disability. Journal of Intellectual Disability Research, 46(1), 61–75.

    Article  PubMed  Google Scholar 

  • *Douma, J. C. H., Dekker, M. C., Verhulst, F. C., & Koot, H. M. (2006). Self-reports on mental health problems of youth with moderate to borderline intellectual disabilities. Journal of the American Academy of Child and Adolescent Psychiatry, 45(10), 1224–1231.

    Article  PubMed  Google Scholar 

  • Duhig, A. M., Renk, K., Epstein, M. K., & Phares, V. (2000). Interparental agreement on internalizing, externalizing and total behavior problems: A meta-analysis. Clinical Psychology: Science and Practice, 7, 435–453.

    Google Scholar 

  • Einfeld, S. L., Piccinin, A. M., Mackinnon, A., Hofer, S. M., Taffe, J., Gray, K. M., et al. (2006). Psychopathology in young people with intellectual disability. Journal of the American Medical Association, 296, 1981–1989.

    Article  PubMed Central  PubMed  Google Scholar 

  • Elsabbagh, M., Divan, G., Koh, Y.-J., Kim, Y. S., Kauchali, S., Marcín, C., et al. (2012). Global prevalence of autism and other pervasive developmental disorders. Autism Research, 5, 160–179.

    Article  PubMed Central  PubMed  Google Scholar 

  • *Embregts, P. J. C. M. (2000). Reliability of the child behavior checklist for the assessment of behavioral problems of children and youth with mild mental retardation. Research in Developmental Disabilities, 21(1), 31–41.

    Article  PubMed  Google Scholar 

  • *Emerson, E. (2005). Use of the Strengths and Difficulties Questionnaire to assess the mental health needs of children and adolescents with intellectual disabilities. Journal of Intellectual and Developmental Disability, 30(1), 14–23.

    Article  Google Scholar 

  • *Farmer, C. A., & Aman, M. G. (2010). Psychometric properties of the Children’s Scale of hostility and aggression: Reactive/proactive (C-SHARP). Research in Developmental Disabilities, 31(1), 270–280.

    Article  PubMed  Google Scholar 

  • *Farrugia, S., & Hudson, J. (2006). Anxiety in adolescents with Asperger syndrome: Negative thoughts, behavioral problems, and life interference. Focus on Autism and Other Developmental Disabilities, 21(1), 25–35.

    Article  Google Scholar 

  • Ferdinand, R. F., van der Ende, J., & Verhulst, F. C. (2004). Parent–adolescent disagreement regarding psychopathology in adolescents from the general population as a risk factor for adverse outcome. Journal of Abnormal Child Psychology, 113, 198–206.

    Article  Google Scholar 

  • Ferdinand, R. F., van der Ende, J., & Verhulst, F. C. (2006). Prognostic value of parent–adolescent disagreement in a referred sample. European Child and Adolescent Psychiatry, 15, 156–162.

    Article  PubMed  Google Scholar 

  • Fisher, R. A. (1938). Statistical methods for research workers (7th ed.). Edinburgh: Oliver and Boyd.

    Google Scholar 

  • *Greenaway, R., & Howlin, P. (2010). Dysfunctional attitudes and perfectionism and their relationship to anxious and depressive symptoms in boys with autism spectrum disorders. Journal of Autism and Developmental Disorders, 40, 1179–1187.

    Article  PubMed  Google Scholar 

  • *Hallett, V., Ronald, A., Colvert, E., Ames, C., Woodhouse, E., Lietz, S., et al. (2013). Exploring anxiety symptoms in a large-scale twin study of children with autism spectrum disorders, their co-twins and controls. Journal of Child Psychology and Psychiatry, 54, 1176–1185.

    Article  PubMed  Google Scholar 

  • *Hardiman, S., Guerin, S., & Fitzsimons, E. (2009). A comparison of the social competence of children with moderate intellectual disability in inclusive versus segregated school settings. Research in Developmental Disabilities, 30(2), 397–407.

    Article  PubMed  Google Scholar 

  • Hartley, A. G., Zakriski, A. L., & Wright, J. C. (2011). Probing the depths of informant discrepancies: Contextual influences on divergence and convergence. Journal of Clinical Child and Adolescent Psychology, 40, 54–66.

    Article  PubMed Central  PubMed  Google Scholar 

  • Hedges, L. V. (1982). Fitting categorical models to effect sizes from a series of experiments. Journal of Educational Statistics, 7, 119–137.

    Google Scholar 

  • Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.

    Google Scholar 

  • *Hellings, J. A., Nickel, E. J., Weckbaugh, M., McCarter, K., Mosier, M., & Schroeder, S. R. (2005). The Overt Aggression Scale for rating aggression in outpatient youth with autistic disorder: Preliminary findings. The Journal of Neuropsychiatry and Clinical Neurosciences, 17(1), 29–35.

    Article  PubMed  Google Scholar 

  • Hoyt, W. T. (2000). Rater bias in psychological research: When is it a problem and what can we do about it? Psychological Methods, 5, 64–86.

    Article  PubMed  Google Scholar 

  • *Hurtig, T., Kuusikko, S., Mattila, M., Haapsamo, H., Ebeling, H., Jussila, K., et al. (2009). Multi-informant reports of psychiatric symptoms among high-functioning adolescents with Asperger syndrome or autism. Autism, 13(6), 583–598.

    Article  PubMed  Google Scholar 

  • Israel, P., Thomsen, P. H., Langeveld, J. H., & Stormark, K. M. (2007). Parent–youth discrepancy in the assessment and treatment of youth in usual clinical care setting: Consequences to parent involvement. European Child and Adolescent Psychiatry, 16, 138–148.

    Article  PubMed  Google Scholar 

  • *Jepsen, M. I., Gray, K. M., & Taffe, J. R. (2012). Agreement in multi-informant assessment of behaviour and emotional problems and social functioning in adolescents with autistic and Asperger’s disorder. Research in Autism Spectrum Disorders, 6(3), 1091–1098.

    Article  Google Scholar 

  • *Johnson, S. A., Filliter, J. H., & Murphy, R. R. (2009). Discrepancies between self- and parent-perceptions of autistic traits and empathy in high functioning children and adolescents on the autism spectrum. Journal of Autism and Developmental Disorders, 39(12), 1706–1714.

    Article  PubMed  Google Scholar 

  • *Kalyva, E. (2010). Multirater congruence on the social skills assessment of children with Asperger syndrome: Self, mother, father, and teacher ratings. Journal of Autism and Developmental Disorders, 40(10), 1202–1208.

    Article  PubMed  Google Scholar 

  • *Kanne, S. M., Abbacchi, A. M., & Constantino, J. N. (2009). Multi-informant ratings of psychiatric symptom severity in children with autism spectrum disorders: The importance of environmental context. Journal of Autism and Developmental Disorders, 39(6), 856–864.

    Article  PubMed Central  PubMed  Google Scholar 

  • *Knott, F., Dunlop, A.-W., & Mackay, T. (2006). Living with ASD: How do children and their parents assess their difficulties with social interaction and understanding? Autism, 10, 609–617.

    Article  PubMed  Google Scholar 

  • *Koning, C., & Magill-Evans, J. (2001). Social and language skills in adolescent boys with Asperger syndrome. Autism, 5(1), 23–36.

    Article  PubMed  Google Scholar 

  • *Lane, B. R., Paynter, J., & Sharman, R. (2013). Parent and teacher ratings of adaptive and challenging behaviours in young children with autism spectrum disorders. Research in Autism Spectrum Disorders, 7, 1196–1203.

    Article  Google Scholar 

  • Lecavalier, L. (2006). Behavioral and emotional problems in young people with pervasive developmental disorders: Relative prevalence, effects of subject characteristics, and empirical classification. Journal of Autism and Developmental Disorders, 36, 1101–1114.

    Article  PubMed  Google Scholar 

  • *Lecavalier, L., Gadow, K. D., Devincent, C. J., Houts, C. R., & Edwards, M. C. (2011). Validity of DSM-IV syndromes in preschoolers with autism spectrum disorders. Autism, 15(5), 527–543.

    Article  PubMed  Google Scholar 

  • *Lecavalier, L., Leone, S., & Wiltz, J. (2006). The impact of behaviour problems on caregiver stress in young people with autism spectrum disorders. Journal of Intellectual Disability Research, 50(3), 172–183.

    Article  PubMed  Google Scholar 

  • Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.

    Google Scholar 

  • *Locke, J., Kasari, C., & Wood, J. J. (2014). Assessing social skills in early elementary-aged children with autism spectrum disorders: The social skills Q-sort. Journal of Psychoeducational Assessment, 32, 62–76.

    Article  Google Scholar 

  • *Lopata, C., Toomey, J. A., Fox, J. D., Volker, M. A., Chow, S. Y., Thomeer, M. L., et al. (2010). Anxiety and depression in children with HFASDs: Symptom levels and source differences. Journal of Abnormal Child Psychology, 38(6), 765–776.

    Article  PubMed  Google Scholar 

  • *Macintosh, K., & Dissanayake, C. (2006). Social skills and problem behaviours in school aged children with high-functioning autism and Asperger’s disorder. Journal of Autism and Developmental Disorders, 36(8), 1065–1076.

    Article  PubMed  Google Scholar 

  • *Magiati, I., Chan, J. Y., Tan, W.-L. J., & Poon, K. K. (2014). Do non-referred young people with autism spectrum disorders and their caregivers agree when reporting anxiety symptoms? A preliminary investigation using the Spence Children’s Anxiety Scale. Research in Autism Spectrum Disorders, 8, 546–558.

    Article  Google Scholar 

  • Mash, E. J., & Hunsley, J. (2005). Evidence-based assessment of child and adolescent disorders: Issues and challenges. Journal of Clinical Child and Adolescent Psychology, 34, 362–379.

    Article  PubMed  Google Scholar 

  • *Matson, J. L., Horovitz, M., Mahan, S., & Fodstad, J. (2013). Reliability of the Matson evaluation of social skills with youngsters (MESSY) for children with autism spectrum disorders. Research in Autism Spectrum Disorders, 7, 405–410.

    Article  Google Scholar 

  • *Mazefsky, C. A., Borue, X., Day, T. N., & Minshew, N. J. (2014). Emotion regulation patterns in adolescents with high-functioning autism spectrum disorder: Comparison to typically developing adolescents and association with psychiatric symptoms. Autism Research, 7, 344–354.

    Article  PubMed Central  PubMed  Google Scholar 

  • McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1, 30–46.

    Article  Google Scholar 

  • *Miller, M. L., Fee, V. E., & Jones, C. J. (2004). Psychometric properties of ADHD rating scales among children with mental retardation II: Validity. Research in Developmental Disabilities, 25(5), 477–492.

    Article  PubMed  Google Scholar 

  • *Murray, D. S., Ruble, L. A., Willis, H., & Molloy, C. A. (2009). Parent and teacher report of social skills in children with autism spectrum disorders. Language, Speech, and Hearing Services in Schools, 40(2), 109–115.

    Article  PubMed  Google Scholar 

  • *Ozsivadjian, A., Hibberd, C., & Hollocks, M. J. (2013). Brief report: The use of self-report measures in young people with autism spectrum disorder to assess symptoms of anxiety, depression and negative thoughts. Journal of Autism and Developmental Disorders. doi:10.1007/s10803-013-1937-1.

    Google Scholar 

  • Panichelli-Mindel, S. M., Flannery-Schroeder, E., Kendall, P. C., & Angelosante, A. G. (2005). Disclosure of distress among anxiety-disordered youth: Differences in treatment outcome. Journal of Anxiety Disorders, 19, 403–422.

    Article  PubMed  Google Scholar 

  • *Pearson, D. A., Aman, M. G., Arnold, L. E., Lane, D. M., Loveland, K. A., Santos, C. W., et al. (2012). High concordance of parent and teacher attention—deficit/hyperactivity disorder ratings in medicated and unmedicated children with autism spectrum disorders. Journal of Child and Adolescent Psychopharmacology, 22(4), 284–291.

    Article  PubMed Central  PubMed  Google Scholar 

  • *Reed, P., & Osborne, L. A. (2012). The role of parenting stress in discrepancies between parent and teacher ratings of behavior problems in young children with autism spectrum disorder. Journal of Autism and Developmental Disorders, 43(2), 471–477.

    Google Scholar 

  • Renk, K., & Phares, V. (2004). Cross-informant ratings of social competence in children and adolescents. Clinical Psychology Review, 24, 239–254.

    Article  PubMed  Google Scholar 

  • *Renno, P., & Wood, J. J. (2013). Discriminant and convergent validity of the anxiety construct in children with autism spectrum disorders. Journal of Autism and Developmental Disorders, 43, 2135–2146.

    Article  PubMed Central  PubMed  Google Scholar 

  • *Reszka, S. S., Boyd, B. A., McBee, M., Hume, K. A., & Odom, S. L. (2013). Brief report: Concurrent validity of autism symptom severity measures. Journal of Autism and Developmental Disorders. doi:10.1007/s10803-013-1879-7.

    Google Scholar 

  • *Rojahn, J., Rowe, E. W., Macken, J., Gray, A., Delitta, D., Booth, A., et al. (2010). Psychometric evaluation of the behavior problems inventory-01 and the Nisonger child behavior rating form with children and adolescents. Journal of Mental Health Research in Intellectual Disabilities, 3(1), 28–50.

    Article  Google Scholar 

  • Rosenthal, R. (1991). Meta-analytic procedures for social research. Newbury Park, CA: Sage.

    Book  Google Scholar 

  • *Schanding, G. T., Nowell, K. P., & Goin-Kochel, R. (2012). Utility of the Social Communication Questionnaire—Current and Social Responsiveness Scale as teacher-report screening tools for autism spectrum disorders. Journal of Autism and Developmental Disorders, 42(8), 1705–1716.

    Article  PubMed  Google Scholar 

  • *Tassé, M. J., & Lecavalier, L. (2000). Comparing parent and teacher ratings of social competence and problem behaviors. American Journal on Mental Retardation, 105(4), 252–259.

    Article  PubMed  Google Scholar 

  • *van Steensel, F. J. A., Deutschman, A. A. C. G., & Bögels, S. M. (2013). Examining the screen for child anxiety-related emotional disorder-71 as an assessment tool for anxiety in children with high-functioning autism spectrum disorders. Autism, 17, 681–692.

    Article  PubMed  Google Scholar 

  • *Vickerstaff, S., Heriot, S., Wong, M., Lopes, A., & Dossetor, D. (2007). Intellectual ability, self-perceived social competence, and depressive symptomatology in children with high-functioning autistic spectrum disorders. Journal of Autism and Developmental Disorders, 37(9), 1647–1664.

    Article  PubMed  Google Scholar 

  • *White, S. W., Schry, A. R., & Maddox, B. B. (2012). Brief report: The assessment of anxiety in high-functioning adolescents with autism spectrum disorder. Journal of Autism and Developmental Disorders, 42, 1138–1145.

    Article  PubMed  Google Scholar 

  • Wolf, F. M. (1986). Meta-analysis: Quantitative methods for research synthesis. Newbury Park, CA: Sage.

    Google Scholar 

Download references

Acknowledgments

The authors would like to thank Steven Beck, Marc Tassé, and Michael Vasey for their feedback and helpful comments related to this project.

Author information

Authors and Affiliations

  1. Department of Psychology, Nisonger Center, Ohio State University, 371D McCampbell Hall, 1581 Dodd Drive, Columbus, OH, 43210, USA

    Elizabeth A. Stratis & Luc Lecavalier

  2. Department of Psychiatry, Nisonger Center, Ohio State University, 371D McCampbell Hall, 1581 Dodd Drive, Columbus, OH, 43210, USA

    Luc Lecavalier

Authors
  1. Elizabeth A. Stratis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luc Lecavalier
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Luc Lecavalier.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Stratis, E.A., Lecavalier, L. Informant Agreement for Youth with Autism Spectrum Disorder or Intellectual Disability: A Meta-analysis. J Autism Dev Disord 45, 1026–1041 (2015). https://doi.org/10.1007/s10803-014-2258-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10803-014-2258-8

Keywords

Search

Navigation