Skip to main content
Log in

The Relationship Between Clinicians’ Confidence and Accuracy, and the Influence of Child Characteristics, in the Screening of Autism Spectrum Disorder

  • Original Paper
  • Published:
Journal of Autism and Developmental Disorders Aims and scope Submit manuscript

Abstract

The study examined the confidence accuracy relationship, and the influence of child characteristics on clinician confidence, when predicting a diagnosis of Autism Spectrum Disorder during screening of 125 referred children aged under 3.5 years. The diagnostic process included observation, interview, language and developmental testing. Clinical judgement accuracy was compared against final diagnosis for high and low confidence levels (with confidence assessed on a 0–100 % scale). We identified a significant CA relationship with predictive accuracy highest at confidence levels of 90–100 %. Parent report of unusual behaviors was the only significant independent predictor of confidence. Clinicians’ confidence may be important when evaluating decisions to refer, or not to refer, children for further diagnostic assessment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. This may occur during cognitive processing. For example, Insabato et al. (2010) pose a two-stage decision mechanism where a second decision is based on confidence associated with the previous decision. More pragmatically, future decisions by other clinicians may also be influenced by earlier judgements, clinical notes, recommendations, etc.

  2. Intellectual and developmental disability clinic (n = 15); behavioral health services (n = 7); psychological assessment (n = 1); speech services (n = 2); neurodevelopmental clinic (n = 1); physical therapy (n = 1).

  3. We were able to assess 11 of the 33 children not referred for a diagnostic assessment with either the ADOS-2 or ADI-R. Four children did not meet ASD criteria on the ADI-R, five were under cut-off for ASD on the ADOS-2 and two children were in the low risk range but above cut-off for Autism Spectrum. Of the children who met ASD criteria on the ADOS-2, one had been assessed in the intellectual and developmental delay clinic and received a diagnosis of a language disorder and disruptive behavioral disorder; the other was not assessed further.

References

  • Achenbach, T. M., & Rescorla, L. A. (2000). Manual for the ASEBA Preschool forms and profiles. Burlington, VT: University of Vermont Department of Psychiatry.

    Google Scholar 

  • American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: American Psychiatric Publishing.

    Google Scholar 

  • Barbaro, J., Ridgway, L., & Dissanayake, C. (2011). Developmental surveillance of infants and toddlers by maternal and child health nurses in an Australian community-based setting: Promoting the early identification of Autism Spectrum Disorders. Journal of Pediatric Nursing, 26, 334–347.

    Article  PubMed  Google Scholar 

  • Bauminger, N., Solomon, M., & Rogers, S. J. (2010). Internalizing and externalizing behaviors in ASD. Autism Research, 3, 101–112.

    Article  PubMed  Google Scholar 

  • Brewer, N. (2006). Uses and abuses of eyewitness confidence. Legal and Criminological Psychology, 11, 3–23.

    Article  Google Scholar 

  • Brewer, N., & Wells, G. (2006). The confidence-accuracy relationship in eyewitness identification: Effects of lineup instructions, foil similarity, and target-absent base rates. Journal of Experimental Psychology: Applied, 12, 11–30.

    PubMed  Google Scholar 

  • Carlin, A. S., & Hewitt, P. L. (1990). The discrimination of patient generated and randomly generated MMPIs. Journal of Personality Assessment, 54, 24–29.

    Article  PubMed  Google Scholar 

  • Centers for Disease Control and Prevention. (2012). Prevalence of autism spectrum disorders—Autism and Developmental Disabilities Monitoring Network, 14 sites, United States, 2008. MMWR 2012; 61 (No. SS-3).

  • Charman, T., & Baird, G. (2002). Practitioner review: Diagnosis of autism spectrum disorder in 2-and 3-year-old children. Journal of Child Psychology and Psychiatry, 43, 289–305.

    Article  PubMed  Google Scholar 

  • Corsello, C., Akshoomoff, N., & Stahmer, A. (2013). Diagnosis of autism spectrum disorders in 2-year-olds: A study of community practice. Journal of Child Psychology and Psychiatry, 54, 178–185.

    Article  PubMed  Google Scholar 

  • Dawson, G., Jones, E. J. H., Merkle, K., Venema, K., Lowy, R., Faja, S., & Webb, S. J. (2012). Early behavioral intervention is associated with normalized brain activity in young children with autism. Journal of the American Academy of Child and Adolescent Psychiatry, 51, 1150–1159.

    Article  PubMed  PubMed Central  Google Scholar 

  • Dawson, G., Rogers, S., Munson, J., Smith, M., Winter, J., Greenson, J., & Varley, J. (2010). Randomized, controlled trial of an intervention for toddlers with autism: The Early Start Denver Model. Pediatrics, 125, 17–23.

    Article  Google Scholar 

  • Goldstein, S., & Naglieri, J. A. (2010). Autism Spectrum Rating Scales. North Tonawanda, NY: Multi Health Systems.

    Google Scholar 

  • Guthrie, W., Swineford, L., Nottke, C., & Wetherby, A. (2013). Early diagnosis of autism spectrum disorder: Stability and change in clinical diagnosis and symptom presentation. Journal of Child Psychology and Psychiatry, 54, 582–590.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hedley, D., Nevill, R. E., Monroy Moreno, Y., Fields, N., Wilkins, J., Butter, E., & Mulick, J. A. (2015). Efficacy of the ADEC in identifying Autism Spectrum Disorder in clinically referred toddlers in the US. Journal of Autism and Developmental Disorders, 45, 2337–2348.

    Article  PubMed  Google Scholar 

  • Insabato, A., Pannunzi, M., Rolls, E. T., & Deco, G. (2010). Confidence-related decision making. Journal of Neurophysiology, 104, 539–547.

    Article  PubMed  Google Scholar 

  • Levy, S. E., Giarelli, E., Lee, L. C., Schieve, L. A., Kirby, R. S., Cunniff, C., & Rice, C. E. (2010). Autism spectrum disorder and co-occurring developmental, psychiatric, and medical conditions among children in multiple populations of the United States. Journal of Developmental and Behavioral Pediatrics, 31, 267–275.

    Article  PubMed  Google Scholar 

  • Lichtenstein, S., & Fischhoff, B. (1977). Do those who know more also know more about how much they know? Organizational Behavior and Human Performance, 20, 159–183.

    Article  Google Scholar 

  • Lichtenstein, S., Fischhoff, B., & Phillips, L. (1982). Calibration of probabilities: The state of the art to 1980. In D. Kahneman, P. Slavic, & A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases. New York: Cambridge University Press.

    Google Scholar 

  • Lord, C., Luyster, R. J., Gotham, K., & Guthrie, W. (2012a). Autism diagnostic observation schedule, Second Edition (ADOS-2) Manual (Part II): Toddler Module. Torrance, CA: Western Psychological Services.

    Google Scholar 

  • Lord, C., Risi, S., DiLavore, P. S., Shulman, C., Thurm, A., & Pickles, A. (2006). Autism from 2 to 9 years of age. Archives of General Psychiatry, 63, 694–701.

    Article  PubMed  Google Scholar 

  • Lord, C., Rutter, M., DiLavore, P. C., Risi, S., Gotham, K., & Bishop, S. L. (2012b). Autism diagnostic observation schedule, Second Edition (ADOS-2) Manual (Part I): Modules 1–4. Torrance, CA: Western Psychological Services.

    Google Scholar 

  • Messinger, D., Young, G. S., Ozonoff, S., Dobkins, K., Carter, A., Zwaigenbaum, L., & Sigman, M. (2013). Beyond autism: A baby siblings research consortium study of high-risk children at three years of age. Journal of the American Academy of Child and Adolescent Psychiatry, 52, 300–308.

    Article  PubMed  PubMed Central  Google Scholar 

  • Miller, D. J., Spengler, E. S., & Spengler, P. M. (2015). A meta-analysis of confidence and judgment accuracy in clinical decision making. Journal of Counselling Psychology, 62, 553–567.

    Article  Google Scholar 

  • Mullen, E. M. (1995). Mullen scales of early learning (AGS ed.). Circle Pines, MN: American Guidance Service Inc.

    Google Scholar 

  • Murphy, A. H., & Winkler, R. L. (1974). Credible interval temperature forecasting: Some experimental results. Monthly Weather Review, 102, 784–794.

    Article  Google Scholar 

  • National Autism Center. (2015). Findings and conclusions: National Standard Project, Phase 2: Addressing the need for evidence-based practice guidelines for Autism Spectrum Disorder. Randolph, MA: National Autism Center.

    Google Scholar 

  • Oskamp, S. (1965). Overconfidence in case-study judgements. Journal of Consulting Psychology, 29, 261–265.

    Article  PubMed  Google Scholar 

  • Ozonoff, S., Heung, K., Byrd, R., Hansen, R., & Hertz-Picciotto, I. (2008). The onset of Autism: Patterns of symptom emergence in the first years of life. Autism Research, 1, 320–328.

    Article  PubMed  PubMed Central  Google Scholar 

  • Ozonoff, S., Young, G. S., Steinfeld, M. B., Hill, M. M., Cook, I., Hutman, T., & Sigman, M. (2009). How early do parent concerns predict later Autism diagnosis? Journal of Developmental and Behavioral Pediatrics, 30, 367–375.

    Article  PubMed  PubMed Central  Google Scholar 

  • Ronis, D. L., & Yates, J. F. (1987). Components of probability judgement accuracy: Individual consistency and effects of subject matter and assessment method. Organizational Behavior and Human Decision Processes, 40, 193–218.

    Article  Google Scholar 

  • Shimabukuro, T. T., Grosse, S. D., & Rice, C. (2008). Medical expenditures for children with an autism spectrum disorder in a privately insured population. Journal of Autism and Developmental Disorders, 38, 546–552.

    Article  PubMed  Google Scholar 

  • Simonoff, E., Pickles, A., Charman, T., Chandler, S., Loucas, T., & Baird, G. (2008). Psychiatric disorders in children with autism spectrum disorders: Prevalence, comorbidity, and associated factors in a population-derived sample. Journal of the American Academy of Child and Adolescent Psychiatry, 47, 921–929.

    Article  PubMed  Google Scholar 

  • Sparrow, S., Cicchetti, D., & Balla, D. (2005). Vineland adaptive behavior scales (2nd ed.). Minneapolis, MN: Pearson Assessment.

    Google Scholar 

  • Wiggins, L. D., Baio, J., & Rice, C. (2006). Examination of the time between first evaluation and first Autism Spectrum Diagnosis in a population-based sample. Developmental and Behavioral Pediatrics, 27, 79–87.

    Article  Google Scholar 

  • Wolff, J., Botteron, K. N., Dager, S. R., Elison, J. T., Estes, A. M., Gu, H., & Piven, J. (2014). Longitudinal patterns of repetitive behavior in toddlers with autism. Journal of Child Psychology and Psychiatry, 55, 945–953.

    Article  PubMed  PubMed Central  Google Scholar 

  • Young, R. (2007). Autism detection in early childhood: ADEC. Camberwell, VIC: ACER Press.

    Google Scholar 

Download references

Acknowledgments

This research was supported by Nationwide Children’s Hospital Clinical and Translational Intramural Grant #203213 awarded to Darren Hedley. We would like to thank the staff and families who participated in the study. We also thank Sarah Beinkampen, Natalie Fields, Mark Jones, Emily Mariotti, Yessica Monroy Moreno and Brianna Murphy for their assistance with data collection, and Dr. Elizabeth Kryszak and Dr. Nathan Weber for their contributions to the study design.

Author Contributions

DH, NB and JAM conceived and designed the study. DH and RN collected the data. DH, NB, RN and MU performed the analysis. All authors contributed to the writing and review of the manuscript. All authors approved the final version.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Darren Hedley.

Ethics declarations

Conflict of interest

The authors confirm that there are no conflict of interest to declare.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study where required to do so by the Institutional Review Board.

Appendices

Appendix 1

Questionnaire

We would like to know your view regarding your clinical impression of the child you assessed. Please take a few minutes to answer the following questions immediately following the clinical interview.

Please circle one response for the following questions:

1a) Based on the information available do you think the child has ASD?

YES (answer 1b only)

NO (answer 1c only)

1b) If you think the child has ASD, how confident are you about this?

0 %

10 %

20 %

30 %

40 %

50 %

60 %

70 %

80 %

90 %

100 %

Not confident at all

     

Absolutely certain

1c) If you think the child does not have ASD, how confident are you about this?

0 %

10 %

20 %

30 %

40 %

50 %

60 %

70 %

80 %

90 %

100 %

Not confident at all

     

Absolutely certain

Appendix 2

See Table 4.

Table 4 Correlation table of variables entered into the regression analysis (n in parentheses)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hedley, D., Brewer, N., Nevill, R. et al. The Relationship Between Clinicians’ Confidence and Accuracy, and the Influence of Child Characteristics, in the Screening of Autism Spectrum Disorder. J Autism Dev Disord 46, 2340–2348 (2016). https://doi.org/10.1007/s10803-016-2766-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10803-016-2766-9

Keywords

Navigation