Skip to main content
Log in

Comparing Diagnostic Outcomes of Autism Spectrum Disorder Using DSM-IV-TR and DSM-5 Criteria

  • Original Paper
  • Published:
Journal of Autism and Developmental Disorders Aims and scope Submit manuscript

Abstract

Controversy exists regarding the DSM-5 criteria for ASD. This study tested the psychometric properties of the DSM-5 model and determined how well it performed across different gender, IQ, and DSM-IV-TR sub-type, using clinically collected data on 227 subjects (median age = 3.95 years, majority had IQ > 70). DSM-5 was psychometrically superior to the DSM-IV-TR model (Comparative Fit Index of 0.970 vs 0.879, respectively). Measurement invariance revealed good model fit across gender and IQ. Younger children tended to meet fewer diagnostic criteria. Those with autistic disorder were more likely to meet social communication and repetitive behaviors criteria (p < .001) than those with PDD-NOS. DSM-5 is a robust model but will identify a different, albeit overlapping population of individuals compared to DSM-IV-TR.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. In essence favoring more complex models (Breivik and Olsson 2001).

  2. The sample size adjusted BIC-SABIC was preferred to the corrected AIC (Hurvich and Tsai 1989) for the following reason: the AICc has proven useful for the time series autoregressive models for which it was originally developed. There is to date little evidence on its utility in other types of analyses (Brockwell and Davis 1991; McQuarrie and Tsai 1998). With small, less complex models and medium sample sizes, as was the present case, both AIC and AICc will generate similar estimates. Based on the early work of Rafterty (1995), Gignac and Watkins (2013) have recommended that effect sizes need to be suggested for AIC and BIC. They recommended that difference AIC/BIC values of 2, 6, 10 or >10 units reflect “weak”, “positive”, “strong”, and “very strong” effects in favor of the simpler model.

References

  • Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716–723.

    Article  Google Scholar 

  • Akaike, H. (1980). Likelihood and the Bayes procedure. In J. M. Bernardo (Ed.), Bayesian Statistics (Vol. 31, pp. 143–166). Valencia: University Press.

    Google Scholar 

  • American Psychiatric Association. (2000). Diagnostic and Statistical Manual of Mental Disorders-IV-Text Revision. Washington, DC: American Psychiatric Publishing.

    Google Scholar 

  • American Psychiatric Association. (2013). Diagnostic and Statistical Manual of Mental Disorders-Fifth Edition. Arlington, VA: American Psychiatric Publishing.

    Book  Google Scholar 

  • Amir, R. E., Van den Veyver, I. B., Wan, M., Tran, C. Q., Francke, U., & Zoghbi, H. Y. (1999). Rett syndrome is caused by mutations in X-linked MECP2, encoding methyl-CpG-binding protein 2. Nature Genetics, 23(2), 185–188. doi:10.1038/13810.

    Article  PubMed  Google Scholar 

  • Barton, M. L., Robins, D. L., Jashar, D., Brennan, L., & Fein, D. (2013). Sensitivity and specificity of proposed DSM-5 criteria for autism spectrum disorder in toddlers. Journal of Autism and Developmental Disorders, 43(5), 1184–1195. doi:10.1007/s10803-013-1817-8.

    Article  PubMed Central  PubMed  Google Scholar 

  • Bayley, N. (2006). Manual for the Bayley Scales of Infant and Toddler Development (3rd ed.). San Antonio: The Psychological Corporation.

    Google Scholar 

  • Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238–246.

    Article  PubMed  Google Scholar 

  • Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88(3), 588–606.

    Article  Google Scholar 

  • Bollen, K. A. (1989). Structural equations with latent variables. New York, NY: Wiley.

    Book  Google Scholar 

  • Bond, T. G., & Fox, C. M. (2001). Applying the Rasch model (2nd ed.). Mahwah, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Breivik, E., & Olsson, U. (2001). Adding variables to improve fit: the effect of model size on fit assessment in Lisrel. In R. Cudeck, S. du Toit, & D. Sorbom (Eds.), Structural equation modeling: Present and future (pp. 169–194). Lincolnwood, IL: Scientific Software International.

    Google Scholar 

  • Brockwell, P. J., & Davis, R. A. (1991). Time series: theory and methods (2nd ed.). New York: Springer.

    Book  Google Scholar 

  • Charan, S. H. (2012). Childhood disintegrative disorder. Journal of Pediatric Neurosciences, 7(1), 55–57. doi:10.4103/1817-1745.97627.

    Article  PubMed Central  PubMed  Google Scholar 

  • Elliott, C. (2007). Differential ability scales (2nd ed.). San Antonio: Pearson.

    Google Scholar 

  • Enders, C. K., & Tofighi, D. (2008). The impact of misspecifying class-specific residual variances in growth mixture models. Structural Equation Modeling: A Multidisciplinary Journal, 15, 75–95.

    Article  Google Scholar 

  • Frazier, T. W., Youngstrom, E. A., Kubu, C. S., Sinclair, L., & Rezai, A. (2008). Exploratory and confirmatory factor analysis of the autism diagnostic interview-revised. Journal of Autism and Developmental Disorders, 38(3), 474–480. doi:10.1007/s10803-007-0415-z.

    Article  PubMed  Google Scholar 

  • Frazier, T. W., Youngstrom, E. A., Speer, L., Law, P., Constantino, J., Findling, R. L., et al. (2012). Validation of proposed DSM-5 criteria for autism spectrum disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 51(1), 28–40 e3. doi:10.1016/j.jaac.2011.09.021.

    Article  PubMed Central  PubMed  Google Scholar 

  • Gibbs, V., Aldridge, F., Chandler, F., Witzlsperger, E., & Smith, K. (2012). Brief report: an exploratory study comparing diagnostic outcomes for autism spectrum disorders under DSM-IV-TR with the proposed DSM-5 revision. Journal of Autism and Developmental Disorders, 42(8), 1750–1756. doi:10.1007/s10803-012-1560-6.

    Article  PubMed  Google Scholar 

  • Gignac, G. E., & Watkins, M. W. (2013). Bifactor modeling and the estimation of model-based reliability in the WAIS-IV. Multivariate Behavioral Research, 48, 639–662.

    Article  Google Scholar 

  • Gotham, K., Risi, S., Pickles, A., & Lord, C. (2007). The Autism Diagnostic Observation Schedule: revised algorithms for improved diagnostic validity. Journal of Autism and Developmental Disorders, 37(4), 613–627. doi:10.1007/s10803-006-0280-1.

    Article  PubMed  Google Scholar 

  • Guthrie, W., Swineford, L. B., Wetherby, A. M., & Lord, C. (2013). Comparison of DSM-IV and DSM-5 factor structure models for toddlers with autism spectrum disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 52(8), 797–805 e2. doi:10.1016/j.jaac.2013.05.004.

    Article  PubMed  Google Scholar 

  • Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston: Kluwer.

    Book  Google Scholar 

  • Happe, F., Ronald, A., & Plomin, R. (2006). Time to give up on a single explanation for autism. Nature Neuroscience, 9(10), 1218–1220. doi:10.1038/nn1770.

    Article  PubMed  Google Scholar 

  • Hu, L. T., & Bentler, P. M. (1995). Evaluating model fit. In R. H. Hoyle (Ed.), Structural equation modeling concepts, issues, and applications (pp. 76–99). London: Sage.

    Google Scholar 

  • Hu, L. T., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3, 424–453.

    Article  Google Scholar 

  • Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55.

    Article  Google Scholar 

  • Huerta, M., Bishop, S. L., Duncan, A., Hus, V., & Lord, C. (2012). Application of DSM-5 criteria for autism spectrum disorder to three samples of children with DSM-IV diagnoses of pervasive developmental disorders. The American Journal of Psychiatry, 169(10), 1056–1064. doi:10.1176/appi.ajp.2012.12020276.

    Article  PubMed  Google Scholar 

  • Hurvich, C. M., & Tsai, C.-L. (1989). Regression and time series model selection in small samples. Biometrika, 76(2), 297–307.

    Article  Google Scholar 

  • Insel, T. R. (2014). The NIMH Research Domain Criteria (RDoC) Project: precision medicine for psychiatry. The American Journal of Psychiatry, 171(4), 395–397. doi:10.1176/appi.ajp.2014.14020138.

    Article  PubMed  Google Scholar 

  • Insel, T., Cuthbert, B., Garvey, M., Heinssen, R., Pine, P. S., Quinn, K., et al. (2010). Research domain criteria (RDoC): Toward a new classification framework for research on mental disorders. The American Journal of Psychiatry, 167(7), 748–751. doi:10.1176/appi.ajp.2010.09091379.

    Article  PubMed  Google Scholar 

  • Joreskog, K. (1973). A general method for estimating a linear structural equation system. In A. S. Goldberger & O. D. Duncan (Eds.), Structural equation models in the social sciences (pp. 85–112). New York: Seminar Press.

    Google Scholar 

  • Kenny, D. A. (2012). Measuring model fit. Retrieved October 22, 2012 from http://www.davidakenny.net/cm/fit.htm.

  • Kent, R. G., Carrington, S. J., LeCouteur, A., Gould, J., Wing, L., Maljaars, J., et al. (2013). Diagnosing Autism Spectrum Disorder: who will get a DSM-5 diagnosis? Journal of Child Psychology and Psychiatry, 54(11), 1242–1250. doi:10.1111/jcpp.12085.

    Article  PubMed Central  PubMed  Google Scholar 

  • Kim, Y. S., Fombonne, E., Koh, Y. J., Kim, S. J., Cheon, K. A., & Leventhal, B. L. (2014). A comparison of DSM-IV pervasive developmental disorder and DSM-5 autism spectrum disorder prevalence in an epidemiologic sample. Journal of the American Academy of Child and Adolescent Psychiatry, 53(5), 500–508. doi:10.1016/j.jaac.2013.12.021.

    Article  PubMed  Google Scholar 

  • Kulage, K. M., Smaldone, A. M., & Cohn, E. G. (2014). How Will DSM-5 affect autism diagnosis? A systematic literature review and meta-analysis. Journal of Autism and Developmental Disorders, 44(8), 1918–1932. doi:10.1007/s10803-014-2065-2.

    Article  PubMed  Google Scholar 

  • Lecavalier, L., Gadow, K. D., DeVincent, C. J., Houts, C., & Edwards, M. C. (2009). Deconstructing the PDD clinical phenotype: Internal validity of the DSM-IV. Journal of Child Psychology and Psychiatry, 50(10), 1246–1254. doi:10.1111/j.1469-7610.2009.02104.x.

    Article  PubMed  Google Scholar 

  • Loehlin, J. C. (2004). Latent variable models: An introduction to factor, path, and structural equation analysis. Mahwah, NJ: Lawrence.

    Google Scholar 

  • Lord, C., & Jones, R. M. (2012). Annual research review: Re-thinking the classification of autism spectrum disorders. Journal of Child Psychology and Psychiatry, 53(5), 490–509. doi:10.1111/j.1469-7610.2012.02547.x.

    Article  PubMed Central  PubMed  Google Scholar 

  • Lord, C., Petkova, E., Hus, V., Gan, W., Lu, F., Martin, D. M., et al. (2012). A multisite study of the clinical diagnosis of different autism spectrum disorders. Archives of General Psychiatry, 69(3), 306–313. doi:10.1001/archgenpsychiatry.2011.148.

    Article  PubMed Central  PubMed  Google Scholar 

  • Lord, C., Risi, S., Lambrecht, L., Cook, E. H., Leventhal, B. L., DiLavare, P. C., et al. (2000). The autism diagnostic observation schedule-generic: A standard measure of social and communication deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders, 30(3), 205–223.

    Article  PubMed  Google Scholar 

  • MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1(2), 130–149.

    Article  Google Scholar 

  • Maglione, M. A., Gans, D., Das, L., Timbie, J., & Kasari, C. (2012). Nonmedical interventions for children with ASD: Recommended guidelines and further research needs. Pediatrics, 130(Suppl 2), S169–S178. doi:10.1542/peds.2012-0900O.

    Article  PubMed  Google Scholar 

  • Mahoney, W. J., Szatmari, P., Maclean, J. E., Bryson, S. E., Bartolucci, G., Walter, S. D., et al. (1998). Reliability and accuracy of differentiating pervasive developmental disorder subtypes. Journal of the American Academy of Child and Adolescent Psychiatry, 37(3), 278–285. doi:10.1097/00004583-199803000-00012.

    Article  PubMed  Google Scholar 

  • Mandy, W. P., Charman, T., & Skuse, D. H. (2012). Testing the construct validity of proposed criteria for DSM-5 autism spectrum disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 51(1), 41–50. doi:10.1016/j.jaac.2011.10.013.

    Article  PubMed  Google Scholar 

  • Marsh, H. W., Hau, K.-T., & Grayson, D. (2005). Goodness of fit in structural equation models. In A. Maydeu-Olivares & J. J. McArdle (Eds.), Contemporary psychometrics. A Festschrift for Roderick P. McDonald. Mahwah, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Matson, J. L., Hattier, M. A., & Williams, L. W. (2012a). How does relaxing the algorithm for autism affect DSM-V prevalence rates? Journal of Autism and Developmental Disorders, 42(8), 1549–1556. doi:10.1007/s10803-012-1582-0.

    Article  PubMed  Google Scholar 

  • Matson, J. L., Kozlowski, A. M., Hattier, M. A., Horovitz, H., & Sipes, M. (2012b). DSM-IV vs DSM-5 diagnostic criteria for toddlers with autism. Developmental Neurorehabilitation, 15(3), 185–190. doi:10.3109/17518423.2012.672341.

    Article  PubMed  Google Scholar 

  • McPartland, J. C., Reichow, B., & Volkmar, F. R. (2012). Sensitivity and specificity of proposed DSM-5 diagnostic criteria for autism spectrum disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 51(4), 368–383. doi:10.1016/j.jaac.2012.01.007.

    Article  PubMed Central  PubMed  Google Scholar 

  • McQuarrie, A. D. R., & Tsai, C.-L. (1998). Regression and time series model selection. Singapore: World Scientific.

    Book  Google Scholar 

  • Muthen, L. K., & Muthen, B. O. (2007). Mplus user’s guide 4. Los Angeles, CA: Muthen & Muthen.

    Google Scholar 

  • Norris, M., Lecavalier, L., & Edwards, M. C. (2012). The structure of autism symptoms as measured by the autism diagnostic observation schedule. Journal of Autism and Developmental Disorders, 42(6), 1075–1086. doi:10.1007/s10803-011-1348-0.

    Article  PubMed  Google Scholar 

  • Raftery, A. E. (1995). Bayesian model selection in social research. Sociological Methodology, 25, 111–163.

    Article  Google Scholar 

  • Rasch, G. (1980). Probabilistic models for some intelligence and attainment tests. Chicago, IL: The University of Chicago Press.

    Google Scholar 

  • Raykov, T. (2005). Studying group and time invariance in maximal reliability for multiple-component measuring instruments via covariance structure modelling. The British Journal of Mathematical and Statistical Psychology, 58(Pt 2), 301–317. doi:10.1348/000711005X38591.

    Article  PubMed  Google Scholar 

  • Raykov, T., & Marcoulides, G. (2000). A first course in structural equation modeling. Mahwah, NJ: Lawrence.

    Google Scholar 

  • Reise, S. (1990). A comparison of item and person fit methods of assessing model fit in IRT. Applied Psychological Measurement, 42, 127–137.

    Article  Google Scholar 

  • Rieske, R. D., Matson, J. L., Beighley, J. S., Cervantes, P. E., Goldin, R. L., & Jang, J. (2013). Comorbid psychopathology rates in children diagnosed with autism spectrum disorders according to the DSM-IV-TR and the proposed DSM-5. Developmental Neurorehabilitation: Advance online publication. doi:10.3109/17518423.2013.790519.

    Google Scholar 

  • Rigdon, E. E. (1996). CFI versus RMSEA: A comparison of two fit indexes for structural equation modeling. Structural Equation Modeling, 3(4), 369–379.

    Article  Google Scholar 

  • Ronald, A., Happe, F., Bolton, P., Butcher, M., Price, T. S., & Plomin, R. (2006). Genetic heterogeneity between the three components of the autism spectrum: A twin study. Journal of the American Academy of Child and Adolescent Psychiatry, 45(6), 691–699. doi:10.1097/01.chi.0000215325.13058.9d.

    Article  PubMed  Google Scholar 

  • Schwarz, G. E. (1978). Estimating the dimension of a model. The Annals of Statistics, 6(2), 461–464.

    Article  Google Scholar 

  • Sideridis, G. D., Simos, P., Papanicolaou, A., & Fletcher, J. (2014). Using structural equation modeling to assess functional connectivity in the brain: Power and sample size considerations. Educational and Psychological Measurement, 74, 733–758.

    Article  PubMed Central  PubMed  Google Scholar 

  • Smith, E. V, Jr. (2002). Detecting and evaluating the impact of multidimensionality using item fit statistics and principal component analysis of residuals. Journal of Applied Measurement, 3, 205–231.

    PubMed  Google Scholar 

  • Smith, R. M., Schumacker, R. E., & Bush, M. J. (1998). Using item mean squares to evaluate fit to the Rasch model. Journal of Outcome Measurement, 2, 66–78.

    PubMed  Google Scholar 

  • Steiger, J. H. (1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research, 25(2), 173–180.

    Article  Google Scholar 

  • Steiger, J. H. (2000). Point Estimation, hypothesis testing, and interval estimation using the RMSEA: Some comments and a reply to Hayduk and Glaser. Structural Equation Modeling, 7(2), 149–162.

    Article  Google Scholar 

  • Steiger, J. H. (2007). Understanding the limitations of global fit assessment in structural equation modeling. Personality and Individual Differences, 42, 893–898.

    Article  Google Scholar 

  • Stuive, I., Kiers, H. A. L., Timmerman, M. E., & ten Berge, J. M. F. (2008). The empirical verification of an assignment of items to subtests: The oblique multiple group method versus the confirmatory common factor method. Educational and Psychological Measurement, 68(6), 923–939.

    Article  Google Scholar 

  • Taheri, A., & Perry, A. (2012). Exploring the proposed DSM-5 criteria in a clinical sample. Journal of Autism and Developmental Disorders, 42(9), 1810–1817. doi:10.1007/s10803-012-1599-4.

    Article  PubMed  Google Scholar 

  • Tofghi, D., & Enders, C. K. (2007). Identifying the correct number of classes in mixture models. In G. R. Hancock & K. M. Samulelsen (Eds.), Advances in latent variable mixture models (pp. 317–341). Greenwich, CT: Information Age.

    Google Scholar 

  • Tucker, L. R., & Lewis, C. (1973). The reliability coefficient for maximum likelihood factor analysis. Psychometrica, 38, 1–10.

    Article  Google Scholar 

  • Widaman, K. F., & Thompson, J. S. (2003). On specifying the null model for incremental fit indices in structural equation modeling. Psychological Methods, 8(1), 16–37.

    Article  PubMed  Google Scholar 

  • Worley, J. A., & Matson, J. L. (2012). Comparing symptoms of autism spectrum disorders using the current DSM-IV-TR diagnostic criteria and the proposed DSM-V diagnostic criteria. Research in Autism Spectrum Disorders, 6, 965–970.

    Article  Google Scholar 

  • Young, R. L., & Rodi, M. L. (2013). Redefining Autism Spectrum Disorder Using DSM-5: The Implications of the Proposed DSM-5 Criteria for Autism Spectrum Disorders. Journal of Autism and Developmental Disorders, 44(4), 758–765. doi:10.1007/s10803-013-1927-3.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Elizabeth B. Harstad.

Appendices

Appendix 1

Multi-level model estimated to test the potential influence of inter-rater variability (bias).

Level-1 Model

$$ DSM - 5_{ij} = \beta_{0j} + \beta_{1j} *\left( {Team_{ij} } \right) \, + \beta_{2j} *\left( {Age_{ij} } \right) \, + \beta_{3j} *\left( {Gender_{ij} } \right) \, + \beta_{4j} *\left( {DSM - 4_{ij} } \right) \, + r_{ij} $$

Level-2 Model

$$ \beta_{0j} = \gamma_{00} + u_{0j} $$
$$ \beta_{1j} = \gamma_{10} $$
$$ \beta_{2j} = \gamma_{20} $$
$$ \beta_{3j} = \gamma_{30} $$
$$ \beta_{4j} = \gamma_{40} $$

The prediction of DSM-5 scores is a function of the intercept β 0j and the partial regression coefficients related to inter-rater team variability β 1j participant’s age β 2j gender β 3j and DSM-IV-TR variability β 4j plus the error of estimate r ij . The classification variable at level-2 involved the number of different diagnostic teams.

The presence of differential rater effects would be suggestive of bias in the criteria employed to clinically diagnose children with ASD. Thus, it represented an important prerequisite to validly testing the different classification systems.

Appendix 2

See Table 5.

Table 5 Power analysis from simulating the responses of the two-factor model using IRT

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Harstad, E.B., Fogler, J., Sideridis, G. et al. Comparing Diagnostic Outcomes of Autism Spectrum Disorder Using DSM-IV-TR and DSM-5 Criteria. J Autism Dev Disord 45, 1437–1450 (2015). https://doi.org/10.1007/s10803-014-2306-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10803-014-2306-4

Keywords

Navigation