Skip to main content
SpringerLink
Log in

A hybrid framework for detection of autism using ConvNeXt-T and embedding clusters

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Autism spectrum disorder (ASD) is a neurodevelopment disorders, which is very difficult to diagnose at early stages of life, and is a complex spectrum disorder, involving many persistent challenges with lack of social interactions, eye contact and facial expressions. Facial features of children having ASD are different from typically developing (TD) children, which is a major clue for early detection of autism using demographics of facial features. In this paper, a novel framework for early detection of autism has been developed, using a semi-supervised approach that consists ConvNeXt-T model and clustering of image embedding vectors. State-of-the-art ConvNeXt-T model is trained for autism dataset, and after that, image embedding clusters are created based on cosine similarity threshold. By doing this, model’s classification layer and embedding generation layers are trained for the subject data. The proposed framework provides embeddings and autism classifier from the trained model, where cluster embeddings can be used in semi-supervised classification. Hence, after training process, inference architecture has two modes: (i) classification using ConvNeXt-T and (ii) classification using embedding clusters. The classification accuracy of framework using ConvNeXt-T is 89.5%, where clustering accuracy using cosine similarity of image embedding is 93%. Most of the other techniques in this area focus only on classification using deep neural networks. The proposed framework is also providing classification based on embedding clusters and can work on unlabelled dataset. Moreover, the proposed framework is based on server–client architecture, which allows easy usability and integration without multi acceleration devices and extra resource consumption. A single image of child is required to detect autism using the proposed framework in real-life scenario.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data availability

Data shall be made available on request.

References

  1. van’t Hof M, Tisseur C, van Berckelear-Onnes I, van Nieuwenhuyzen A, Daniels AM, Deen M, Ester WA (2021) Age at autism spectrum disorder diagnosis: a systematic review and meta-analysis from 2012 to 2019. Autism 25(4):862–873

    Article  Google Scholar 

  2. Wergeland GJH, Posserud MB, Fjermestad K, Njardvik U, Öst LG (2022) Early behavioral interventions for children and adolescents with autism spectrum disorder in routine clinical care: a systematic review and meta-analysis. Clin Psychol Sci Pract 29(4):400

    Article  Google Scholar 

  3. Hyman SL, Levy SE, Myers SM, Kuo DZ, Apkon S, Davidson LF, Bridgemohan C (2020) Identification, evaluation, and management of children with autism spectrum disorder. Pediatrics. https://doi.org/10.1542/peds.2019-3447

    Article  Google Scholar 

  4. Goedeke S, Shepherd D, Landon J, Taylor S (2019) How perceived support relates to child autism symptoms and care-related stress in parents caring for a child with autism. Res Autism Spectr Disorders 60:36–47

    Article  Google Scholar 

  5. Chen MH, Huang CF, Lin YS, Chiu YN, Gau SSF, Wu YY (2023) Validation of the mandarin chinese version of the autism diagnostic observation schedule-2 for autism spectrum disorder. Res Autism Spectr Disorders 105:102184

    Article  Google Scholar 

  6. Raj S, Masood S (2020) Analysis and detection of autism spectrum disorder using machine learning techniques. Proc Comput Sci 167:994–1004

    Article  Google Scholar 

  7. Nogay HS, Adeli H (2020) Machine learning (ML) for the diagnosis of autism spectrum disorder (ASD) using brain imaging. Rev Neurosci 31(8):825–841

    Article  Google Scholar 

  8. Tan DW, Gilani SZ, Maybery MT, Mian A, Hunt A, Walters M, Whitehouse AJ (2017) Hypermasculinised facial morphology in boys and girls with autism spectrum disorder and its association with symptomatology. Sci Rep 7(1):9348

    Article  Google Scholar 

  9. Boutrus M, Gilani SZ, Alvares GA, Maybery MT, Tan DW, Mian A, Whitehouse AJ (2019) Increased facial asymmetry in autism spectrum conditions is associated with symptom presentation. Autism Res 12(12):1774–1783

    Article  Google Scholar 

  10. Mahapatra P, Pati S, Sinha R, Chauhan AS, Nanda RR, Nallala S (2019) Parental care-seeking pathway and challenges for autistic spectrum disorders children: a mixed method study from Bhubaneswar. Odisha Indian J Psychiatr 61(1):37

    Google Scholar 

  11. Ismail NAS, Ramli NS, Hamzaid NH, Hassan NI (2020) Exploring eating and nutritional challenges for children with autism spectrum disorder: parents’ and special educators’ perceptions. Nutrients 12(9):2530

    Article  Google Scholar 

  12. Anagnostopoulou P, Alexandropoulou V, Lorentzou G, Lykothanasi A, Ntaountaki P, Drigas A (2020) Artificial intelligence in autism assessment. Int J Emerg Technol Learn (iJET) 15(6):95–107

    Article  Google Scholar 

  13. de Belen RAJ, Bednarz T, Sowmya A, Del Favero D (2020) Computer vision in autism spectrum disorder research: a systematic review of published studies from 2009 to 2019. Transl Psychiatr 10(1):333

    Article  Google Scholar 

  14. Abdelrahman M, Ali A, Farag A, Casanova MF, Farag A (2012) New approach for classification of autistic vs. typically developing brain using white matter volumes. In: Proc. Ninth Conference on Computer and Robot Vision, pp 284–289

  15. Eslami T, Saeed F (2015) Auto-ASD-Network: A Technique Based on Deep Learning and Support Vector Machines for Diagnosing Autism Spectrum Disorder Using fMRI data. In: Proc 10th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, pp 646–651 (Association for Computing Machinery)

  16. Kalantarian H et al (2018) A Gamified Mobile System for Crowdsourcing Video for Autism Research. In: Proc IEEE International Conference on Healthcare Informatics (ICHI), pp 350–352

  17. Guha T, Yang Z, Grossman RB, Narayanan SS (2018) A computational study of expressive facial dynamics in children with autism. IEEE Trans Affect Comput 9:14–20

    Article  Google Scholar 

  18. Shi L et al (2015) Different visual preference patterns in response to simple and complex dynamic social stimuli in preschool-aged children with autism spectrum disorders. PLoS ONE 10:e0122280

    Article  Google Scholar 

  19. Chong E et al (2017) Detecting gaze towards eyes in natural social interactions and its use in child assessment. Proc ACM Interact Mob Wearable Ubiquitous Technol. https://doi.org/10.1145/3131902

    Article  Google Scholar 

  20. Dawson G et al (2018) Atypical postural control can be detected via computer vision analysis in toddlers with autism spectrum disorder. Sci Rep 8:17008

    Article  Google Scholar 

  21. Zunino A et al (2018) Video Gesture Analysis for Autism Spectrum Disorder Detection In: Proc 24th International Conference on Pattern Recognition (ICPR), pp 3421–3426

  22. https://drive.google.com/drive/folders/1XQU0pluL0m3TIlXqntano12d68peMb8A

  23. Shukla P, Gupta T, Saini A, Singh P, Balasubramanian R (2017) A Deep Learning Frame-Work for Recognizing Developmental Disorders. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), IEEE, pp 705–714

  24. Haque MIU, Valles D.(2018) A Facial Expression Recognition Approach Using DCNN for Autistic Children to Identify Emotions. In: 2018 IEEE 9th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), IEEE, pp 546–551

  25. Rudovic O, Utsumi Y, Lee J, Hernandez J, Ferrer EC, Schuller B, Picard RW (2018). Culturenet: A Deep Learning Approach for Engagement Intensity Estimation from Face Images of Children with Autism. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE, pp 339–346

  26. Dong S, Wang P, Abbas K (2021) A survey on deep learning and its applications. Comput Sci Rev 40:100379

    Article  MathSciNet  Google Scholar 

  27. Liu Z, Mao H, Wu CY, Feichtenhofer C, Darrell T, Xie S (2022) A Convnet for the 2020s. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 11976–11986

  28. Trinh TH, Luong MT, Le QV (2019) Selfie: Self-Supervised Pretraining for image Embedding. arXiv preprint arXiv:1906.02940

  29. Park K, Hong JS, Kim W (2020) A methodology combining cosine similarity with classifier for text classification. Appl Artif Intell 34(5):396–411

    Article  Google Scholar 

  30. Yang E, Zhou W, Qian X, Yu L (2022) MGCNet: multilevel gated collaborative network for RGB-D semantic segmentation of indoorscene. IEEE Signal Process Lett 29:2567–2571

    Article  Google Scholar 

  31. Haque MIU, Valles D (2018) A Facial Expression Recognition Approach Using DCNN for Autistic Children to Identify Emotions. In: 2018 IEEE 9th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), IEEE, pp 546–551

  32. Jahanara S, Padmanabhan S (2021) Detecting autism from facial image. Int J Adv Res, Ideas Innov Technol 7(2):219–225

    Google Scholar 

  33. Alsaade FW, Alzahrani MS (2022) Classification and detection of autism spectrum disorder based on deep learning algorithms. Comput Intell Neurosci. https://doi.org/10.1155/2022/8709145

    Article  Google Scholar 

  34. Akter T, Ali MH, Khan MI, Satu MS, Uddin MJ, Alyami SA, Moni MA (2021) Improved transfer-learning-based facial recognition framework to detect autistic children at an early stage. Brain Sci 11(6):734

    Article  Google Scholar 

  35. Alam MS, Rashid MM, Roy R, Faizabadi AR, Gupta KD, Ahsan MM (2022) Empirical study of autism spectrum disorder diagnosis using facial images by improved transfer learning approach. Bioengineering 9:710. https://doi.org/10.3390/bioengineering9110710

    Article  Google Scholar 

  36. Khosla Y, Ramachandra P and Chaitra N (2021) Detection of Autistic Individuals Using Facial Images and Deep Learning. In: 2021 IEEE International Conference on Computation System and Information Technology for Sustainable Solutions (CSITSS), Bangalore, India, pp 1–5

  37. Tufail AB, Anwar N, Othman MTB, Ullah I, Khan RA, Ma Y-K, Adhikari D, Rehman AU, Shafiq M, Hamam H (2022). Early-Stage Alzheimer’s Disease Categorization Using PET Neuroimaging Modality and Convolutional Neural Networks in the 2D and 3D Domains. In: Sensors (Vol 22, Issue 12, p 4609). MDPI AG. https://doi.org/10.3390/s22124609

  38. Kaur G, Sinha R, Tiwari PK, Yadav SK, Pandey P, Raj R, Vashisth A, Rakhra M (2021) Face mask recognition system using CNN model. Neurosci Inform 2:100035

    Article  Google Scholar 

Download references

Acknowledgements

The authors extend their appreciation to the Deanship of Scientific Research at King Khalid University for funding this work through large group Research Project under Grant number RGP2/249/44.

Funding

Not applicable.

Author information

Authors and Affiliations

  1. School of Mechanical and Manufacturing Engineering, National University of Sciences and Technology (NUST), Islamabad, 44000, Pakistan

    Ayesha Kanwal, Kashif Javed & Sara Ali

  2. Department of Computer Engineering, College of Computing and Informatics, University of Sharjah, Sharjah, 27272, UAE

    Saddaf Rubab

  3. Department of Computer Science, HITEC University, Taxila, Pakistan

    Muhammad Attique Khan

  4. College of Computer Science, King Khalid University, 61413, Abha, Saudi Arabia

    Areej Alasiry & Mehrez Marzougui

  5. Model Institute of Engineering and Technology, Jammu, Jammu and Kashmir, India

    Mohammad Shabaz

  6. Department of Computer Science and Mathematics, Lebanese American University, Beirut, Lebanon

    Muhammad Attique Khan

Authors
  1. Ayesha Kanwal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kashif Javed
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sara Ali
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Saddaf Rubab
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Muhammad Attique Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Areej Alasiry
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mehrez Marzougui
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Mohammad Shabaz
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

A.K., K.J., and S.A. wrote the main manuscript text and provided conceptualization, methodology, and software. S.R., M.A.K., and A.A. prepared figures and provided data curation. M.M. and M.S. provided validation and investigation and edited the manuscript. All authors reviewed the manuscript.

Corresponding author

Correspondence to Mohammad Shabaz.

Ethics declarations

Competing interests

The authors declare no competing interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kanwal, A., Javed, K., Ali, S. et al. A hybrid framework for detection of autism using ConvNeXt-T and embedding clusters. J Supercomput 80, 8156–8178 (2024). https://doi.org/10.1007/s11227-023-05761-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-023-05761-8

Keywords

Search

Navigation