Machine Learning Prediction of Autism Spectrum Disorder From a Minimal Set of Medical and Background Information

Key Points Question Can a machine learning (ML) model based on minimal background and medical information accurately predict autism spectrum disorder (ASD)? Findings This diagnostic study of 30 660 participants using ML prediction of ASD with only 28 features found high predictive accuracy, sensitivity, and specificity. Validation on independent cohorts showed good generalizability, and developmental milestones and eating behavior emerged as important predictive factors. Meaning The model developed in this study shows promise in the early identification of individuals with an elevated likelihood of ASD, using minimal information, which could affect early diagnosis and intervention strategies.


Data pre-processing
The following data pre-processing and individual selection measures were done.In the medical screening data, the presence of 'null' for a variable indicates the absence of a condition.Therefore, we replaced the 'null' values with 0. The samples with a missing value for any variables were removed from the final set before model training.We converted all categorical predictor variables into one-hot encoding vectors, while numerical values are standardized to zero mean and unit variance (see Table S2).
We utilized data augmentation methods to handle the sample size imbalance between autistic and non-autistic samples but did not find significant performance improvements compared to a down-sampling method.Therefore, we employed a down-sampling strategy by choosing an equal number of non-autistic participants corresponding to the autistic participant sample size at random.
The predictor variable values were standardized between the SPARK and SSC cohorts prior to the model testing.There were two features -"Age in months when first combined words into short phrases or sentences with an action word" and "Is child/dependent a twin or part of a multiple birth?"for which there were no equivalent mappings in the SSC database.Therefore, we coded these features as unavailable and used the remaining features for validation.The variable mapping and their value standardization are provided in Table S2.

Model Training Information
Data splits for each fold are done using the same random_state = 1 for reproducibility across different runs of experiments.The scikit-learn ColumnTransformer is used with Standard Scaler and one-hot encoding for standardizing the numerical features and encoding categorical features respectively.Note that the transforms are applied using the ColumnTransformer module with standard_scaler and onehot_encoder.The scaler was fit to the train data only, and the resulting scaler was used to transform the test data.The value for the gest_age measure in the SPARK v8 cohort is available only if preterm birth is selected.Therefore, for those samples that are not preterm, this field is blank.We replaced it with 40 weeks before model training.Bayesian optimization is used for hyperparameter tuning for each fold during the model training.The hyperparameters tuned for different algorithms are given below.

Additional model development exercises
We conducted two additional model development exercises to test the effectiveness of individual medical screening and background history items.We did similar model training with the same dataset but only using the items from medical screening or background history.Additionally, we conducted a sensitivity analysis for the "sex" predictor variable in our cohort.We did identical model training and evaluation procedures without the "sex" variable using the SPARK v8 cohort and validated its generalizability using the SPARK v10 and SSC cohorts.
To study the sensitivity of model performance and to determine discriminating predictors across different age groups at their evaluation, we analyzed the model performance with participants at 0 -2 years, 2 -4 years, and 4 -10 years, race, and sex.
eMethods.Data Preprocessing and Model Training and Development eTable 1. List of Predictors From the SPARK Version 8 Cohort Used for Model Development and Validation eTable 2. Performance of ML Algorithms eTable 3. Performance of the XGBoost Algorithm AutMedAI Stratified by Age eTable 4. Performance of the AutMedAI Assessed Using the SPARK Version 10 Cohort eTable 5. Performance of AutMedAI Using the SSC Cohort With 2854 Individuals With ASD eTable 6. Differences in Quantitative Measures for Those in the ASD Group (Correct vs Wrong Prediction) and Those in the Non-ASD Group (Correct vs Wrong Prediction) eTable 7.Frequency and Odds Ratio for Different Behavioral and Developmental Diagnoses Between Those in the ASD Group (Correct vs Wrong Prediction) and Those in the Non-ASD Group (Correct vs Wrong Prediction) eFigure 1. Overview of the Study and the ML Model Development eFigure 2. Influencing Predictors for Autism Detection in the Group Aged 0 to 2 Years eFigure 3. A Beeswarm Plot for All Test Set Samples in the SPARK Version 10 Cohort Aged 2 to 4 Years eFigure 4. A Beeswarm Plot for all Test Set Samples in the SPARK Version 10 Cohort Aged 4 to 10 Years eFigure 5.A Beeswarm Plot for Test Set Samples of the SPARK Version 10 Cohort eFigure 6.The Influence of the Top 20 Features for 6 Individuals in the SPARK Version 10 Cohort eFigure 7. Decision Plot Showing the Influence of Features on Model Prediction This supplementary material has been provided by the authors to give readers additional information about their work.

eFigure 1 .
Overview of the Study and the ML Model Development A flow diagram to illustrate the pipeline and approach, including the participant selection process prior to model training.eFigure 2. Influencing Predictors for Autism Detection in the Group Aged 0 to 2 Years A beeswarm plot for all test set samples in the age group of 0 -2 years of the SPARK v10 cohort shows SHAP values for features.The x-axis represents the features' contribution to the final model prediction.A positive or negative value is associated with an increase in the likelihood of predicting autism or non-autism, respectively.Every point in the plot represents a test sample.The color gradient indicates the range of feature values.eFigure 3. A Beeswarm Plot for All Test Set Samples in the SPARK Version 10 Cohort Aged 2 to 4 Years A beeswarm plot for all test set samples in the 2 -4 years of SPARK v10 cohort shows SHAP values for features.The x-axis represents the features' contribution to the final model prediction.The positive or negative value is associated with an increase in the likelihood of predicting autism or non-autism, respectively.Every point in the plot represents a test sample.The color gradient indicates the range of feature values.Cohort Aged 4 to 10 Years A Beeswarm plot for all test set samples in the age group of 4 -10 years of the SPARK v10 cohort showing SHAP values for features.The x-axis represents features' contribution to the final model prediction.A positive or negative value is associated with an increase in the likelihood of predicting autism or non-autism, respectively.Every point in the plot represents a test sample.The color gradient indicates the range of feature values.eFigure 5.A Beeswarm Plot for Test Set Samples of the SPARK Version 10 Cohort A Beeswarm plot for test set samples of the SPARK v10 cohort showing Shapley additive explanations (SHAP) values for features.The x-axis represents the features' contribution to the final model prediction.The positive or negative value is associated with an increase in the likelihood of predicting autism or non-autism, respectively.Every point in the plot represents a single participant.The color gradient indicates the range of feature values.eFigure 6.The Influence of the Top 20 Features for 6 Individuals in the SPARK Version C) Figures represent these features and their influence on three individuals diagnosed with autism, and the proposed XGBoost "AutMedAI" ML model prediction was correct.(D -F) Figures represent these features and their influence in three individuals diagnosed with autism, and the proposed XGBoost "AutMedAI" ML model was incorrectly predicted as non-Autism.The x-axis represents SHAP values, and the y-axis lists the top 20 features.The red colored bar against each feature indicates prediction towards Autism, with SHAP values written inside the bar indicating the extent of influence towards Autism prediction.The blue bar with negative values indicates the features influencing the reverse direction, i.e., towards non-autism prediction.The grey values before the feature names are the actual feature values for an individual.

Features Minimal Background Information without sex at birth Logistic Regression
2024 Rajagopalan SS et al.JAMA Network Open. 4 eTable 1. List of Predictors From the SPARK Version 8 Cohort Used for Model Development and Validation A list of predictors from the SPARK v8 cohort used for model development and validation, and information on these variables is also provided in Simons Simplex Cohort.The mean and the range of values for the sample set used for model development and validation are shown for all numerical variables.Performance of ML Algorithms The performance of ML algorithms uses a different feature selection and tests the effect of the sex at birth variable as a predictor.The AUC σ value indicates the standard deviation of AUC values across 10-fold cross-validation using the SPARK v8 cohort.Performance of the XGBoost Algorithm AutMedAI Stratified by Age The AUC σ value indicates the standard deviation of AUC values across 10-fold crossvalidation using the SPARK v8 cohort.Performance of the AutMedAI Assessed Using the SPARK Version 10 Cohort (N=11,936, Autism=10,476, Non-Autism=1,460) and stratified by age, race, and sex.Performance of AutMedAI Using the SSC Cohort With 2854 Individuals With ASD no non-autistic data available.Differences in Quantitative Measures for Those in the ASD Group (Correct vs Wrong Prediction) and Those in the Non-ASD Group (Correct vs Wrong Prediction) The results are visualized in Figure 2B.Frequency and Odds Ratio for Different Behavioral and Developmental Diagnoses Between Those Those in the ASD Group (Correct vs Wrong Prediction) and Those in the Non-ASD Group (Correct vs Wrong Prediction) © 2024 Rajagopalan SS et al.JAMA Network Open. 5 © 2024 Rajagopalan SS et al.JAMA Network Open. 10 eTable 7.